# bluegit-runner — separate CI runner

`bluegit-runner` runs BlueGit CI jobs on your own machine, in Docker, just
like the built-in runner. It only makes **outgoing** HTTPS connections to
BlueGit (long-poll); no port has to be opened. The built-in BlueGit runner
stays available.

## Network

- HTTPS (port 443) to the BlueGit server for heartbeat, jobs, logs and artifacts;
- internet access for whatever the jobs fetch (Docker images, packages).

**No inbound ports** are needed on the runner machine, also not behind NAT
or a home router. A proxy works through `HTTPS_PROXY` in the service
environment. The runner's own web interface listens on `127.0.0.1` only by default.

## Install (Arch / AUR)

```sh
yay -S bluegit-runner
```

The package creates the system user `bluegit-runner` (member of `docker`),
installs and enables `bluegit-runner.service`. The service starts once
`/etc/bluegit-runner/config.yml` exists; an upgrade restarts it.

## Install elsewhere (binary + systemd)

```sh
git clone https://git.timfalken.com/timfa/bluegit.git && cd bluegit
go build -trimpath -ldflags "-X main.version=$(date +%Y%m%d%H%M%S)" -o bluegit-runner ./cmd/bluegit-runner
sudo install -Dm755 bluegit-runner /usr/bin/bluegit-runner
sudo install -Dm644 packaging/runner/bluegit-runner.service /usr/lib/systemd/system/bluegit-runner.service
sudo install -Dm644 packaging/runner/sysusers.conf /usr/lib/sysusers.d/bluegit-runner.conf
sudo install -Dm644 packaging/runner/tmpfiles.conf /usr/lib/tmpfiles.d/bluegit-runner.conf
sudo systemd-sysusers && sudo systemd-tmpfiles --create
sudo usermod -aG docker bluegit-runner
sudo systemctl daemon-reload && sudo systemctl enable bluegit-runner
```

Without systemd: `bluegit-runner run --config path/to/config.yml` as a user that may use Docker.

## Register

- **1.** In BlueGit: **Settings → Runners → New registration token** (account runner) or **Group → Settings → Runners** (group runner, maintainers). The token (`bgrr_…`) is valid 24 hours, single use, shown once.
- **2.** On the machine: `sudo bluegit-runner register --url https://git.example.com --token bgrr_... [--name my-pc]`. The runner gets its own token (`bgr_…`); the server stores only its hash. `register` starts the service and prints how to open the web interface.

## Config

`/etc/bluegit-runner/config.yml`: `url`, `token`, `name`, `docker`, `git`,
`work_dir` (`/var/lib/bluegit-runner`), `network` (`bridge`), `concurrency`
(start value), `ui_listen` (`127.0.0.1:8765`, `off` disables the UI) and
`language` (`en`, `nl`, `de`, `fr`). Settings changed in the web interface
are stored in `work_dir/settings.json`.

## Web interface

Every runner has its own admin screen in BlueGit style, in English, Dutch,
German and French.

- **Address**: `http://127.0.0.1:8765` by default. Change `ui_listen` or use an SSH tunnel: `ssh -L 8765:127.0.0.1:8765 runner-host`.
- **Access**: an admin token (`bgru_…`) created locally on first start in `work_dir/ui-token` (mode 0600). It is never logged; show it on the machine with `sudo bluegit-runner ui-token`.
- **Contents**: status (online, version, server, last contact), hardware with ★ recommendations, active and recent jobs, a *safe to upgrade* indicator, **Pause (drain)** / **Resume**, all settings, image presets and a docs page at `/docs`.

| setting | default | |
|---|---|---|
| name | hostname | shown in BlueGit and job logs; synced to the server on save and every heartbeat |
| labels | — | your own labels |
| jobs at the same time | `concurrency` | 1–64, live |
| maximum cached images | 20 | least recently used images above this are removed |
| maximum job duration | — | caps the pipeline timeout |
| CPUs / memory / processes per job | no limit | `docker run --cpus`, `--memory`, `--pids-limit` |
| disk budget | no limit | images, checkouts and cache together |
| pull policy | pull when missing | `always` or `never`; presets are always built locally |
| remove work dir after job | on | off keeps `work_dir/work/<job>` for debugging |
| remove dangling images | on | `docker image prune -f` after each job |
| maximum job cache | 10 GB | oldest cache directories first |

Recommendations assume ~2 cores and ~2 GB per job, keep 1 core and 1 GB
for the host and ~10 GB disk per job.

**Image presets** work like on Steve, the built-in runner: `image:
bluegit/node` is a Dockerfile built locally on first use and reused while it
stays the same. Add your own as a fixed image, a Dockerfile (optionally with
another `FROM`), or only another base image for a built-in preset.

**Pause (drain)** lets running jobs finish and picks up nothing new. The
heartbeat reports `draining`; BlueGit shows *paused on the runner* (API
`status: "draining"`) and routes jobs to the next runner or keeps them queued.

**Local API** (Bearer admin token): `GET /api/status`, `GET/PUT /api/settings`,
`GET /api/jobs`, `POST /api/pause`, `POST /api/resume`.

```sh
T=$(sudo cat /var/lib/bluegit-runner/ui-token); U=http://127.0.0.1:8765
curl -fsS -X POST -H "Authorization: Bearer $T" $U/api/pause
until curl -fsS -H "Authorization: Bearer $T" $U/api/status | grep -q '"safe_to_upgrade":true'; do sleep 10; done
sudo pacman -Syu bluegit-runner
curl -fsS -X POST -H "Authorization: Bearer $T" $U/api/resume
```

## Built-in runner

Site administrators get the same screen for the runner inside BlueGit at
**/admin/runner**, and the API `/api/v1/admin/runner/{status,settings,jobs,pause,resume}`
(admin token, scope `runners` or `api`; others get `403`). Settings live in
`runner-settings.json` in the state directory. The self-update only waits for
jobs that are actually *running*, so a paused built-in runner does not block
an update; queued jobs start after resuming.

## Git and API from a job (`CI_JOB_TOKEN`)

A job can reach its own repo without its own login, also when the repo is private. `origin` points to `CI_PROJECT_URL.git`, and the runner sets `http.<project url>.extraHeader` (`Authorization: Bearer $CI_JOB_TOKEN`) through `GIT_CONFIG_*` environment variables, only for this project's URL. No file is written, so it works in any image (including `bluegit/base`) with git ≥ 2.31. `git ls-remote origin`, `git fetch` and `git clone` work, as do `GET` API routes for the own repo with the job token (read-only). Pushing, other repos and any use after the job ends are refused.

## Security and isolation

- Every job runs in its own Docker container with the checkout on `/workspace`; services get a network per job. Per-job limits are set in the web interface.
- Docker access is root-equivalent on that machine; account runners only get jobs of their owner's repositories, group runners only of their group.
- The runner token is only in `config.yml` (0600); the server keeps a hash. It may only clone the repository of a job the runner is running right now.
- Job secrets are masked in logs. The service runs with `NoNewPrivileges`, `ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`.

## Online, offline and lost runners

Heartbeat every **30 s**; **online** means contact in the last **90 s**.
If a runner disappears **during a job**, BlueGit fails the job with
`runner_system_failure` after **5 minutes** without contact; retry rules apply.

## Remove or replace

Revoke the runner in BlueGit (the token stops working immediately), then
`sudo systemctl disable --now bluegit-runner` and remove the package. To
replace a machine: register the new one, revoke the old one, and copy
`settings.json` or use `PUT /api/settings`.

## Troubleshooting

| Symptom | Cause and fix |
|---|---|
| stays *offline* | `journalctl -u bluegit-runner -f`; test `curl -I https://<server>/healthz` (DNS, proxy, outgoing 443). |
| `401` from the server | runner revoked: register again. |
| `registration failed (403)` | registration token expired or used. |
| `runner_system_failure: docker … is not available` | `sudo -u bluegit-runner docker info` must work (group `docker`, `docker.service`). |
| job stays *queued* | runner paused (locally or in BlueGit), all slots busy, or the runner does not serve this repository. |
| disk full | lower max images and job cache, enable dangling-image cleanup. |
| web interface does not open | check `ui_listen` and `ss -ltnp`; lost token: `sudo bluegit-runner ui-token`. |

Runner management through the BlueGit API for agents: [agents.md](agents.md#runners).
