Bring your own runners

Run jobs on your own machines, such as a Mac mini for code signing, a GPU box or a fleet inside your VPC. Pipemesh stays hosted, and those jobs' code and secrets never touch hosted compute.

A runner is a name your jobs give, plus the machines that joined it. They run the open-source gitlab-runner agent, as-is. It only talks to Pipemesh, and your repositories stay on GitHub.

1. Create a runner

Organization owners create it in Settings → Organization → Runners, or with the CLI from inside one of the organization's repositories:

pipemesh runners create build-farm

Names use lower-case letters, digits, ., _ and -. Names starting linux-, macos- or windows- are reserved for hosted runners. The join command, with its registration token, is shown once. Add a machine (or pipemesh runners token build-farm) makes another.

Pull request jobs are off by default. They never run on your runner's machines, because on a public repository a pull request's code can come from anyone's fork. Turn them on for a runner whose repositories are private, or whose machines you can afford to expose.

2. Join machines

Run the join command on each machine, with the executor of your choice:

gitlab-runner register \
  --non-interactive \
  --url https://pipemesh.io \
  --registration-token pmr_XXXXXXXX_YYYYYYYY \
  --executor shell \
  --description "build-farm-$(hostname)"

Each machine gets its own credential and shows up under the runner, online if it asked for work in the last two minutes. Any number of machines can join.

3. Run jobs on it

sign_app:
  job_type: build      # signs what it consumes; checks out only its script
  stage: release
  runner: mac-signing  # runs only on the machines of the runner "mac-signing"
  consumes:
    - build/app
  checkout:
    - scripts/sign.sh
  script:
    - ./scripts/sign.sh
  produces:
    signed: dist/app-signed.zip

The job runs only on the runner's machines, and only for your organization's repositories. It never falls back to hosted compute, and your runners aren't metered.

  • No machine can take it? It waits, and its card says why: no runner by that name, no machine online, or the runner doesn't take pull request jobs.
  • A machine goes silent mid-job? The job fails at its timeout_seconds (one hour when absent).

What a job gets

The same as on hosted runners: its image: (with the Docker and Kubernetes executors), a workspace with exactly what its checkout: lists and the full history, its secrets as masked variables, and its artifacts:, cache: and publish:. A bootstrap sets this up as the first script line, so the image needs bash, git, curl and tar.

Remove machines and runners

  • A machine: run gitlab-runner unregister on it, or remove it from the runner. Its credential stops working at once.
  • A registration token: revoke it to stop new machines joining with it. Machines that joined keep working.
  • A runner: delete it (or pipemesh runners delete build-farm) to remove its machines and revoke its tokens. Jobs that name it wait.

GitHub Actions

Give a job github_actions: in place of script:, and an Actions workflow runs it on any runner Actions accepts, GitHub-hosted or self-hosted. The job settles on the run's verdict and keeps its job type, so a deploy is still a deploy. Release stages, gates and promotion stay in Pipemesh (see The definition).

GitHub's own runner (actions/runner) speaks GitHub's proprietary broker protocol, so it can't connect to Pipemesh today. Use gitlab-runner, and Pipemesh handles the checkout credentials. Native support is being assessed.