GitHub integration

Pipemesh connects to GitHub through the Pipemesh GitHub App. Install it and you're connected: no personal token, no webhooks to set up. Getting started shows how.

GitLab and self-hosted repositories are coming soon.

What the App can do

It reads a lot and writes very little:

Permission Access Used for
Metadata Read finding your repositories
Contents Read commits, files, diffs; never pushes
Commit statuses Read/write held for plain statuses; results report as check runs
Checks Read/write native check runs on pull requests
Pull requests Read pull request workflows; the pull request a merge commit links
Actions Read/write starting and cancelling jobs that run on GitHub Actions (see The definition)

The App can't push code or edit workflow files. Pipemesh reads with short-lived installation tokens, and only writes check runs and Actions dispatches.

When the App asks for a new permission, each organization must approve it on GitHub (a review request on the installation). Until then, its jobs that run on GitHub Actions fail with a message saying so.

Webhooks

The App sends pushes, pull requests, check runs and workflow runs as they happen. A push starts its pipeline in seconds, and a job on GitHub Actions settles the moment its run finishes.

Deliveries are signed (X-Hub-Signature-256) and backed by polling, so a lost one costs a poll interval, never work.

Merged pull requests get linked to the revision or workflow run at the merge's commit (merge, squash or last rebased commit), in the revision panel and a job's Revision and Run tabs. Only the webhook makes this link, so a lost delivery means none.

Checks on pull requests

  • One check per job and commit, named pipemesh/<job>. A child carries its path: pipemesh/build/commit_lint.
  • Native check runs, with skipped among the conclusions, so branch protection can require a job that may skip.
  • Useful failures: the summary shows the failing task's log excerpt, and compiler output (javac, tsc) becomes file and line annotations on the diff.
  • Re-run works: GitHub's Re-run on a failed check retries the job.

Pull request workflows

Workflows with on: pull_request run once per pull request head, on the head commit: the branch as pushed, never a hypothetical merge. Results show up on the pull request within seconds.

  • Builds (skip: built) reuse the outputs of an earlier successful run with the same fingerprint: trusted runs first, then this pull request's own, never another's.
  • Tasks (skip: never) always run.
  • skip: unchanged is for pipelines; a workflow with it doesn't load.

Jobs have full history, so a task can compute the merge base itself:

git merge-base "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" HEAD

GitHub merge queue

Mark Pipemesh checks as required and turn on the merge queue. That's all. Each group runs your on: pull_request workflows once on the queue's commit and reports the checks the queue waits for.

  • Which workflows: those whose targets: include the queue's base branch, or that have no targets:.
  • Build reuse: the queue's commit holds the pull requests ahead in the group too, so a build reuses only if what it checks out there matches an earlier successful run.
  • Secrets and caches: as for a pull request, since the commit isn't on the base branch yet.
  • Spotting a queue run: CI_MERGE_REQUEST_EVENT_TYPE=merge_train, a gh-readonly-queue/… ref, and a title starting with merge queue:.

Job identity

A job proves who it is with a short-lived OIDC token from Pipemesh's own issuer. Cloud accounts and other services trust it directly: no stored keys, nothing to rotate.

Request a token

On an instance with a public URL, like pipemesh.io, every job gets:

Variable What it is
PIPEMESH_ID_TOKEN_REQUEST_TOKEN The job's credential for requesting tokens, masked in logs. Only Pipemesh accepts it and no other party sees it, so one party's token can't be turned into another's.
PIPEMESH_ID_TOKEN_REQUEST_URL https://<instance>/api/oidc/token

Request one token per party, with that party's audience:

token=$(curl -fsS -X POST \
  -H "Authorization: Bearer $PIPEMESH_ID_TOKEN_REQUEST_TOKEN" \
  --get --data-urlencode "audience=sts.amazonaws.com" \
  "$PIPEMESH_ID_TOKEN_REQUEST_URL")

The response is the token, valid until two hours after the job started:

Claim Value
iss https://<instance>/api/oidc (discovery and keys under it)
sub the job's subject (copy it)
aud the audience the job requested
context what started the run: pipeline, push, schedule, manual, tag or pull_request
pipeline, pipeline_id, job, ref, sha, tenant the pipeline or workflow's alias and id, the job, ref, commit and organization
repository_id, repository_owner_id, job_id GitHub's ids for the repository and its owner, and the job's own id (renaming the job with was: keeps it)

Trust it on the other side

The party you connect to (a cloud account, a cache, a registry) needs:

  • Issuer and subject: copy them from the repository's Settings → Job identity, or print them with npx pipemesh identity. Never write a subject by hand.
  • Audience: what your job requests. Use the party's own, such as sts.amazonaws.com for AWS.

Each job has its own subject: trust only the jobs that need access. Where patterns work (AWS StringLike), Settings also gives one for every job of the repository.

Never trust the pull_request context for anything that can write: anyone who can open a pull request runs code in it. The subjects in Settings are for the repository's branch, so a pull request's jobs never match them.

Components that do it for you

  • aws/role@1 (a setup: step) requests an sts.amazonaws.com token and exports the AWS SDK's web-identity environment, so every AWS call after it assumes the role.
  • vercel/turborepo-token@1 exchanges a https://vercel.com/<team> token at Vercel for a Turborepo token. See Turborepo monorepo.

Renames and transfers

Pipemesh follows a repository by the id GitHub keeps through renames and transfers. There's nothing to do.

  • Renamed repository or organization: pipelines, aliases and settings follow within seconds. Old URLs and aliases work while GitHub redirects.
  • Transferred to a connected organization (with the repository in its installation): the pipeline moves with its history and the repository's own variables and secrets. The old organization keeps its own settings and past usage.
  • Transferred where Pipemesh can't read it: the pipeline stops polling and its page says where it went, until you connect that organization with the repository.
  • Caches and job images start cold, as they're keyed by the repository's path.
  • Job identity subjects use ids, so renaming the repository, its organization or a job (with was:) doesn't change them. Trusts follow a transferred repository to its new owner; remove any that shouldn't.

A missed webhook only delays a rename: Pipemesh also checks on its own, at least hourly.