The definition (pipemesh.yaml)

pipemesh.yaml sits at the root of your repository. Its pipemesh: key names every workload the repository runs, as a pipeline or a workflow. Every other root key is yours (a job, a body, helpers, an included file), and pipemesh: reaches it with !ref.

builds: !include .pipemesh/build.yaml

deploy_staging:
  type: job
  job_type: deploy
  production: false
  stage: staging
  consumes:
    APP_IMAGE: build/image/app     # $APP_IMAGE is its ref@digest
  checkout:
    - scripts
    - charts
  script: |
    ./scripts/deploy.sh staging "$APP_IMAGE"
    echo "deployed $CI_COMMIT_SHORT_SHA"

deployment:
  type: pipeline
  stages:
    - build
    - staging
  jobs:
    build:
      job_type: workflow
      stage: build
      body: !ref builds
    deploy_staging: !ref deploy_staging

pipemesh:
  pipelines:
    pipeline: !ref deployment
  workflows:
    checks:
      body: !include .pipemesh/ci.yaml
      on: pull_request
    nightly:
      body: !include .pipemesh/nightly.yaml
      on: schedule
      cron: "0 3 * * *"

Three layers

1. Bodies say what runs: a type: (workflow or pipeline), stages: and jobs:. Write one at the root, in an included file, or inline.
2. Registration names a workload under pipemesh.pipelines or pipemesh.workflows, and the body's type: must match. The entry is the body itself, or a map with it under body:, beside inputs: and triggers.
3. Triggers say when a workflow runs. Pipelines take none: every commit on the default branch is a revision.

• Bodies are whole: nothing extends or merges. Reuse a job with a type: job definition and !ref; a job that varies is a component. Triggers live on the registration, never in a body, so a body can back a registration and a job alike.
• A workload is active once named, with a URL, a history and a run counter.
• Names are unique across both sections and match [a-z0-9][a-z0-9_-]*. Reserved: settings, jobs, runs, events, workflows, pipelines, repository, and pipeline for a workflow (it's the pipeline's slot).
• Every root key must be reached from pipemesh:, directly or through another definition. An unreached key is a load error naming it, which catches typos.

Triggers

pipemesh:
  workflows:
    release:
      body: !include .pipemesh/release.yaml
      inputs:
        channel:
          default: stable
      triggers:
        stable:
          on: tag
          tags:
            - "v*"
        rc:
          on: tag
          tags:
            - "rc-*"
          inputs:
            channel: rc
        nightly:
          on: schedule
          cron: "0 3 * * *"
          inputs:
            channel: nightly
    checks:
      body: !include .pipemesh/ci.yaml
      on: pull_request
    smoke:
      body: !include .pipemesh/smoke.yaml   # no triggers: run it from its page

• push fires on each new commit of the repository's branch: the repository's default branch unless its settings name another. Other branches start nothing, so branches: can only repeat that name, and any other never fires.
• on: or triggers:, not both. on: names the trigger after its event, triggers: after its key. The name filters the run list (?trigger=tag:rc) and fires the trigger by API.
• A workload's triggers share its history and run counter. For one event to start several workloads, declare it on each.
• inputs: on the entry declares names and defaults. Triggers supply values, frozen into each run's variables; an undeclared input is a load error. Same history, different values: a trigger with inputs. Separate history: a second entry sharing the body.
• No triggers means manual only: listed in the catalog, run from its page or the API.

Types

type: is a structure's shape (job, workflow, pipeline or component), checked where it's used. A !ref to the wrong definition is a load error.

• Required where a structure stands alone: a root definition, a job or body file brought in with !include, a job a !ref picks out of another definition (!ref ci.jobs.lint), every component.
• Optional, but checked, inline: a job under jobs:, a body as body:, an entry under pipemesh:.
• None on plain data: a cache: declaration in its own file, a list of script lines, a map of helpers.

job_type: is what a job is, and must agree with its body's type:. Here they disagree:

orders:
  job_type: workflow                      # runs a workflow and waits for it
  stage: dispatch
  body: !include .pipemesh/service.yaml   # a file that says type: pipeline

That's a load error naming both sides:

pipemesh.pipelines.pipeline: job orders is job_type: workflow, so its
body must be type: workflow; the body is type: pipeline — to hand
revisions to it, write job_type: pipeline

Job types

job_type: says what a job is and where it may appear. It decides when the job runs, and sets its checkout default.

pipeline:
  type: pipeline
  stages:
    - build
    - staging
    - production
  jobs:
    build:
      job_type: build                # checkout: true; reuses an earlier pass
      stage: build
      script: npm ci && npm test && npm run build
      produces:
        dist: dist
    deploy_staging:
      job_type: deploy               # checkout: false; skips what it already shipped
      production: false
      stage: staging
      consumes:
        DIST: build/dist
      checkout:                      # and the deploy scripts
        - deploy
      script: deploy/deploy.sh staging
    smoke:
      job_type: task                 # runs on every revision
      stage: staging
      needs:
        - deploy_staging
      checkout:
        - smoke.sh
      script: ./smoke.sh https://staging.example.com
    deploy_prod:
      job_type: deploy
      production: true
      stage: production
      needs:
        - smoke
      consumes:
        DIST: build/dist
      github_actions:
        workflow: deploy.yml
        inputs:
          environment: production

Every job says its job_type:; a job without one doesn't load.

• build is hermetic: an earlier run with the same fingerprint stands in for it, shown as reused (its outputs, or its pass). Tests, lint, image builds and build-graph fingerprint jobs (nx/fingerprint@1, …) are builds, as is a job that signs or packages what it consumes, with checkout: false.
• deploy ships what it consumes to an environment, when that changed since its last success. Deploys belong to pipelines, which alone have a last success, each environment's revision, rollback and holds. A pull-request preview or manual hotfix deploy is a task in a workflow, not tracked as a deployment.
• Every deploy says production: true or production: false. Production deploys are what DORA metrics, pipemesh deployments and pipemesh deployed count, wherever they sit. Matrix variants share it, and it doesn't change how the job runs.
• task runs on every revision that reaches it: smoke tests, notifications, checks that read pull-request context such as a merge-base diff.
• workflow and pipeline run no script: they hand the revision to a body.

The job type decides when a job runs; nothing on the job changes it. A job that must run every time is a task. skip: is deprecated: a job that still writes it loads, the key is ignored, and its Rules tab and pipemesh check say so. Set checkout: on the job to name what it reads. The job's Rules tab shows its job type, its effective checkout (defaults: when it came from the job type), when it skips, and what Pipemesh added to the checkout (added:). job_type: belongs to the job, never a component, so one component can be a build here and a task there.

A build, deploy or task names exactly one executor:

What a job checks out

checkout: true        # the whole repository
checkout: false       # nothing
checkout:             # exactly these
  - services/orders
  - libs/money
  - package.json
checkout:             # everything but these
  - "**"
  - "!docs"
  - "!README.md"

• A path covers itself and everything under it (services/api, go.mod); globs cut finer (services/*/go.mod, **/Dockerfile).
• Paths are anchored at the repository root: for a job with repo:, that repository's.
• Prefer whole directories, and exclusions to long lists. An entry starting with ! excludes. The list is read as git reads a non-cone sparse checkout: each file follows the most specific path an entry matches (the file, then its directory, then the one above), and among entries on the same path, the last one decides. ["**", "!docs"] is everything but docs/; ["**", "!docs", "docs/api"] keeps docs/api; ["**", "!**/*.md"] drops every Markdown file.
• Quote exclusions (- "!docs"): a bare ! starts a YAML tag.
• The workspace holds exactly the checkout, on hosted runners and your own, so a job can't read a file its fingerprint leaves out. List the root files it reads too: package.json, lockfiles, tsconfig.json, .nvmrc, wrapper scripts, tool pins.
• One commit deep: the clone holds the job's commit only, even in a job that checks out nothing. A job that needs history fetches it (pull request workflows).
• Pipemesh adds what it reads itself: each file a cache key hashes (${checksum:package-lock.json}), and each image's build context and Dockerfile. They join the checkout and fingerprint, under added: in the Rules tab.
• Load errors: an empty list or ["**"] alone (write false or true), "**" with no exclusion beside it, only exclusions.
• On GitHub Actions, the checkout comes from the workflow file.

When a job runs

A job runs when its fingerprint is new. The fingerprint hashes the job's definition and parameters, the digests of what it consumes, the files it checks out (in its repository and each mount), and the versions of its secrets and Settings variables. The job type says which earlier fingerprint lets it skip.

• After a failed or cancelled run, a deploy runs in full until a run succeeds: a half-finished deploy leaves the service in an unknown state. A successful rollback counts as a success.
• A build in a pull request reuses outputs from trusted runs and its own, never another pull request's. When a pull request merges, the passes of its last run become trusted: the next pull request that leaves a job's inputs alone reuses them, and so does a pipeline job defined the same way, which then ships what the pull request built (its card says "Reused pull request #N's build"; its outputs are kept as its own).
• reuse: trusted on a build keeps its pipeline runs to passes a run on the default branch made: it builds again rather than ship what a pull request built.

A build or deploy that reads nothing runs once. With no input (no checkout, consumes:, image_from:, repos:, secrets: or a !settings variable), its fingerprint is its definition alone. It loads, with a warning in its job panel. Say what it reads, or make it a task.

Release flow covers fingerprints and skips across a pipeline.

Jobs

Job keys will look familiar: job_type, stage, script, needs, runner, dockerd, image, image_from, variables, timeout_seconds, retry, allow_failure, secrets, id_tokens, cache, produces, consumes, reports, checkout. Names are lowercase letters, digits, _ and -.

• A job waits only for what it names: its needs:, and the producers of what it consumes:. stage: places it on the board but doesn't order the run, so a job that names nothing starts with the revision.
• A job that consumes promotes only a revision its producers built: it takes the outputs of the revision it promotes (in a workflow run, of that run), never an earlier one's. When build ran at 12, covering 10 and 11, and test stopped at 10, deploy waits and promotes 12; when build stopped at 10 and test covered up to 12, deploy promotes 10. A skip counts as built: the outputs it kept carry to every revision it covers. A producer with no output at that revision fails the consumer before its script.
• services: is accepted but starts nothing: no runner runs service containers. Start a database in your script, or in Docker with dockerd: true.

Where a job runs

• The same sizes run on x86-64 as linux-amd64-micro, -small, -medium and -large (Hosted runners). runner: can also name one of your organization's own runners.
• Micro is for jobs that do almost nothing (a notification, a tag): no Docker daemon, and a large checkout can outgrow its 512 MiB.
• dockerd: true adds a Docker daemon (Testcontainers, docker compose). An image Pipemesh builds (build:) implies it. A fork's pull request runs its Docker jobs on nodes apart from the organization's other jobs (What a fork's pull request gets).
• A hosted job's log ends with what it used: [runner] linux-arm64-small: peak memory 1.2 GiB of 6.5 GiB, CPU time 34 s.
• KUBERNETES_* variables are refused: they set a pod's resources, which is runner:'s job.

Variables and secrets

Every job gets its commit: CI_COMMIT_SHA, CI_COMMIT_SHORT_SHA, CI_COMMIT_REF_NAME, and CI_COMMIT_TAG on a tag. A pipeline job also gets PIPEMESH_REVISION (its revision), and a workflow job PIPEMESH_RUN_NUMBER (its run's number). A workflow a pipeline runs gets both: PIPEMESH_REVISION is the pipeline's revision there, so one number names a release across the pipeline and its workflows. They only grow, so they make release numbers (:r$PIPEMESH_REVISION).

Run numbers aren't in the fingerprint, and a reused build hands over what it built back then. Apply the number where you tag or release, not in what you build.

deploy_staging:
  job_type: deploy
  production: false
  checkout:
    - deploy
  variables:
    API_URL: !settings
    LOG_LEVEL: info
  secrets:
    - DB_PASSWORD
  script: deploy/deploy.sh staging

variables: names the variables the job gets. A plain value is the definition's; NAME: !settings reads the value set in the repository's Settings → Secrets and variables (or its organization's). secrets: names the repository secrets it gets. It gets no others, and a !settings variable or secret that isn't set fails the job before its script.

• One source per variable. A name is written in the definition or read from Settings, never both: !settings on a name the body or the run that started the job also sets is an error.
• Each revision pins its Settings. Every edit is a new version, and a job runs with the versions current when its revision started, never an edit made since.
• An edit starts a revision on the current commit, for each pipeline with a job that reads it (edits a few seconds apart share one). So staging gets a new value before production, and rolling back restores the old one.
• Variables aren't secret. Any job that declares one can print it, a pull request's included, and the repository's maintainers see its value in Settings, inherited from the organization or not. Anything sensitive is a secret: each secret names the runs that get it, and pull requests get none unless you allow it.

id_tokens: names OIDC tokens the job gets as masked variables, each for its audience (VAULT_ID_TOKEN: { aud: https://vault.example.com }), as GitLab CI declares them: see Job identity.

A job on a runner also gets CI_JOB_TOKEN, which is Pipemesh's job token, not GitLab's: it authenticates the runner protocol, the job's calls to Pipemesh and GET /api/v4/job, and it dies with the run. Services that checked GitLab's job token take an identity token instead.

Images

image: names a registry image. image: !var NAME takes it from one of the job's variables, so a deploy can run in an image pinned in Settings:

deploy_prod:
  job_type: deploy
  production: true
  variables:
    DEPLOY_IMAGE: !settings      # e.g. ghcr.io/acme/deployer@sha256:…
  image: !var DEPLOY_IMAGE
  script: deploy/deploy.sh prod

• A !settings image belongs to the revision, resolved when the revision is created, against the Settings it pins. Changing the variable starts a revision in the new image; rolling back restores the old one. The revision's details show it, and the log says [image] DEPLOY_IMAGE = ….
• An unset variable, or one that isn't an image reference, stops the revision: a commit's definition doesn't load (like a missing !include file), and an edit's revision doesn't start. The repository says why until a commit or edit fixes it.
• A plain variable resolves when the definition loads, so the same value can name the image and reach the script.
• !var is the whole value, only image: takes it, and it names a variable, never a secret. pipemesh check can't read your Settings, so it checks the name only.

image_from: runs the job in an oci artifact's image, named by path and pinned by digest. It's consumed like any artifact (the job waits for the producer), and a child body reaches its parent's image with ../. image: and image_from: are exclusive.

An oci artifact with build: is an image Pipemesh builds from a directory and keeps for you in its registry, with no registry token to store:

build_ci:
  job_type: build
  stage: image
  checkout:
    - ci                       # runs only when ci/ changes
  script: echo "building the CI image"
  produces:
    ci_image:
      build: ci                # the directory; its Dockerfile is ci/Dockerfile
test:
  job_type: build
  stage: test
  image_from: build_ci/ci_image
  script:
    - make test

• build: is a directory, or {context, dockerfile, args}. The image is built after the script succeeds, so the script can prepare the directory. The directory joins the job's checkout, the job gets a Docker daemon, and the artifact is oci without saying so. It needs a hosted runner.
• It's tagged by its build context's fingerprint, so an image already built from the same content is reused, not rebuilt.
• An image anywhere else is your script's: build and push it with your own credentials, then record it with pipemesh produce oci <key> --ref <ref> --digest <digest> (an example).
• A pull request builds into a place of its own, which default-branch runs never read. A build reuses, so a pull request that doesn't touch the directory runs in the default branch's image.

Caches

webapp_build:
  job_type: build
  checkout:
    - webapp
  cache:
    key: npm-${checksum:webapp/package-lock.json}
    restore_keys:
      - npm-
    paths:
      - .npm
  script: cd webapp && npm ci --cache ../.npm && npm run build

cache: restores its paths: before the script and saves them under key: after a successful run.

• An entry never changes once saved, so put a checksum of what the paths depend on in the key.
• On a miss, restore_keys: prefixes are tried in order; the newest match wins.
• Trusted runs save; pull requests only read. A pull request restores its own workflow's entries first, then what any pipeline or workflow of the same definition saved for the same paths:. Every pull request can read those: keep credentials out of cached paths.
• An entry unused for 14 days is evicted, and a pipeline's entries together stay under 10 GB, the least recently used going first. Both are the instance's defaults; an admin can change them, for the instance or one organization.

Scripts and timeouts

script: is a block scalar: real multi-line shell, verbatim, with # comments allowed. Blank lines and trailing spaces are kept, so a heredoc writes exactly what it says.

script: |
  for f in a b; do
    echo "part $f"    # loops need no escaping
  done

• before_script:, script: and after_script: run as one script, in that order, in one shell that stops at the first failing command. So after_script: doesn't run after a failure, and a failing line in it fails the job.
• timeout_seconds: defaults to one hour (3600). At the limit the job is stopped and fails. A fork's pull request stops at six hours whatever it says.

Artifacts

• produces: artifacts reach only jobs that name them in consumes:, never one that only needs: the producer. Consuming orders the job after the producer and puts the artifact's digest in its fingerprint.

• consumes: is a map, variable to artifact: APP_IMAGE: build/image is the artifact image of the job build, and the script reads it as $APP_IMAGE. A file artifact's variable is its path in the workspace, where the file lands; an oci or npm artifact's is ref@digest, and the job pulls with its own credentials. Names follow the variable rules ([A-Z][A-Z0-9_]*, never PIPEMESH_ or CI_), and one that variables: or secrets: also sets is a load error.

• An artifact consumed without a name is no variable: image_from: orders the job after the image's build and puts the digest in its fingerprint, and the job runs in that image; a delegating job consumes what its body reaches with ../ the same way, since it runs no script.

• Only a job allowed to fail consumes from one: a job with allow_failure: true that fails still counts as succeeded, with whatever it uploaded before failing, so its artifacts may be missing or partial. A job that consumes from it, or from a job inside it, must set allow_failure: true too; otherwise the definition doesn't load.

• A file artifact is kept 30 days after its run, or what the instance or your organization sets instead. expire: on the entry wins: dist: {path: dist, expire: 90d} takes Nd, Nh, Nm or never. A changed setting applies to artifacts stored after it.

• Past its date an artifact is expired, except the artifacts of the job's two newest successful runs, which are kept so a waiting job and a rollback still have them. The Artifacts tab shows each date. An expired artifact downloads no more, a stored run whose outputs expired is not reused (the job builds again), a rollback to it isn't offered, and a job that consumes it fails before its script, naming the producer and the revision to re-run to make it again. pipemesh artifacts <job> lists a job's entries with their dates, and pipemesh storage your organization's retention.

Reports

reports: are what a job writes for people to read: test results as a site, a coverage summary, a screenshot. Each shows in the job panel's Reports tab.

test:
  job_type: build
  script: ./gradlew test
  reports:
    tests: build/reports/tests/test         # a directory with index.html: a site
    coverage:
      path: build/reports/coverage/summary.md
      title: Coverage                        # the tab's label; default the key
      expire: 30d                            # default: the artifact default

• A key is one path, a file or a directory inside the workspace, or { path, title, expire }. The format comes from the content: a directory with an index.html is a site, a .md file a document, a .png, .jpg, .gif, .webp or .svg file an image. Anything else is offered as a download.
• A report uploads whatever the job's outcome, after the script: failing tests are when it matters most. A missing one never fails the job. The log says what happened to each, and the Reports tab tells a job that failed before writing one from a job that succeeded and wrote nothing there, which usually means its output path changed.
• No job consumes a report, and its content is in no fingerprint. An output a job reads and people look at goes under both keys: produces: for the job, reports: for the tab.
• Script jobs and GitHub Actions jobs declare them. On a github_actions: job a report is the run's Actions artifact of the same name, imported whatever the conclusion. A workflow or pipeline job's body's jobs report.
• Reports count toward the organization's artifact storage and are kept as long as artifacts unless expire: says otherwise. A site holds at most 10,000 files.
• A report is shown sandboxed: its page runs in a sandbox with no access to Pipemesh or your session, from a link that lasts a few hours. A document is rendered with its raw HTML shown as text.

Renaming a job

A pipeline job remembers the revision it last ran, and its runs are its record. Renamed, it would start empty and its old runs would become unreachable. was: takes them over:

deploy_prod:
  job_type: deploy
  production: true
  was: deploy_staging        # takes over deploy_staging's cursor and runs
  checkout:
    - deploy
  script: deploy/deploy.sh prod
deploy_staging:              # a new job of the old name, starting empty
  job_type: deploy
  production: false
  checkout:
    - deploy
  script: deploy/deploy.sh staging

• It applies when the definition lands, once nothing of the pipeline is running. Runs, artifact sets and fingerprints move to the new name, which continues from the revision the job was at. Older runs show "as deploy_staging".
• It applies once: the was: key can then stay or go.
• A name that already has runs of its own can't take over another.

Workflow and pipeline jobs

A job_type: workflow or job_type: pipeline job hands the revision to a body of the matching type:, given with body: (!ref, !include or inline), and sets the child's variables:.

build:
  job_type: workflow                 # one run per revision, waited for
  stage: build
  body: !ref builds

service_a:
  job_type: pipeline                 # the revision, handed to a child pipeline
  stage: fanout
  body: !include .pipemesh/service.yaml
  variables:
    SERVICE: service_a
  fingerprint:
    checkout:
      - services/service_a
      - libs

Both also take stage, needs, timeout_seconds, allow_failure, matrix and was, and none of the executor keys.

• A workflow job waits for its run: its status is the run's verdict, its artifacts what the run produced.
• A pipeline job dispatches. The child pipeline promotes the revision through its own stages and gates, on its own timeline. The job settles once the revision arrives, shown as Dispatched. If a revision must pass some jobs before going on, make them a workflow.
• Only pipelines dispatch. A workflow can't feed a pipeline, whose revisions come from its branch: job_type: pipeline in a workflow's body is a load error, as job_type: deploy is.

A workflow job always runs. Each revision that reaches it starts its body, and the jobs inside reuse what they can: a build whose fingerprint matches an earlier pass hands up that pass's artifacts. checkout: is a load error: its jobs say what they read.

A pipeline job hands every revision over, unless its fingerprint: says what it watches. The child then decides each of its jobs on its own fingerprint.

• With fingerprint:, it holds a revision back until something it names changed. A shared body that picks its service by a variable lets each job watch its own service.
• A Settings edit reaches the child through the job: the edit starts a revision of the parent, and the job hands it over.

orders:
  job_type: pipeline
  stage: fanout
  body: !ref service
  variables:
    SERVICE: orders
  fingerprint:
    artifacts:
      - graph/orders          # the build graph's verdict for this service

• artifacts: lists produced artifacts by path (<job>/<key>). The job waits for each, and for every artifact the body reaches with ../, but only those it lists count.
• checkout: lists this repository's paths it watches. Left out (or false), no files count.
• The body's text, the job's variables and the Settings variables and secrets the body reads always count.
• Only a pipeline job has fingerprint:. A workflow job always runs, and its jobs decide for themselves.

The child is part of the job.

• It's registered under the job's name, under its parent's URL (…/repo/-/pipeline/build), and its runs carry an upstream trigger.
• Editing its body is a structure change of the parent, and its type: is checked at load.
• Children of one parent run in parallel.
• It runs in its parent run's context: a pull request's child (or a merge queue's) is a pull request's run, with its secrets, caches, images and job identity, and no variables: changes that.

Artifacts cross the edge by name.

• Down: ../ names the parent's artifacts. In a child body, ../build/jar in consumes: is the jar of the parent's build job, at the parent revision that started the child. Each .. goes one level up (../../graph/orders is the grandparent's). The parent job waits for build; a new jar changes its fingerprint when its fingerprint: lists build/jar.
• Up: consumers name the body's artifacts. A workflow job has no artifacts of its own: build/test/report is the report of the test job in the build job's body.

Other repositories

A pipeline can read other repositories, declared under repos: by alias and URL. Each must be one Pipemesh reaches in the same organization: every repository a connection reaches is there, with a pipemesh.yaml of its own or not.

• A branch goes after @: github.com/acme/infra@release/2.0.
• Without one, the repository is followed on the branch its own pipemesh.yaml is set up on.
• A branch that doesn't exist fails the load, and the pipeline keeps its last good definition until it's fixed.
• $self is the pipeline's own repository on its own branch. Another branch of the same repository is declared by its URL, like any other: app_v1: github.com/acme/app@release/1.x.

pipeline:
  type: pipeline
  repos:
    infra: github.com/acme/infra
  jobs:
    plan_infra:
      job_type: task
      repo: infra                       # works in infra, from its root
      checkout: true
      script: terraform plan
    deploy_staging:
      job_type: deploy
      production: false
      repo: infra
      checkout:
        - envs/staging
        - modules
      repos:
        $self:
          checkout:
            - shared-config
          mount: app
      script: cd envs/staging && terraform apply

• Each revision pins a version of every declared repository, and its jobs use those, whatever is pushed later. A push to one starts a revision on the pipeline's current commit (changes a few seconds apart share one).
• repo: is where a job works: it's checked out where the pipeline's repository would be, and the job starts there. Without it, a job works in the pipeline's own repository, $self. checkout: lists files of the job's repo:.
• repos: mounts repositories into the workspace, keyed by alias or $self, each with a required checkout: (true or a list) and a mount: path in the workspace.
• A mount can't leave the workspace (no leading /, no ..), be a directory the workspace repository has, or nest in another. Mounts stay out of the workspace repository's git status.
• The fingerprint covers every repository checked out, at the pinned versions. Secrets, Settings variables and commit statuses stay the pipeline's own repository's.
• A fork's pull request clones only public repositories besides its own: a repo: or repos: entry naming a private one fails the job (What a fork's pull request gets).

A pipeline that builds or releases several branches of one repository (one per supported version, say) reads more clearly from a repository of its own whose pipemesh.yaml declares each branch (app_v1: …@release/1.x, app_v2: …@release/2.x). Kept in one of those branches, that branch's pipemesh.yaml would drive the others' releases, and which file is in charge is easy to misread.

Several repositories, one pipeline walks through an example.

Running on GitHub Actions

github_actions: runs a build, deploy or task as a GitHub Actions workflow, in place of script: or uses:. Pipemesh dispatches it, shows the run's jobs as the job's tasks, and settles the job with the run's verdict. Your pipeline still runs the release; Actions is the compute.

deploy_staging:
  job_type: deploy
  production: false
  stage: staging
  needs:
    - verify
  github_actions:
    workflow: deploy.yml       # file under .github/workflows
    inputs:
      environment: staging

• Keys: workflow, inputs, ref (the branch or tag to dispatch on; default: the revision's branch). github_actions: deploy.yml is short for workflow: deploy.yml alone. The job waits for the run.
• An input may name the run's variables: revision: ${PIPEMESH_REVISION} or ${CI_COMMIT_SHORT_SHA} is resolved when the run is dispatched, so the workflow learns the revision it acts on without the number ever entering a fingerprint.
• The job takes stage, needs, consumes, produces, checkout, timeout_seconds, retry, allow_failure and matrix, but none of the keys that run a script on Pipemesh: script, setup, uses, runner, dockerd, image, image_from, services, secrets, id_tokens, cache, publish, or its own artifacts.
• The Pipemesh GitHub App needs Actions: read and write on the organization (GitHub integration).
• Not in forks' pull requests. A github_actions: job in a fork's pull request fails without dispatching: the workflow would run in your repository, with its secrets, on code the fork proposed. Pull requests from your own branches dispatch as usual.

The receiving workflow declares workflow_dispatch with:

• pipemesh_sha (the revision to act on) and pipemesh_run (the job run id), which Pipemesh always sends;
• every input in the definition, at most 8 (GitHub caps a dispatch at 10).

GitHub reads the workflow file from the branch, so the workflow checks out pipemesh_sha itself. Put pipemesh_run in run-name: so Pipemesh matches the run exactly.

on:
  workflow_dispatch:
    inputs:
      pipemesh_sha:
        type: string
        required: true
      pipemesh_run:
        type: string
        required: true
      environment:
        type: string
        required: true
run-name: deploy ${{ inputs.environment }} · ${{ inputs.pipemesh_run }}
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: "${{ inputs.pipemesh_sha }}"
      - run: ./deploy.sh ${{ inputs.environment }}

On the board, the card expands into the run's jobs, each with its own status, duration and log, sectioned by GitHub's steps. GitHub releases a log's text when its job finishes. Cancelling or superseding in Pipemesh cancels the GitHub run.

What the run checks out

Pipemesh reads the dispatched .github/workflows/<file> at the revision, and takes the job's checkout from its jobs' steps:

• In cone mode (the action's default), a pattern is a directory, as in checkout:, and the root files come too: sparse-checkout: deploy means deploy and the root files.
• Other steps don't change it. The workflow above checks out everything, so its job's checkout is true.
• The workflow file, and every local workflow it calls, is in the fingerprint: editing it runs the job again.
• A checkout: on the job overrides what was read. Pipemesh can't enforce an Actions run's checkout: a narrowed one is a promise the workflow keeps.

Passing artifacts

A produces: file artifact is the run's Actions artifact of the same name, imported when the run succeeds, so any job can consume it. Artifacts the job consumes: are handed to the run, which fetches them with the pipemesh/consume action. From pipemesh/demo-actions:

build:
  job_type: build
  stage: build
  timeout_seconds: 900
  produces:
    dist: file
  github_actions: build.yml

verify:
  job_type: build
  stage: build
  consumes:
    DIST: build/dist              # the Actions artifact, unpacked at dist/
  checkout: false
  script: |
    echo "built version $(cat dist/version.txt) at $(cat dist/revision.txt)"
    test "$(cat dist/revision.txt)" = "$CI_COMMIT_SHA"

Matrix

matrix: expands a job at load time into real jobs, one per combination of its variables' values (at most 256). as: picks what the variants become; nothing new happens at runtime.

test:
  job_type: build
  stage: test
  matrix:                     # 2×2 = 4 sibling jobs
    stream:
      - aws
      - docker
    arch:
      - amd64
      - arm64
  script: |
    ./test.sh ${{ matrix.stream }} --arch ${{ matrix.arch }}

Values are job variables when the name is shell-exportable ($stream), and always available as ${{ matrix.<variable> }}.

as: jobs (the default) makes sibling jobs (test[stream=aws,arch=amd64], …), each addressable, and in a pipeline each with its own promotion cursor, so one broken lane never blocks the others. needs: selects them:

needs:
  - test                             # bare name: every variant
needs:
  - test[stream=aws]                 # partial selector: the matching slice
  - test[stream=docker, arch=arm64]  # full selector: exactly one variant
  - job: test                        # structured form: the same as test[stream=aws]
    matrix:
      stream: aws

• Selectors are always key=value, never positional, so a new axis widens them instead of breaking them. They resolve at load: one that matches nothing is an error naming the declared values.
• A block list takes bracket selectors as they are. Inside a flow list ([...]) they need quotes: that's YAML, not Pipemesh.
• A value can pick a variant's checkout ("streams/${{ matrix.stream }}.txt"), so editing one stream's files runs only that lane.
• Lanes chain by interpolating their own binding, never by shared variable names:

publish:
  job_type: deploy
  production: true
  stage: publish
  matrix:
    stream:
      - aws
      - docker
  needs:
    - test[stream=${{ matrix.stream }}]   # publish[stream=aws] → test[stream=aws]
  checkout:
    - publish.sh
  script: ./publish.sh ${{ matrix.stream }}

as: workflow makes the job one node over a generated child workflow of the variants: one verdict, one cursor, referenced only as the node (suite in needs:). Use it when the variants pass or fail as a unit, like a GitHub Actions CI matrix:

suite:
  job_type: build
  stage: test
  matrix:
    stream:
      - aws
      - docker
      - slack
    as: workflow
  script: ./suite.sh ${{ matrix.stream }}

Files and !include

!include <path> puts a file's whole content where the tag stands: a body, a job, a cache: declaration.

# .pipemesh/ci.yaml — and the same job file in .pipemesh/build.yaml
type: workflow
stages:
  - verify
jobs:
  backend_build: !include .pipemesh/jobs/backend_build.yaml
  lint:
    type: job                  # a !ref reaches it (see below)
    job_type: build
    stage: verify
    script: make lint

# .pipemesh/jobs/backend_build.yaml
type: job
job_type: build
stage: verify
checkout:
  - backend
script: make -C backend build

• Paths are relative to the repository root and name a .yaml or .yml file, read at the including file's commit. Included files may include others.
• An included file says its shape with type:, unless it's plain data, and is used whole: nothing merges or overrides.
• Editing one is like editing the definition: a pipeline that includes it sees a structure change, and a workflow's next run uses the new content.
• Load errors naming the file: a missing or empty file, a cycle, a path outside the repository, any other YAML tag. A body is never named by path: file: or a path in body: fails, saying what to write instead.

References and !ref

!ref <path> puts the value at that path where the tag stands. The path names every key from the root of the tag's own file.

common:                                     # plain data: no type:
  login:
    script:
      - aws sso login
      - aws sts get-caller-identity
ci: !include .pipemesh/ci.yaml

deployment:
  type: pipeline
  stages:
    - verify
  jobs:
    lint: !ref ci.jobs.lint                 # one job out of an included file; it says type: job
    deploy:
      job_type: deploy
      production: true
      stage: verify
      checkout:
        - deploy.sh
      script:
        - !ref common.login.script          # a list splices in
        - ./deploy.sh

• A reference never leaves its file. In .pipemesh/ci.yaml, !ref reaches that file's keys and what it includes, never the definition's.
• A reference is a whole value: never part of a string, never merged with keys beside it. As a list item, a reference to a list adds its items.
• Its type: is checked where it lands (Types). A cycle or a missing key is a load error naming the file and line.
• YAML anchors, aliases and merge keys (&x, *x, <<:) are load errors: !ref is the one way to reuse a value, and nothing overrides.

Typed components

A component is a reusable job with parameters, in the same repository, called with uses: and with:. Its file has type: component, a name:, params: (each with a type: string, int, bool or list), and the job: it runs.

# .pipemesh/components/eks-deploy.yaml
type: component
name: eks-deploy
params:
  cluster:
    type: string
    required: true
  replicas:
    type: int
    default: 2
job:
  script: |
    helm upgrade app charts/app --kube-context ${{ params.cluster }} --set replicas=${{ params.replicas }}

# pipemesh.yaml, in the pipeline's jobs:
deploy_production:
  job_type: deploy
  production: true
  stage: production
  checkout:
    - charts
  uses: ./.pipemesh/components/eks-deploy.yaml
  with:
    cluster: prod
    replicas: 4

• ${{ params.x }} interpolates at load time. The resolved values are part of the pipeline's structure, so editing a component is a structure change like any other.
• Registry components add a stream and a major: uses: aws/role@1 is the registry's aws/role at its current major, pinned by digest, so a registry advance re-anchors your structure deliberately.
• No inheritance, no merging. A component is authored (its own script:, setup:, image:) or an instance of another (a single uses: + with:), never both. To specialize one, write an instance that binds some of its params and re-exposes the rest.
• Placement keys belong to the job (job_type, stage, needs, checkout, runner, timeouts), execution keys to the component (script, setup, image). An execution key beside uses: is a load error.
• To let callers add behavior, a component declares a hook param and interpolates it: a contract, never a splice.
• A component declares the identity tokens it needs with id_tokens: in its job:, and they join the job's, params interpolated (aud: ${{ params.audience }}). It reads its token from the variable, as the job would. A name the job or another component also declares must name the same audiences.

setup: runs script-only components before the job's own script:

setup:
  - uses: ./.pipemesh/components/aws-role.yaml
    with:
      arn: "arn:aws:iam::123:role/deploy"
      region: us-east-1
script: |
  aws sts get-caller-identity

Editor validation

Point yaml-language-server (built into the VS Code YAML extension) at Pipemesh's schemas with a modeline at the top of each file:

# yaml-language-server: $schema=https://pipemesh.io/api/meta/pipemesh-definition.schema.json

Use pipemesh-body.schema.json for .pipemesh/*.yaml bodies and pipemesh-component.schema.json for components. They reject unknown keys as the loader does, so a typo is a red squiggle before it's a load error. Declare the tags in your editor settings so they aren't flagged:

"yaml.customTags": ["!include scalar", "!ref scalar", "!settings scalar", "!var scalar"]

The editor reads a tagged value as a plain string; the loader resolves and checks it.