The definition (pipemesh.yaml)

pipemesh.yaml sits at the root of your repository. Its pipemesh: key names every workload the repository runs, as a pipeline or a workflow. Every other root key is yours (a job, a body, helpers, an included file), and pipemesh: reaches it with !ref.

builds: !include .pipemesh/build.yaml

deploy_staging:
  type: job
  job_type: deploy
  production: false
  stage: staging
  consumes:
    - build/image/app
  checkout:
    - scripts
    - charts
  script: |
    ./scripts/deploy.sh staging
    echo "deployed $CI_COMMIT_SHORT_SHA"

deployment:
  type: pipeline
  stages:
    - build
    - staging
  jobs:
    build:
      job_type: workflow
      stage: build
      body: !ref builds
    deploy_staging: !ref deploy_staging

pipemesh:
  pipelines:
    pipeline: !ref deployment
  workflows:
    checks:
      body: !include .pipemesh/ci.yaml
      on: pull_request
    nightly:
      body: !include .pipemesh/nightly.yaml
      on: schedule
      cron: "0 3 * * *"

Three layers

1. Bodies say what runs: a type: (workflow or pipeline), stages: and jobs:. Write one at the root, in an included file, or inline.
2. Registration names a workload under pipemesh.pipelines or pipemesh.workflows, and the body's type: must match. The entry is the body itself, or a map with it under body:, beside inputs: and triggers.
3. Triggers say when a workflow runs. Pipelines take none: every commit on the default branch is a revision.

• Bodies are whole: nothing extends or merges. Reuse a job with a type: job definition and !ref; a job that varies is a component. Triggers live on the registration, never in a body, so a body can back a registration and a job alike.
• A workload is active once named, with a URL, a history and a run counter.
• Names are unique across both sections and match [a-z0-9][a-z0-9_-]*. Reserved: settings, jobs, runs, events, workflows, pipelines, repository, and pipeline for a workflow (it's the pipeline's slot).
• Every root key must be reached from pipemesh:, directly or through another definition. An unreached key is a load error naming it, which catches typos.

Triggers

pipemesh:
  workflows:
    release:
      body: !include .pipemesh/release.yaml
      inputs:
        channel:
          default: stable
      triggers:
        stable:
          on: tag
          tags:
            - "v*"
        rc:
          on: tag
          tags:
            - "rc-*"
          inputs:
            channel: rc
        nightly:
          on: schedule
          cron: "0 3 * * *"
          inputs:
            channel: nightly
    checks:
      body: !include .pipemesh/ci.yaml
      on: pull_request
    smoke:
      body: !include .pipemesh/smoke.yaml   # no triggers: run it from its page

• push fires on each new commit of the branch the repository was added with (its default branch). Other branches start nothing, so branches: can only repeat that name, and any other never fires.
• on: or triggers:, not both. on: names the trigger after its event, triggers: after its key. The name filters the run list (?trigger=tag:rc) and fires the trigger by API.
• A workload's triggers share its history and run counter. For one event to start several workloads, declare it on each.
• inputs: on the entry declares names and defaults. Triggers supply values, frozen into each run's variables; an undeclared input is a load error. Same history, different values: a trigger with inputs. Separate history: a second entry sharing the body.
• No triggers means manual only: listed in the catalog, run from its page or the API.

Types

type: is a structure's shape (job, workflow, pipeline or component), checked where it's used. A !ref to the wrong definition is a load error.

• Required where a structure stands alone: a root definition, a job or body file brought in with !include, a job a !ref picks out of another definition (!ref ci.jobs.lint), every component.
• Optional, but checked, inline: a job under jobs:, a body as body:, an entry under pipemesh:.
• None on plain data: a cache: declaration in its own file, a list of script lines, a map of helpers.

job_type: is what a job is, and must agree with its body's type:. Here they disagree:

orders:
  job_type: workflow                      # runs a workflow and waits for it
  stage: dispatch
  body: !include .pipemesh/service.yaml   # a file that says type: pipeline

That's a load error naming both sides:

pipemesh.pipelines.pipeline: job orders is job_type: workflow, so its
body must be type: workflow; the body is type: pipeline — to hand
revisions to it, write job_type: pipeline

Job types

job_type: says what a job is and where it may appear. It sets the job's checkout and skip defaults.

pipeline:
  type: pipeline
  stages:
    - build
    - staging
    - production
  jobs:
    build:
      job_type: build                # checkout: true, skip: built
      stage: build
      script: npm ci && npm test && npm run build
      produces:
        dist: dist
    deploy_staging:
      job_type: deploy               # checkout: false, skip: unchanged
      production: false
      stage: staging
      consumes:
        - build/dist
      checkout:                      # and the deploy scripts
        - deploy
      script: deploy/deploy.sh staging
    smoke:
      job_type: task                 # skip: never: runs on every revision
      stage: staging
      needs:
        - deploy_staging
      checkout:
        - smoke.sh
      script: ./smoke.sh https://staging.example.com
    deploy_prod:
      job_type: deploy
      production: true
      stage: production
      needs:
        - smoke
      consumes:
        - build/dist
      github_actions:
        workflow: deploy.yml
        inputs:
          environment: production

• build is hermetic: an earlier run with the same fingerprint stands in for it, shown as reused (its outputs, or its pass). Tests, lint, image builds and build-graph fingerprint jobs (nx/fingerprint@1, …) are builds, as is a job that signs or packages what it consumes, with checkout: false.
• deploy ships what it consumes to an environment, when that changed since its last success. Deploys belong to pipelines, which alone have a last success, each environment's revision, rollback and holds. A pull-request preview or manual hotfix deploy is a task in a workflow, not tracked as a deployment.
• Every deploy says production: true or production: false. Production deploys are what DORA metrics, pipemesh deployments and pipemesh deployed count, wherever they sit. Matrix variants share it, and it doesn't change how the job runs.
• task (the default) runs on every revision that reaches it: smoke tests, notifications, checks that read pull-request context such as a merge-base diff.
• workflow and pipeline run no script: they hand the revision to a body.

Override any default on the job, such as a build's checkout: list or a task's skip: unchanged. The job's Rules tab shows its job type, effective checkout and skip, which came from the job type (defaults:), and what Pipemesh added to the checkout (added:). job_type: belongs to the job, never a component, so one component can be a build here and a task there.

A build, deploy or task names exactly one executor:

What a job checks out

checkout: true        # the whole repository
checkout: false       # nothing
checkout:             # exactly these
  - services/orders
  - libs/money
  - package.json
checkout:             # everything but these
  - "**"
  - "!docs"
  - "!README.md"

• A path covers itself and everything under it (services/api, go.mod); globs cut finer (services/*/go.mod, **/Dockerfile).
• Paths are anchored at the repository root: for a job with repo:, that repository's.
• Prefer whole directories, and exclusions to long lists. An entry starting with ! excludes. The list is read as git reads a non-cone sparse checkout: each file follows the most specific path an entry matches (the file, then its directory, then the one above), and among entries on the same path, the last one decides. ["**", "!docs"] is everything but docs/; ["**", "!docs", "docs/api"] keeps docs/api; ["**", "!**/*.md"] drops every Markdown file.
• Quote exclusions (- "!docs"): a bare ! starts a YAML tag.
• The workspace holds exactly the checkout, on hosted runners and your own, so a job can't read a file its fingerprint leaves out. List the root files it reads too: package.json, lockfiles, tsconfig.json, .nvmrc, wrapper scripts, tool pins.
• History is complete: the clone isn't shallow, so git log, git diff and git merge-base work in every job, even one that checks out nothing.
• Pipemesh adds what it reads itself: each file a cache key hashes (${checksum:package-lock.json}), and each publish: context and Dockerfile. They join the checkout and fingerprint, under added: in the Rules tab.
• Load errors: an empty list or ["**"] alone (write false or true), "**" with no exclusion beside it, only exclusions.
• On GitHub Actions, the checkout comes from the workflow file.

When a job runs

A job runs when its fingerprint is new. The fingerprint hashes the job's definition and parameters, the digests of what it consumes, the files it checks out (in its repository and each mount), and the versions of its secrets and Settings variables. skip: says which earlier fingerprint lets it skip.

• unchanged is a pipeline's policy: a workflow body that declares it fails to load.
• After a failed or cancelled run, unchanged runs in full until a run succeeds: a half-finished deploy leaves the service in an unknown state. A successful rollback counts as a success.
• With built, a pull request reuses outputs from trusted runs and its own, never another pull request's.

A job that reads nothing runs once. If its skip: isn't never and it has no input (no checkout, consumes:, image_from:, repos:, secrets: or a !settings variable), its fingerprint is its definition alone. It loads, with a warning in its job panel. Say what it reads, or give it skip: never. Tasks never skip by default; deploys and pipeline jobs that consume nothing, and builds that check out nothing, can.

Release flow covers fingerprints and skips across a pipeline.

Jobs

Job keys will look familiar: job_type, stage, script, needs, runner, dockerd, image, image_from, variables, timeout_seconds, retry, allow_failure, secrets, artifacts, cache, publish, produces, consumes, checkout, skip. Names are lowercase letters, digits, _ and -.

• A job waits only for what it names: its needs:, and the producers of what it consumes:. stage: places it on the board but doesn't order the run, so a job that names nothing starts with the revision.
• services: is accepted but starts nothing: no runner runs service containers. Start a database in your script, or in Docker with dockerd: true.

Where a job runs

• The same sizes run on x86-64 as linux-amd64-small, -medium and -large (Hosted runners). runner: can also name one of your organization's own runners.
• dockerd: true adds a Docker daemon (Testcontainers, docker compose). publish: implies it.
• A hosted job's log ends with what it used: [runner] linux-arm64-small: peak memory 1.2 GiB of 6.5 GiB, CPU time 34 s.
• KUBERNETES_* variables are refused: they set a pod's resources, which is runner:'s job.

Variables and secrets

Every job gets its commit: CI_COMMIT_SHA, CI_COMMIT_SHORT_SHA, CI_COMMIT_REF_NAME, and CI_COMMIT_TAG on a tag. A pipeline job also gets PIPEMESH_REVISION (its revision), and a workflow job PIPEMESH_RUN_NUMBER (its run's number). Both only grow, so they make release numbers (:r$PIPEMESH_REVISION).

Run numbers aren't in the fingerprint, and a reused build hands over what it built back then. Apply the number where you tag or release, not in what you build.

deploy_staging:
  job_type: deploy
  production: false
  checkout:
    - deploy
  variables:
    API_URL: !settings
    LOG_LEVEL: info
  secrets:
    - DB_PASSWORD
  script: deploy/deploy.sh staging

variables: names the variables the job gets. A plain value is the definition's; NAME: !settings reads the value set in the repository's Settings → Variables (or its organization's). secrets: names the repository secrets it gets. It gets no others, and a !settings variable or secret that isn't set fails the job before its script.

• One source per variable. A name is written in the definition or read from Settings, never both: !settings on a name the body or the run that started the job also sets is an error.
• Each revision pins its Settings. Every edit is a new version, and a job runs with the versions current when its revision started, never an edit made since.
• An edit starts a revision on the current commit, for each pipeline with a job that reads it (edits a few seconds apart share one). So staging gets a new value before production, and rolling back restores the old one.

Images

image: names a registry image. image: !var NAME takes it from one of the job's variables, so a deploy can run in an image pinned in Settings:

deploy_prod:
  job_type: deploy
  production: true
  variables:
    DEPLOY_IMAGE: !settings      # e.g. ghcr.io/acme/deployer@sha256:…
  image: !var DEPLOY_IMAGE
  script: deploy/deploy.sh prod

• A !settings image belongs to the revision, resolved when the revision is created, against the Settings it pins. Changing the variable starts a revision in the new image; rolling back restores the old one. The revision's details show it, and the log says [image] DEPLOY_IMAGE = ….
• An unset variable, or one that isn't an image reference, stops the revision: a commit's definition doesn't load (like a missing !include file), and an edit's revision doesn't start. The repository says why until a commit or edit fixes it.
• A plain variable resolves when the definition loads, so the same value can name the image and reach the script.
• !var is the whole value, only image: takes it, and it names a variable, never a secret. pipemesh check can't read your Settings, so it checks the name only.

image_from: runs the job in an oci entry's image, named by path and pinned by digest. It's consumed like any entry (the job waits for the producer), and a child body reaches its parent's image with ../. image: and image_from: are exclusive.

publish: without repo: builds a job image Pipemesh keeps for you in its registry, with no registry token to store:

build_ci:
  job_type: build
  stage: image
  checkout:
    - ci
  script: echo "building the CI image"
  publish:
    - context: ci              # no repo: Pipemesh keeps it
      key: ci_image
test:
  job_type: build
  stage: test
  image_from: build_ci/ci_image
  script:
    - make test

• It's tagged by its build context's fingerprint, so an image already built from the same content is reused, not rebuilt.
• A pull request builds into a place of its own, which default-branch runs never read. A build reuses (skip: built), so a pull request that doesn't touch the directory runs in the default branch's image.

Caches

webapp_build:
  job_type: build
  checkout:
    - webapp
  cache:
    key: npm-${checksum:webapp/package-lock.json}
    restore_keys:
      - npm-
    paths:
      - .npm
  script: cd webapp && npm ci --cache ../.npm && npm run build

cache: restores its paths: before the script and saves them under key: after a successful run.

• An entry never changes once saved, so put a checksum of what the paths depend on in the key.
• On a miss, restore_keys: prefixes are tried in order; the newest match wins.
• Trusted runs save; pull requests only read. A pull request restores its own workflow's entries first, then what any pipeline or workflow of the same definition saved for the same paths:. Every pull request can read those: keep credentials out of cached paths.

Scripts and timeouts

script: is a block scalar: real multi-line shell, verbatim, with # comments allowed.

script: |
  for f in a b; do
    echo "part $f"    # loops need no escaping
  done

• before_script:, script: and after_script: run as one script, in that order, in one shell that stops at the first failing command. So after_script: doesn't run after a failure, and a failing line in it fails the job.
• timeout_seconds: defaults to one hour (3600). At the limit the job is stopped and fails.

Artifacts and entries

• artifacts: flow along needs: edges, with what the needs inherited too: a deploy that names only its test lane still gets the bundle that lane validated. An artifact arriving over two edges is unpacked once, and keeps its provenance (which job built it, at which revision).
• produces: entries reach only jobs that name them in consumes: (build/jar), never one that only needs: the producer. Consuming orders the job after the producer and puts the entry's digest in its fingerprint.

Renaming a job

A pipeline job remembers the revision it last ran, and its runs are its record. Renamed, it would start empty and its old runs would become unreachable. was: takes them over:

deploy_prod:
  job_type: deploy
  production: true
  was: deploy_staging        # takes over deploy_staging's cursor and runs
  checkout:
    - deploy
  script: deploy/deploy.sh prod
deploy_staging:              # a new job of the old name, starting empty
  job_type: deploy
  production: false
  checkout:
    - deploy
  script: deploy/deploy.sh staging

• It applies when the definition lands, once nothing of the pipeline is running. Runs, artifact sets and fingerprints move to the new name, which continues from the revision the job was at. Older runs show "as deploy_staging".
• It applies once: the was: key can then stay or go.
• A name that already has runs of its own can't take over another.

Workflow and pipeline jobs

A job_type: workflow or job_type: pipeline job hands the revision to a body of the matching type:, given with body: (!ref, !include or inline), and sets the child's variables:.

build:
  job_type: workflow                 # one run per revision, waited for
  stage: build
  body: !ref builds

service_a:
  job_type: pipeline                 # the revision, handed to a child pipeline
  stage: fanout
  body: !include .pipemesh/service.yaml
  variables:
    SERVICE: service_a
  checkout:
    - services/service_a
    - libs

Both also take stage, needs, consumes, skip, timeout_seconds, allow_failure, matrix and was, and none of the executor keys.

• A workflow job waits for its run: its status is the run's verdict, its artifacts what the run produced.
• A pipeline job dispatches. The child pipeline promotes the revision through its own stages and gates, on its own timeline. The job settles once the revision arrives, shown as Dispatched. If a revision must pass some jobs before going on, make them a workflow.
• Only pipelines dispatch. A workflow can't feed a pipeline, whose revisions come from its branch: job_type: pipeline in a workflow's body is a load error, as job_type: deploy is.

A workflow job reads what its jobs read.

• Its fingerprint combines their inputs: their checkouts (one true makes it the whole repository), the Settings variables and secrets they read, what they consume from outside the body, and the body's own text.
• When every job reuses, a revision none of them reads reuses the whole run, entries included.
• skip: never is the only override, and checkout: is a load error.
• It runs every time when nested in a workflow run, or when a job works in or mounts another repository (repo:, repos:), whose inputs can't be combined at load.

A pipeline job should hand over only when its service changed. The child decides each job on its own fingerprint, so name what selects the service: its entry from a build-graph job (graph/orders in consumes:) or, with no build graph, its files (services/orders and libs/shared in checkout:; nothing is checked out, as the job runs no script). Given neither, its defaults (checkout: false, skip: unchanged) hand the revision over once, and never again.

The child is part of the job.

• It's registered under the job's name, under its parent's URL (…/repo/-/pipeline/build), and its runs carry an upstream trigger.
• Editing its body is a structure change of the parent, and its type: is checked at load.
• Children of one parent run in parallel.

Artifacts and entries cross the edge.

• Down: the child's root jobs (no needs: of their own) receive what the parent job inherits, and pass it on along their own edges. Nothing is copied.
• Up: when a workflow child settles, what it produced becomes the parent job's artifacts, inherited by jobs that needs: it. A workflow job has no entries of its own: consumers name its body's (build/test/report).
• ../ names the parent's entries. In a child body, ../build/jar in consumes: is the jar of the parent's build job, at the parent revision that started the child. Each .. goes one level up (../../graph/orders is the grandparent's). The parent job consumes the same entry, so it waits for build, and a new jar changes its fingerprint.

Other repositories

A pipeline can read other repositories, declared under repos: by alias and URL. Each must be added to the same organization (with or without a pipeline of its own), and is followed on its default branch.

pipeline:
  type: pipeline
  repos:
    infra: github.com/acme/infra
  jobs:
    plan_infra:
      job_type: task
      repo: infra                       # works in infra, from its root
      checkout: true
      script: terraform plan
    deploy_staging:
      job_type: deploy
      production: false
      repo: infra
      checkout:
        - envs/staging
        - modules
      repos:
        $self:
          checkout:
            - shared-config
          mount: app
      script: cd envs/staging && terraform apply

• Each revision pins a version of every declared repository, and its jobs use those, whatever is pushed later. A push to one starts a revision on the pipeline's current commit (changes a few seconds apart share one).
• repo: is where a job works: it's checked out where the pipeline's repository would be, and the job starts there. Without it, a job works in the pipeline's own repository, $self. checkout: lists files of the job's repo:.
• repos: mounts repositories into the workspace, keyed by alias or $self, each with a required checkout: (true or a list) and a mount: path in the workspace.
• A mount can't leave the workspace (no leading /, no ..), be a directory the workspace repository has, or nest in another. Mounts stay out of the workspace repository's git status.
• The fingerprint covers every repository checked out, at the pinned versions. Secrets, Settings variables and commit statuses stay the pipeline's own repository's.

Several repositories, one pipeline walks through an example.

Running on GitHub Actions

github_actions: runs a build, deploy or task as a GitHub Actions workflow, in place of script: or uses:. Pipemesh dispatches it, shows the run's jobs as the job's tasks, and settles the job with the run's verdict. Your pipeline still runs the release; Actions is the compute.

deploy_staging:
  job_type: deploy
  production: false
  stage: staging
  needs:
    - verify
  github_actions:
    workflow: deploy.yml       # file under .github/workflows
    inputs:
      environment: staging

• Keys: workflow, inputs, ref (the branch or tag to dispatch on; default: the revision's branch) and artifacts. github_actions: deploy.yml is short for workflow: deploy.yml alone. The job waits for the run.
• The job takes stage, needs, consumes, produces, checkout, skip, timeout_seconds, retry, allow_failure and matrix, but none of the keys that run a script on Pipemesh: script, setup, uses, runner, dockerd, image, image_from, services, secrets, cache, publish, or its own artifacts.
• The Pipemesh GitHub App needs Actions: read and write on the organization (GitHub integration).

The receiving workflow declares workflow_dispatch with:

• pipemesh_sha (the revision to act on) and pipemesh_run (the job run id), which Pipemesh always sends;
• every input in the definition, at most 8 (GitHub caps a dispatch at 10).

GitHub reads the workflow file from the branch, so the workflow checks out pipemesh_sha itself. Put pipemesh_run in run-name: so Pipemesh matches the run exactly.

on:
  workflow_dispatch:
    inputs:
      pipemesh_sha:
        type: string
        required: true
      pipemesh_run:
        type: string
        required: true
      environment:
        type: string
        required: true
run-name: deploy ${{ inputs.environment }} · ${{ inputs.pipemesh_run }}
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: "${{ inputs.pipemesh_sha }}"
      - run: ./deploy.sh ${{ inputs.environment }}

On the board, the card expands into the run's jobs, each with its own status, duration and log, sectioned by GitHub's steps. GitHub releases a log's text when its job finishes. Cancelling or superseding in Pipemesh cancels the GitHub run.

What the run checks out

Pipemesh reads the dispatched .github/workflows/<file> at the revision, and takes the job's checkout from its jobs' steps:

• In cone mode (the action's default), a pattern is a directory, as in checkout:, and the root files come too: sparse-checkout: deploy means deploy and the root files.
• Other steps don't change it. The workflow above checks out everything, so its job's checkout is true.
• The workflow file, and every local workflow it calls, is in the fingerprint: editing it runs the job again.
• A checkout: on the job overrides what was read. Pipemesh can't enforce an Actions run's checkout: a narrowed one is a promise the workflow keeps.

Passing entries and artifacts

A produces: file entry is the run's Actions artifact of the same name, imported when the run succeeds, so any job can consume it. Entries the job consumes: are handed to the run, which fetches them with the pipemesh/consume action. From pipemesh/demo-actions:

build:
  job_type: build
  stage: build
  timeout_seconds: 900
  produces:
    dist: file
  github_actions: build.yml

verify:
  job_type: build
  stage: build
  consumes:
    - build/dist                  # the Actions artifact, unpacked at dist/
  checkout: false
  script: |
    echo "built version $(cat dist/version.txt) at $(cat dist/revision.txt)"
    test "$(cat dist/revision.txt)" = "$CI_COMMIT_SHA"

An artifacts: list under github_actions: (dist, or "*" for all) keeps the run's artifacts as the job's own artifacts: instead, inherited along needs: edges. Each extracts into a directory named after it: GitHub packs uploads relative to their common root, so you get dist/ whichever path was uploaded.

GitHub Enterprise Server works the same way: for an organization on your GHES host, Pipemesh talks to its /api/v3.

Matrix

matrix: expands a job at load time into real jobs, one per combination of its variables' values (at most 256). as: picks what the variants become; nothing new happens at runtime.

test:
  job_type: build
  stage: test
  matrix:                     # 2×2 = 4 sibling jobs
    stream:
      - aws
      - docker
    arch:
      - amd64
      - arm64
  script: |
    ./test.sh ${{ matrix.stream }} --arch ${{ matrix.arch }}

Values are job variables when the name is shell-exportable ($stream), and always available as ${{ matrix.<variable> }}.

as: jobs (the default) makes sibling jobs (test[stream=aws,arch=amd64], …), each addressable, and in a pipeline each with its own promotion cursor, so one broken lane never blocks the others. needs: selects them:

needs:
  - test                             # bare name: every variant
needs:
  - test[stream=aws]                 # partial selector: the matching slice
  - test[stream=docker, arch=arm64]  # full selector: exactly one variant
  - job: test                        # structured form: the same as test[stream=aws]
    matrix:
      stream: aws

• Selectors are always key=value, never positional, so a new axis widens them instead of breaking them. They resolve at load: one that matches nothing is an error naming the declared values.
• A block list takes bracket selectors as they are. Inside a flow list ([...]) they need quotes: that's YAML, not Pipemesh.
• A value can pick a variant's checkout ("streams/${{ matrix.stream }}.txt"), so editing one stream's files runs only that lane.
• Lanes chain by interpolating their own binding, never by shared variable names:

publish:
  job_type: deploy
  production: true
  stage: publish
  matrix:
    stream:
      - aws
      - docker
  needs:
    - test[stream=${{ matrix.stream }}]   # publish[stream=aws] → test[stream=aws]
  checkout:
    - publish.sh
  script: ./publish.sh ${{ matrix.stream }}

as: workflow makes the job one node over a generated child workflow of the variants: one verdict, one cursor, referenced only as the node (suite in needs:). Use it when the variants pass or fail as a unit, like a GitHub Actions CI matrix:

suite:
  job_type: build
  stage: test
  matrix:
    stream:
      - aws
      - docker
      - slack
    as: workflow
  script: ./suite.sh ${{ matrix.stream }}

Files and !include

!include <path> puts a file's whole content where the tag stands: a body, a job, a cache: declaration.

# .pipemesh/ci.yaml — and the same job file in .pipemesh/build.yaml
type: workflow
stages:
  - verify
jobs:
  backend_build: !include .pipemesh/jobs/backend_build.yaml
  lint:
    type: job                  # a !ref reaches it (see below)
    job_type: build
    stage: verify
    script: make lint

# .pipemesh/jobs/backend_build.yaml
type: job
job_type: build
stage: verify
checkout:
  - backend
script: make -C backend build

• Paths are relative to the repository root and name a .yaml or .yml file, read at the including file's commit. Included files may include others.
• An included file says its shape with type:, unless it's plain data, and is used whole: nothing merges or overrides.
• Editing one is like editing the definition: a pipeline that includes it sees a structure change, and a workflow's next run uses the new content.
• Load errors naming the file: a missing or empty file, a cycle, a path outside the repository, any other YAML tag. A body is never named by path: file: or a path in body: fails, saying what to write instead.

References and !ref

!ref <path> puts the value at that path where the tag stands. The path names every key from the root of the tag's own file.

common:                                     # plain data: no type:
  login:
    script:
      - aws sso login
      - aws sts get-caller-identity
ci: !include .pipemesh/ci.yaml

deployment:
  type: pipeline
  stages:
    - verify
  jobs:
    lint: !ref ci.jobs.lint                 # one job out of an included file; it says type: job
    deploy:
      job_type: deploy
      production: true
      stage: verify
      checkout:
        - deploy.sh
      script:
        - !ref common.login.script          # a list splices in
        - ./deploy.sh

• A reference never leaves its file. In .pipemesh/ci.yaml, !ref reaches that file's keys and what it includes, never the definition's.
• A reference is a whole value: never part of a string, never merged with keys beside it. As a list item, a reference to a list adds its items.
• Its type: is checked where it lands (Types). A cycle or a missing key is a load error naming the file and line.
• YAML anchors, aliases and merge keys (&x, *x, <<:) are load errors: !ref is the one way to reuse a value, and nothing overrides.

Typed components

A component is a reusable job with parameters, in the same repository, called with uses: and with:. Its file has type: component, a name:, params: (each with a type: string, int, bool or list), and the job: it runs.

# .pipemesh/components/eks-deploy.yaml
type: component
name: eks-deploy
params:
  cluster:
    type: string
    required: true
  replicas:
    type: int
    default: 2
job:
  script: |
    helm upgrade app charts/app --kube-context ${{ params.cluster }} --set replicas=${{ params.replicas }}

# pipemesh.yaml, in the pipeline's jobs:
deploy_production:
  job_type: deploy
  production: true
  stage: production
  checkout:
    - charts
  uses: ./.pipemesh/components/eks-deploy.yaml
  with:
    cluster: prod
    replicas: 4

• ${{ params.x }} interpolates at load time. The resolved values are part of the pipeline's structure, so editing a component is a structure change like any other.
• Registry components add a stream and a major: uses: aws/role@1 is the registry's aws/role at its current major, pinned by digest, so a registry advance re-anchors your structure deliberately.
• No inheritance, no merging. A component is authored (its own script:, setup:, image:) or an instance of another (a single uses: + with:), never both. To specialize one, write an instance that binds some of its params and re-exposes the rest.
• Placement keys belong to the job (job_type, stage, needs, checkout, skip, runner, timeouts), execution keys to the component (script, setup, image). An execution key beside uses: is a load error.
• To let callers add behavior, a component declares a hook param and interpolates it: a contract, never a splice.

setup: runs script-only components before the job's own script:

setup:
  - uses: ./.pipemesh/components/aws-role.yaml
    with:
      arn: "arn:aws:iam::123:role/deploy"
      region: us-east-1
script: |
  aws sts get-caller-identity

Editor validation

Point yaml-language-server (built into the VS Code YAML extension) at Pipemesh's schemas with a modeline at the top of each file:

# yaml-language-server: $schema=https://pipemesh.io/api/meta/pipemesh-definition.schema.json

Use pipemesh-body.schema.json for .pipemesh/*.yaml bodies and pipemesh-component.schema.json for components. They reject unknown keys as the loader does, so a typo is a red squiggle before it's a load error. Declare the tags in your editor settings so they aren't flagged:

"yaml.customTags": ["!include scalar", "!ref scalar", "!settings scalar", "!var scalar"]

The editor reads a tagged value as a plain string; the loader resolves and checks it.