Why Pipemesh
Two pains come back on every team that deploys software:
- Lack of control. It's hard to know what was deployed where, when, and by whom. The answer is spread across several views, and someone reconstructs it by hand. With agents landing pull requests many times an hour, nobody can.
- Rebuilds and redeploys of things that didn't change, wasting time and compute every day.
Many tools address one or the other from the outside. Pipemesh takes the view that the fix has to be structural, at the core of the CI/CD system.
The problem is structural
Most CI/CD systems deploy through release workflows. Workflows are good at publishing: an npm package or an OCI image goes to a registry, and the run is the record. They are terrible at deploying services. A release workflow carries rules that may skip the deploy of some components (a skip rule, or a branch of a dynamic workflow that didn't fire), and a skipped deploy hides the previous run that left a component half-deployed before it failed. The workflow is green; the service is inconsistent. Chasing that is a mess: retrying the old workflow deploys an outdated version, so you dig through other views to learn whether a newer run already fixed the environment, or whether you need to commit something just to make one run.
The cause is the shape. Workflow jobs are stateless tasks, and the state of an environment lives outside them. No run can say what's in staging now.
Live pipelines
A live pipeline, in the way Amazon defines it, is a long-lived structure for a project. Every main change (typically a merged pull request) becomes a revision, and revisions flow through the pipeline job by job. Each job remembers the revision it holds. A newer revision supersedes an older one stuck at a slow or failed stage. One board shows what each environment is running and which change is holding the pipeline, and the same answer is one API call away for people and agents.
Watch one live. demo-showcase merges a change every half minute, around the clock. Revisions flow through build, staging and production, a region fails now and then, and the fix supersedes it.
Pipelines have their limits too: they are suboptimal for the tasks where workflows shine. So Pipemesh is a hybrid. Pipelines deploy services. Workflows publish artifacts and run checks, one immutable run per trigger as in GitHub Actions and GitLab CI. Both live in one repository and one definition, and the control you want comes from the pipeline's state and a board built to show it.
| Use case | Kind | Why |
|---|---|---|
| Deploying a service through staging → production | pipeline | one deployment truth, gates, supersede |
| Releasing a library, CLI, or SDK on a tag | workflow | the run is the immutable release record |
| Building and publishing base images on a schedule | workflow | each run is one auditable publish |
| PR checks | workflow | one verdict per PR head |
| Nightly or scheduled test sweeps | workflow | independent, comparable runs |
The two work together:
- A pipeline job can run a workflow from the same repository, wait
for it and inherit what it built (
job_type: workflow). - A pipeline consumes what released workflows produce, pinned by digest.
- A monorepo's pipeline dispatches revisions to a child pipeline per
service, each on its own timeline (
job_type: pipeline).
See The definition.
Skipping what didn't change
The second pain is hard because you usually don't know whether anything a build or a deploy depends on has changed, so the safe move is to rebuild and redeploy everything. Rules like "deploy only when these files changed" aren't good enough: the build drifts away from the rule and jobs get skipped that should have run. Monorepos end up building elaborate graphs to answer the question at all.
But if a job's input really represents what the job will do, another run with the same input can be skipped: either the output artifacts already exist, or the deploy would change nothing. Pipemesh makes the input consistent with two requirements.
Artifacts are part of the contract. A producer declares what it
produces and a consumer declares what it consumes. Those declarations
form the graph, pinned by digest, rather than loose needs: edges,
which remain for a sequence without artifacts, such as "staging before
production".
Paths say what's checked out, not when to run. A job declares the repository paths it needs as glob patterns, and only those files are in its clone. The patterns aren't a rule about which commits touch them. So when the build drifts and starts needing a path the patterns don't cover, it fails, loudly, instead of silently skipping later when that path changes.
From that input Pipemesh computes a fingerprint and decides whether the job can skip.
Builds and deploys skip on different rules. A build can be skipped
when any earlier run, whenever it happened, succeeded with the same
input: the artifacts are reused. A deploy can't; nobody skips a deploy
because the same change was deployed long ago. A deploy skips only when
its input equals its last execution's. The ambiguity is removed by
declaring a job type: build, deploy, or a plain task that always
runs.
Keeping control when changes flow
Fifty revisions in flight should read as clearly as one. Here's how.
Revisions travel together. One line of revisions, not a run per merge. When three agents merge in a minute, you see how far each got and which is current at every deploy. A newer revision supersedes an older one waiting at a slow or failed stage, and nothing is lost.
Below, ten merges land seconds apart. A broken build and a failing region each hold their last good revision, and the newest good one ships.
Green means approved. A revision moves on only when every job it needs has run it, or proved it identical to what it last ran. A failed build holds everything downstream, even a docs-only commit on top (it contains the broken code), while independent deliverables still flow.
Artifacts flow with the revision. Jobs pass artifacts by digest with
produces: and consumes:, in a pipeline or across a workflow job. A
deploy runs only when its artifact is new to it, so the same bytes
rebuilt deploy nothing. A rollback redeploys what shipped, not a rebuild.
Gates and rollbacks live in the pipeline. A maintainer can disable promotions into a job, with a reason, and the board shows what it holds and why. A rollback re-deploys a revision the job already shipped, with its artifacts, and gates the job until someone lets newer ones in. The board also keeps the DORA numbers (deploys, lead time, change failure rate, time to restore) from the same records, so they're measured, not estimated.
Monorepos and many repositories. A monorepo keeps one definition and one mainline. A dispatch job per service sends each revision to that service's child pipeline when its inputs changed, judged by the build tool's own fingerprint (Bazel, Nx and Turborepo each have a component) or by the service's files.
A pipeline can also declare other repositories: each revision pins a commit of each, a push to any starts one, and artifacts cross between them. See Turborepo, Nx, Bazel and several repositories.
Run it anywhere. Jobs run on hosted runners, on your own runners, or as your existing GitHub Actions workflows: a pipeline job dispatches one, shows its jobs as tasks, and settles on its verdict. Keep the workflows you have, and move a job when you want. Jobs that need cloud credentials get a short-lived identity token, not a stored key.
One snapshot, for people and agents. Is the fix in production yet? shouldn't mean matching commits across your CI, deploy and inventory tools. For an agent, that's guessing.
In Pipemesh it's one API call: what was built, which revision each environment runs, what's queued, what a gate holds and why, and what changed since. It's recorded, not inferred: a revision shows in staging because the staging job recorded running it.
How it compares
| Pipemesh | Run-based CI (GitHub Actions, GitLab CI/CD) | Deploy tools (Argo CD, Spinnaker) | |
|---|---|---|---|
| What does a merge create? | a revision that moves through one long-lived pipeline (or a workflow run, for checks and releases) | a new, separate run | a sync of whatever it's pointed at |
| What's in staging right now? | each job records the revision it holds: one view, one API call | piece it together from the run list or an environments page | known, for what the tool itself deploys |
| Ten merges in a few minutes? | one line of revisions; the newest supersedes the ones queued behind it; a failure holds only its own lane | ten runs; find which later one carried your change | not its concern: it deploys what it's given |
| When can a job skip? | when its inputs match its own last successful run, so a change that failed to build is never skipped past | when path filters see no change since the previous commit, even if that commit's build failed | not applicable |
| How do artifacts move between jobs? | declared with produces: and consumes:, pinned by digest, across repositories |
uploaded and downloaded by name, within one run | pulled from a registry |
| Does an unchanged artifact redeploy? | no: same digest, the deploy skips on its own | yes, unless you write a digest check yourself | no: it syncs only when the desired state changes |
| How do you hold or roll back a deploy? | hold promotions into a job with a reason; roll back to the artifacts a revision shipped | environment protection rules; rolling back means re-running an old run | built in |
| Where do jobs run? | hosted runners, your own runners, or your GitHub Actions | the vendor's runners, or self-hosted ones | your cluster |
| Which monorepo services build? | the ones the build tool's own fingerprint says changed (Bazel, Nx, Turborepo) | the ones path filters match | not applicable |
Where others are stronger:
- GitHub Actions and GitLab have the marketplaces, editors and defaults, and most teams' working workflows. That's why a Pipemesh job can run on those workflows rather than replace them.
- A deploy tool that reconciles a cluster knows the cluster's real state; a pipeline doesn't. Pipemesh shows what the pipeline deployed, which can be wrong after a change made outside it.
- Bazel, Nx and Turborepo know their build graphs better than any CI layer, so Pipemesh asks them rather than guessing.
One definition
Everything a repository runs is declared in pipemesh.yaml at its
root, under one reserved key, pipemesh:: the pipeline under
pipelines:, releases and checks under workflows:, each with its
triggers. The rest of the file is yours.
Meshes lists every pipeline and workflow, grouped by repository. Each one gets a URL and a run history, and each pipeline a live board. Start with Getting started and The definition.