Several repositories, one pipeline

A service rarely lives in one repository, and its deployment has to know which commit of each it ships. In Pipemesh, one pipeline can declare the repositories it reads:

Every revision pins a commit of each.
A push to any of them starts a revision.
A job in one repository can consume an artifact built in another.

This guide follows pipemesh/demo-multirepo-pipeline, a repository that holds nothing but its pipemesh.yaml. The code lives in the two repositories it declares:

Repository	What it holds
demo-multirepo-orders	the orders service, and in `client/` the library other services call it through
demo-multirepo-billing	a service that calls orders through that client library

The client library crosses over as an artifact. orders_client builds orders-client.jar from orders' client/ directory, and billing builds against that jar without ever reading orders' sources.

What happens on a change

The push	orders_client	orders	billing	deploys
orders `service/`	reused	runs	reused	orders only
orders `client/`	runs	runs	runs, against the new jar	both
a comment in orders `client/`	runs	runs	reused: the jar is byte-identical	none: the jars are the same
billing `src/`	reused	reused	runs	billing only
the pipeline's own README	reused	reused	reused	none

Two job types make this hold:

A build (job_type: build) reuses (skip: built). When its fingerprint matches a stored run, it takes that run's jar instead of building again. That covers a revert, too.
A deploy (job_type: deploy) checks out nothing and consumes exactly one jar. It runs when that jar is new to it, and skips the one it already shipped.

How it works

repos: declares the other repositories by alias and URL. Each must first be added to the organization in Pipemesh (step 2). Pipemesh then watches its default branch like any other repository. It needs no pipemesh.yaml of its own.
A revision pins every repository: the commit of the pipeline's own repository and of each declared one.
repo: sets a job's workspace. A job with repo: orders checks out orders at the revision's pinned commit, and its checkout: lists files of that repository. A job without repo: works in the pipeline's own repository.
produces: and consumes: cross repositories. billing consumes orders_client/jar: the jar is mounted into billing's workspace, with its path in PIPEMESH_ORDERS_CLIENT_JAR. Consuming is also a dependency: billing waits for orders_client, and a deploy waits for its build.
Fingerprints follow the pinned commits. A job's fingerprint covers its definition, what it checks out in its repository, and what it consumes.

The definition covers repos:, repo:, produces: and consumes: in full.

Set it up

1. The repositories

Keep each service in its own repository, as today. The pipeline can live in one of them or, as in the demo, in a repository of its own.

2. Add every repository to Pipemesh

Open Repositories for the organization and add the code repositories. Each is listed in the catalog, and the pipelines that declare it appear under Used by. Then enable the repository that holds the pipeline.

3. The pipeline: `pipemesh.yaml`

The demo's definition, abridged. The jobs compile with javac; yours run whatever builds your services.

pipeline:
  type: pipeline
  repos:
    orders: github.com/pipemesh/demo-multirepo-orders
    billing: github.com/pipemesh/demo-multirepo-billing
  stages:
    - build
    - staging
    - production
  jobs:
    # The client library: works in orders, checks out only its client/.
    orders_client:
      job_type: build
      stage: build
      repo: orders
      checkout:
        - client
      script: |
        javac -g:none -d out/classes $(find client/src -name '*.java')
        jar --create --date=2026-01-01T00:00:00Z --file out/orders-client.jar -C out/classes .
      produces:
        jar: out/orders-client.jar

    # The orders service, built with the client sources it shares a model with.
    orders:
      job_type: build
      stage: build
      repo: orders
      checkout:
        - client
        - service
      script: |
        javac -g:none -d out/classes $(find client/src service/src -name '*.java')
        jar --create --date=2026-01-01T00:00:00Z --file out/orders.jar -C out/classes .
      produces:
        jar: out/orders.jar

    # Billing, in its own repository, built against the client jar orders produced.
    billing:
      job_type: build
      stage: build
      repo: billing
      checkout:
        - src
      consumes:
        - orders_client/jar
      script: |
        javac -g:none -cp "$PIPEMESH_ORDERS_CLIENT_JAR" -d out/classes $(find src -name '*.java')
        jar --create --date=2026-01-01T00:00:00Z --file out/billing.jar -C out/classes .
      produces:
        jar: out/billing.jar

    # Each deploy reads only the jar it ships.
    deploy_billing_staging:
      job_type: deploy
      production: false
      stage: staging
      consumes:
        - billing/jar
      script: |
        java -jar "$PIPEMESH_BILLING_JAR" --self-test
        echo "billing → staging"
    deploy_orders_staging:
      job_type: deploy
      production: false
      stage: staging
      consumes:
        - orders/jar
      script: |
        java -jar "$PIPEMESH_ORDERS_JAR" --self-test
        echo "orders → staging"
    deploy_billing_prod:
      job_type: deploy
      production: true
      stage: production
      needs:
        - deploy_billing_staging
      consumes:
        - billing/jar
      script: echo "billing → production"
    deploy_orders_prod:
      job_type: deploy
      production: true
      stage: production
      needs:
        - deploy_orders_staging
      consumes:
        - orders/jar
      script: echo "orders → production"

pipemesh:
  pipelines:
    pipeline: !ref pipeline

Two details to copy:

Reproducible jars. -g:none (no debug information) and --date=… make each jar byte-reproducible. A comment in the client compiles to the same classes, so the same jar, and nothing that consumes it moves. Do the same with whatever your build produces, or accept that a cosmetic change rebuilds its consumers.
Narrow checkouts. Each build's checkout: names the files it compiles. Its workspace holds only those, so its fingerprint can't miss one.

4. Watch the board

The Sources stage has a card per repository: the commit it's at, and when it was last polled.
The Revisions stage lists the revisions in flight. One started by a push to orders or billing names that repository.
A revision shows the commit of every repository it pinned, with the pull request that merged it. Compare it with the one before, or any other, to see which repositories moved and link to the diff on GitHub.
A job says why it ran or didn't: a build that reused a stored jar, a deploy that already had its jar. Its Artifacts tab lists what it produced and consumed, even when it reused or skipped. Its Revision tab shows the revision it ran at.

Try it

The pipeline is public at pipemesh.io/github.com/pipemesh/demo-multirepo-pipeline. Push a change to one of the three repositories and watch the table come true.