Dependabot and Renovate do one job well: notice the new version, open the bump. By design they never read your code — so when that bump turns the suite red, the pull request just sits there. Patchery starts exactly there. It reproduces the break, works the fix, re-runs your tests, and then grades its own evidence — publishing how far the proof actually goes, and refusing to ship a change it could not prove.
Keep Dependabot. This is the layer above it.
It pins your repository, applies the upgrade, and runs your test suite. If the failure is not real and repeatable, it stops here — there is nothing to fix.
An agent reads the failure and the changelog, and migrates the call sites. It works inside your checkout, with your tests as the only judge of whether it is done.
The patch ships with its evidence: the failing run before, the passing run after, the files it touched, and a read-only second agent's review. And it says how far that proof goes — a change nothing mechanical ever exercised opens as a draft marked Not verified, instead of borrowing a header it did not earn.
Some breaks have no fix at the call site. A dependency that starts shipping as an ES module renames nothing — your code is fine and can no longer load it. Patchery reports which call sites are affected, why the obvious fix does not apply, and which decision would unblock it. Where the fix does not exist at the call site, that report is what is left — a fallback, not the thing Patchery is for.
Any capable model can produce a patch that looks right. The question is what happens when it is wrong — and the answer most tools give is: it opens the pull request anyway. Patchery ends a run in one of seven states, and it would rather end in the second one than guess.
The break is gone, your tests pass, and the evidence is attached.
It found something it could not prove, so it sent nothing and said which check stopped it.
No change to your code fixes this one — the analysis names what would, and the decision is yours. It counts in the denominator: the run happened, and calling it a non-event would be scoring ourselves generously.
It finished its turn and had nothing worth sending.
The turn budget ran out mid-investigation. It counts against the score: a run that does not finish has not helped anyone.
It sent a patch that does not hold up. This is the number that has to stay honest.
The run could not get far enough to be measured — a build, a container, a model that stopped answering. It says nothing about the fix, so it is named rather than folded into a score. Which of these belong in a denominator is a live argument, and it is being had in the open.
An agent that declines to send what it cannot prove is doing the job. An agent that sends it anyway is the risk you were trying to remove. A report that merges the two is measuring some other product.
Tests are counted while the code is still whole — before anything is broken, so nothing the agent does afterwards can move that baseline. They are counted again after its turn. If fewer tests pass than at the start, the change is reverted and the run ends REFUSED, however green it looks — nothing reaches a pull request, which is the point.
Patchery is not a service you send a repository to. It is a file in your project that runs on your own runners, with your own key, against your own checkout. We host nothing and store nothing, so there is no copy of your source anywhere to ask us for.
What does leave the runner is what a model has to read to do the work: the failing test
output, the files the agent opens, and the diff it writes. That goes to the endpoint you set
in anthropic-base-url and nowhere else — point it at your own deployment
and nothing reaches a third party at all.
A breaking upgrade in Patchery's own repository, fixed end to end by the action. The pull request carries the test runs from either side of the fix and the second agent's review — which raised concerns, and says so at the top of the page.
Candidate upgrades are pulled from public projects, and each one is confirmed to actually turn that project's own suite red before Patchery is allowed near it. That list is in the repository, not in a slide.
Still no success ratio here, deliberately. The first batch ran while the rules for what counts as a failure of the product versus a failure of our harness were still being fixed — and a number produced under rules that changed mid-run is not a measurement. The full table goes here when a set has run under one set of rules: every case, including the ones it could not fix and what stopped it.
What is already measured is the shape of the problem: 11 of the 14 are packaging
breaks, only 3 are changed signatures. "Did the suite break" is a trigger that selects
for packaging — express v4 → v5 is one of the best-known breaking
changes in the ecosystem, and across five repositories that depend on it, not one suite
noticed.
ANTHROPIC_AUTH_TOKEN..github/workflows/patchery.yml.No scheduler and no background scanning: it runs when you dispatch it. Scheduled runs are the next thing on the list.
name: Patchery
on:
workflow_dispatch:
inputs:
package: { required: true }
target-dir: { default: "." }
test-command: { default: "npm test" }
permissions:
contents: write
pull-requests: write
jobs:
fix:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
working-directory: ${{ inputs.target-dir }}
- id: patchery
uses: patchery-dev/Patchery@v0.2
with:
package: ${{ inputs.package }}
target-dir: ${{ inputs.target-dir }}
test-command: ${{ inputs.test-command }}
anthropic-auth-token: ${{ secrets.ANTHROPIC_AUTH_TOKEN }}
- if: steps.patchery.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v7
with:
branch: patchery/${{ inputs.package }}
body-path: ${{ steps.patchery.outputs.pr-body-file }}
add-paths: ${{ steps.patchery.outputs.files }}cors, multer, express-session, formidable — and not one suite went red. Their tests never reach the paths that changed.