PatcheryGitHub ActionnpmMITearly
When a dependency breaks your code, Patchery fixes it and proves the fix.
Your project runs on other people’s code. When they change how it works, your app stops working. Patchery finds what broke, rewrites the lines that need rewriting, and checks the result against your own tests before it shows you anything — and throws its own work away if that check goes badly.
- 1pull request it opened on its own
- $0.2251the last run, priced at list rates
- 0users, customers, logos
- MITevery line of it public
Tell an AI to make the tests pass. It might just delete them. // 4 tests removed · suite green
Deleting the test is the cheapest way to satisfy the instruction, so it is the one an agent reaches for first. Failing that: soften the check, stub out the function, edit the very file that decides what working means. Every one of those makes a broken project look fixed.
Which means writing the fix was never the interesting problem. Everything that happens after it is. Patchery never reads the AI’s own account of what it did — it looks at the files themselves, throws the whole attempt away if anything off-limits moved, and re-runs your tests itself.
The libraries underneath modern software change faster than the teams using them can follow. Tools like Dependabot are good at telling you a newer version exists. They stop exactly where the work starts: the places in your code that used the old way.
That gap used to need a person. It stopped needing one somewhere in the last two years — but only if you can trust the result, which is the same problem again.
openai and
@google/generative-ai — both migrated by hand, both linked below.
In plain termsIt tries the fix, then spends four more steps trying to catch itself cheating.
It has to get
past itself first.
- 01Check it’s really brokenIf your tests already pass, stop here and charge nothing.
- 02Read the notes, fix the codeFind what changed, rewrite the lines that used the old way.
- 03Look at what changed on diskNot what the AI says it did — what the files say it did.
- 04Run your tests againMeasure it. Don’t ask it.
- 05Get a second AI to argue with itA different call, read-only, that tries to refute the fix.
- 06Ask a human to approve itA change request you read and merge yourself. Never automatic.
$ npm test
> toy-project@1.0.0 test
> node app.test.js
node_modules/fake-lib/index.js:3
throw new TypeError("currency is required as of fake-lib@2.0.0");
^
TypeError: formatPrice(amount, currency): currency is required
at formatPrice (node_modules/fake-lib/index.js:3:11)
at renderCartTotal (app.js:4:20)
at Object.<anonymous> (app.test.js:4:16)# fake-lib changelog
## 2.0.0 (breaking)
- Removed formatPrice(amount).
Returned a plain number and rounded
inconsistently across locales.
- Added formatPrice(amount, currency).
Now requires an explicit currency
code, e.g. "USD".
### Migration
formatPrice(19.9) // "19.90"
formatPrice(19.9,"USD") // "$19.90"1 const { formatPrice } = require("fake-lib");
2
3 function renderCartTotal(amount) {
4 return `Total: ${formatPrice(amount)}`;
5 }
6
7 module.exports = { renderCartTotal };changed on disk:
app.js allowed
checked, and untouched:
your tests off-limits
package-lock.json off-limits
node_modules/ off-limits
your CI configuration off-limits
→ nothing off-limits moved. carry on.$ npm test
> toy-project@1.0.0 test
> node app.test.js
PASS: app.test.js$ patchery review --read-only
reviewing 1 file, 2 lines
separate call · cannot edit · never shown
the first agent's reasoning
verdict not refuted
confidence 72 / 100
concerns raised
cost $0.3298fix(deps): migrate fake-lib call sites
a change request, waiting for a human
app.js
function renderCartTotal(amount) {
− return `Total: ${formatPrice(amount)}`;
+ return `Total: ${formatPrice(amount, "USD")}`;
}- 1 file
- +1 −1
- 9 turns
- $0.2251
Opened automatically. Never merged automatically.
In plain termsSome files are off-limits, and touching one throws away the whole attempt — including a fix that worked.
Try to
break it.
The off-limits list is short: your tests, the file that decides which of them run, the file that pins your versions, your build settings, and the installed libraries themselves. Those are the ways an AI can make a broken project look fixed without fixing anything, so none of them may move. Two more rules sit alongside them — nothing may be deleted, and nothing outside the folder you pointed it at may be touched at all.
The box below runs the real check — the same few lines that run inside Patchery’s source, copied character for character, not a demo of it. Give it any file path and see what it decides.
or try one of these
Nothing here yet. Add a file and the verdict appears below.
Six rules on the off-limits list, in the order they run — installed libraries, test files, test folders, build settings, test-harness configuration, version-lock files — plus a ban on deletions and on anything outside the folder you pointed it at. All of it is covered by a test that runs on every change to Patchery itself: offline, in a second, with no AI and no key.
It reads which files moved. It never reads what the change did to them. An AI
that deleted a validation check inside src/auth/validate.ts — a file it is
perfectly entitled to edit — walks straight past everything above. Try that path in the
box and watch it be allowed.
Three things stand behind the hole. Your tests are run again afterwards, by Patchery and not by the AI, so a change that breaks them never reaches a pull request. A second agent then reads the diff and tries to refute the fix — it gets no write access, no sight of the first agent’s reasoning, and a separate call, so it is reviewing the change rather than agreeing with itself. And the pull request still goes to a person: there is no auto-merge and there should not be.
None of that closes the hole. Reading a diff is judgement, and judgement is the thing this check exists to avoid having to trust. The rule above is deterministic and the review is not, which is why the review is a second line and never the only one.
In plain termsEverything below is a link you can open and check for yourself, right now.
It has done this
with nobody watching.
One line, in one file, in a real repository
A library called fake-lib went from version 1 to version 2 and made an
argument required that used to be optional. Patchery ran the tests (they failed), read
what the library had changed, rewrote the one line that needed it, checked which files
it had touched, ran the tests again (they passed), and opened this pull request. The
first human to see it was the reviewer.
Two things about those numbers. The cost is what those turns would come to at Anthropic list prices; the run used a cheaper model, so the amount actually billed was different. And they are the latest run, not the only one: the broken fixture is committed broken on purpose so the demo can be run again, and each run replaces the same branch. Four have succeeded so far. The figures are read out of the pull request itself when this page loads, so they cannot quietly go stale — they already did once, which is why they are read rather than typed.
- files changed1
- lines+1 −1
- testsfailed → passed
- AI turns used9
- cost, at list rates$0.2251
- modelglm-5.3
And here is the same job done by hand, on two projects we don’t own. No AI was involved in either — they are here because doing this work manually is how we learned what the automated version has to survive.
An SDK that moved on without them
The project was still built on a three-year-old way of calling OpenAI, long since replaced. Three places in one file had to be rewritten, along with how errors and responses are read back.
OpenA library that reached its end of life
Their Gemini integration depended on a Google library that stopped being supported on 30 November 2025. Setting up the client and both ways of querying it had to move to the replacement.
OpenAll three are still open. None has been merged. We will change this sentence the day that changes.
In plain termsWe pointed it at real projects and it came back with nothing. Here is every run we kept a proper record of, what it spent, and what it wrote.
Four runs on code we don’t own. Four times it wrote nothing.
Every bar below is one turn of an AI actually thinking, and every one of them was billed. Read the number at the end of each row before you read anything else on this page.
-
ran, wrote nothing giancarloerra/SocratiCode An old library that hadn’t actually broken anything yet
The tests were red, so it started. It read the failures one at a time and worked out that every one of them was about the machine we were running on, not about the library we had sent it to fix. It checked that the old library really was still installed, checked that nothing was breaking when the code actually ran, and then declined to change anything. It spent the rest of its turns holding that position. The easy move was there the whole time and it didn’t take it.
- turns
- 25
- cost
- $0.88
- lines written
- 0
-
ran three times, wrote nothing dwmkerr/terminal-ai Not a renamed function — a different idea of where a conversation lives
We gave it fifteen turns. It read the migration guide, read the files that would have to move, and checked which parts of the new library actually existed — a different, sensible step every turn, never the same one twice. Then Patchery cut it off. Our own watchdog counts turns that change no files, decided this one was going in circles, and fired at the moment it was about to start writing. We raised the limit to twenty-two and it went further, comparing against the project’s existing style and working out that the right shape was the one that keeps no state. Cut off again, at the same moment. At forty it went further still — following how a single message travels through the code, then going to look at whether the project’s own maintainers had already done this upstream — and finished with nothing written. Three runs, each deeper than the last, not one wasted turn. The job was simply bigger than the shape we had given it.
- turns
- 15 · 22 · 40
- cost
- not recorded
- lines written
- 0
And each of those three attempts paid to read the same files from the beginning, because nothing is carried from one run to the next. That is a cost bug and it is still open. The watchdog that cut these runs off is not: it has stopped counting turns without an edit and now counts turns that turn up nothing new, so reading four files it has never read is progress rather than idling. These three runs are the reason it changed.
-
ran, never reached the AI gitroomhq/postiz-agent A real, reported bug that had already healed on our machine
Someone had filed it: a library had moved to a newer module format, the old way of loading it threw, and the person who reported it could reproduce it. Patchery runs your tests before it does anything else, and they passed — so it stopped and said there was nothing to fix, which is exactly what it is supposed to do when the tests are green. The reason they were green is that we ran on a newer version of Node than the reporter did. Node began supporting that combination in 22.12; they were on 22.11, one release below the line. The bug is real and is still real for them. On our machine it had already healed. Whichever version your tests run on decides what Patchery is able to see, and there is no version we could pick that would be the right one for everybody.
- AI turns
- 0
- cost
- $0.00
- lines written
- 0
-
It builds with a package manager we didn’t have installed, and when we installed only the piece we needed, the shared parts it depends on were left unlinked, so nothing could be checked. Behind both we found that the file we had come to fix has no tests at all — which means Patchery would have had nothing to measure a fix against, and refusing would have been the correct answer anyway. A file-path limit stopped the checkout before any of that, but that one was ours: a Windows setting, fixed in a line, and it cannot happen on the machines this actually runs on.
- turns
- 0
- cost
- $0.00
- lines written
- 0
Not one of them
is a wrong fix.
In every run above there was an obvious way to look productive: write something plausible, get the tests to green, open the pull request. It never did. Worth being exact about why, because the exact answer flatters us less: in these four runs the off-limits check and the second test run never came into it, because nothing was ever written for them to judge. What kept a wrong answer off the screen was the AI declining to guess — and, twice, our own watchdog stopping the run before it could write anything at all, which is not a save we get to claim. The gates further down have only ever been tested on runs that produced something.
Two more never became runs at all. On one, someone had already opened a pull request doing exactly this migration, so we closed the tab. On another, the maintainers had already decided against the change on purpose — so we left them the type-by-type mapping we had prepared as a comment, and no pull request. Checking whether a change is wanted before writing it is the cheapest step in the whole process, and the one an automated system is most likely to skip. One more was dropped before it ever became a run: a real break nobody had fixed, one open issue, no pull request against it — and, in five hundred and fifty-nine files, not one real test. Patchery decides whether a fix worked by running your tests. Where a project has none there is nothing to measure against, so it has nothing to offer and walking away is the correct answer. That is now twice on this page, and it is the most common thing that stops it.
In plain termsYou copy one file into your project, and nothing happens until you press a button.
One file in,
one workflow out.
Patchery is not a service you sign up for. It is a file you put in your own project, running on your own machines, with your own key — which is also why there is nothing for us to charge you for yet.
- 01Put your AI key in the repository’s secrets, as
ANTHROPIC_AUTH_TOKEN. - 02Save the file on the right as
.github/workflows/patchery.yml. - 03Start it from the Actions tab. You choose which library it works on.
No scheduler, no background scanning. Nothing runs unless you press the button — that is deliberate, and it is the next thing to change.
Full setup guidename: Patchery
on:
workflow_dispatch:
inputs:
package: { required: true }
target-dir: { default: "." }
test-command: { default: "npm test" }
permissions:
contents: write
pull-requests: write
jobs:
fix:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
working-directory: ${{ inputs.target-dir }}
- id: sma
uses: patchery-dev/Patchery@v0
with:
package: ${{ inputs.package }}
target-dir: ${{ inputs.target-dir }}
test-command: ${{ inputs.test-command }}
anthropic-auth-token: ${{ secrets.ANTHROPIC_AUTH_TOKEN }}
- if: steps.sma.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v7
with:
branch: patchery/${{ inputs.package }}
body-path: ${{ steps.sma.outputs.pr-body-file }}
add-paths: ${{ steps.sma.outputs.files }}We’d rather you audit us
than trust us.
True today
- Anyone can install it right now, and read every line of it. MIT licensed.
- One pull request opened start to finish with no human in the loop.
- The off-limits check is tested on every change, offline, with no AI.
- Keys and passwords are stripped out before anything is written down.
- Works on JavaScript projects. You decide what counts as a passing test.
- Four runs on projects we don’t own produced nothing — and nothing wrong. It has never invented a fix.
- A second agent reviews every fix before it ships. It cannot write, and it does not see the first one’s reasoning.
Not yet
- Nothing hosted. It runs on your machines, not ours.
- No revenue, no users, no logo wall. Nobody is paying for this.
- No open-source pull request accepted yet — the two below are waiting.
- One library at a time. It won’t untangle a chain of breakages.
- Other languages are plausible, but nobody has proven it.
- The off-limits check still reads file names, not changes. The second agent reads the change, but that is judgement, not a rule.
- The cheap model it runs on today sends malformed requests that its provider happens to accept. It works; it is not solid ground.
So here is the page checking itself.
Everything below is written as something that can fail. They run against the real files and the real GitHub API, not against a recording. If one of them ever turns red on this page, the page is wrong — and it will be the first to say so.
Every number in the run and the proof comes from something you can open and read. The numbers in what stopped it do not — those runs happened on a laptop and left no public record, which is exactly why they are written out here in full rather than summarised. And one thing worth saying plainly: the AI reads your code. Check what your model provider does with it before you point this at anything private. The full list of limits →