When your patching agent reaches for a live payload: proving fixes safely with canaries

I asked an agent to fix vulnerabilities and prove each fix works. It reached for rm -rf / and a reverse shell to do the proving. This is the story of how I stopped it: a safety contract baked into the skill, plus a purpose-built canary service that lets a test demonstrate an exploit path without ever firing a real one.

Introduction🔗

In my Deep Scan experiment (pairing deterministic scanners with an agentic triage-and-remediation loop) the scanning was never the scary part. The patching was.

Here’s the trap. When you ask an agent not just to fix a vulnerability but to prove the fix works, it needs a demonstration: show the sink was exploitable before the patch and is inert after. And the most direct way to demonstrate a command-injection sink is… to inject a command. Left to its own judgement, the agent writes exactly the test a pentester would: feed the sink ; rm -rf /, point a proof-of-SSRF at some URL, exfiltrate a fake secret to a domain it invented on the spot. You watch it draft that, and even in a sandbox it is genuinely a little 😱.

The lesson I took away is narrow but important:

An agent that patches security must be constrained not just in what it fixes, but in how it is allowed to demonstrate the fix. Demonstration is a side-effecting act, and side effects need a contract.

This post is about the two controls I added to make that safe: a hard safety contract encoded in the skill, and an inert canary service that gives the agent a benign, owned target to prove capability against. Both live in the open: the skill in the project’s .claude/commands, the canary at github.com/kubeek-sec/canary.

1. The problem: to prove a fix, the agent runs the exploit🔗

Why a demonstration needs a payload🔗

A good security fix should ship with a reachable-before / contained-after test: something that would have tripped on the vulnerable code and passes on the fixed code. That’s what makes a patch reviewable: the reviewer sees the proof, not just a claim.

The problem is that for a whole class of vulnerabilities, the honest proof is the attack:

Every one of those, done literally, means running real exploit code and/or reaching a host you don’t control. In a test suite. On a dev laptop. In CI.

What the agent actually proposed🔗

Unprompted, the agent’s demonstrations were technically excellent and operationally terrifying: destructive shell payloads to prove a shell-injection sink, and outbound calls to arbitrary external hosts to prove an exfiltration path. It wasn’t being reckless; it was being thorough. A real payload is the most convincing proof, so absent any rule, that’s what it optimized for.

The core insight. The agent isn’t wrong that a live payload proves the point. It’s wrong to think proving the point is worth the blast radius. That’s a judgement about acceptable risk, exactly the kind of thing you cannot leave implicit and hope the model shares your risk tolerance.

2. Control #1: a safety contract in the skill🔗

The first fix wasn’t code, it was policy, encoded where the agent reads it. The remediation workflow is a skill (a slash-command, .claude/commands/security-patch.md), and I rewrote the top of it as a non-negotiable contract the agent must honor on every patch.

Two things make it work: an explicit allowlist of the only side effects a demonstration may ever produce, and an explicit denylist of what is forbidden, in prose, before any procedure.

The allowlist, only two inert side effects are ever permitted:

1. Create a `<branch>.demo.txt` marker file whose contents are the branch name; or
2. Call ONE controlled kubeek.com canary endpoint via the canary helper,
   and nothing else.

The denylist, quoted from the skill:

STRICTLY FORBIDDEN. Never write, run, or commit any of:

• real exploit payloads, reverse shells, or data/credential exfiltration to any host other than the kubeek.com canary endpoints;
• destructive or state-changing commands: rm / rmdir, mv or overwrite of real files, chmod, kill, dropping DB tables, writing outside the repo;
• real secrets, or any change outside the current patch branch’s scope.

And the escape hatch that stops the agent “solving” the constraint by getting creative:

If you cannot demonstrate something without a real exploit, describe it in prose instead. Do not implement the exploit.

Put the constraint in the skill, not in your prompt-of-the-day. A skill is versioned, reviewed, and loaded every single run. A one-off “please be careful” in chat is not a control; it’s a wish. Moving the rule into the workflow file is what turns “the agent behaved this time” into “the agent is bounded every time”.

3. Control #2, the canary: an inert, owned target🔗

A contract tells the agent what it may do. But a marker file (<branch>.demo.txt) only proves local behavior. A whole class of vulnerabilities is about the network: SSRF, exfiltration, insecure download/“drop”. To prove those are contained you have to prove an outbound path was reachable and is now closed, which means the demonstration genuinely needs to make a request leave the machine.

So the missing piece was a benign, owned place for that request to go. I built one: canary, a tiny PHP service behind Cloudflare on a domain I control (kubeek.com). Its own README puts it plainly:

“Canary is a project to host some php & files on a website to facilitate your security tests.”

It exposes a handful of inert endpoints. Each one stands in for a capability an exploit would abuse, against a target that does nothing harmful:

That last row is the “managed dropped payload” idea. To prove a download/RCE-staging sink, an exploit would fetch and stage a real payload. Instead the agent fetches run.sh, whose entire body is echo "It runs!". The capability (I can reach out, download, and stage a file) is demonstrated; the payload is neutered by construction. The dangerous verb, none of the danger.

The helper is the enforcement point🔗

Documenting “only call these endpoints” isn’t enough. The agent could still hand-roll a curl to anywhere. So the endpoints ship with a canary helper (canary.sh / canary.ps1 / canary.py, same CLI on every OS) that can only reach the canary domain:

# Every call is on-domain and always sends a User-Agent, nothing else is reachable.
bash .claude/scripts/canary.sh info                 # GET  /info/
bash .claude/scripts/canary.sh monip                # GET  /monip/   (IP-echo beacon)
bash .claude/scripts/canary.sh direct   run.sh      # GET  /run/run.sh  (inert "drop")

The skill tells the agent to prefer the helper precisely because it can’t go off-domain; the tool shape enforces the policy the contract states. Belt and suspenders.

Watch out: two Cloudflare gotchas I hit building this, now baked into the helper so the agent never trips on them: Cloudflare 403s any request with no User-Agent, so the helper always sends one; and /info 301-redirects to /info/, and a followed 301 downgrades POST→GET, which silently corrupts a method-echo test. The helper targets the trailing-slash form directly.

4. Putting it together: reachable-before, contained-after🔗

With both controls in place, a demonstration test for a network-capable sink becomes safe and honest. The pattern: assert the unfixed path would have reached a canary, and the fixed path no longer does.

# A non-regression test for an exfiltration / SSRF sink, using the canary helper.
import sys; sys.path.insert(0, ".claude/scripts")
import canary

# Capability probe: prove the outbound path is reachable at all (inert target).
status, _ = canary.monip()
assert status == 200

# ...then assert the FIXED code path never calls canary.*, e.g. the sanitizer
# strips the user-controlled URL, so no outbound request is ever made.

The epistemics are identical to running the real exploit (the vulnerable path could reach an external host; the patched path cannot), but the blast radius is zero. No destructive command, no attacker infrastructure, no real secret. Every byte that leaves the machine goes to an inert endpoint I own and monitor.

And crucially, the agent never auto-merges. Each fix lands on its own branch, demonstration included, for a human to review. The canary and the contract make the demonstration safe; the review gate makes the decision human.

5. Why it takes all three (defense in depth)🔗

No single control here is sufficient on its own, and that’s the real takeaway. Prompt-injected code or a creative model could, in principle, try to route around any one of them. So they’re layered:

Each layer’s failure mode is caught by the next. The contract makes the right thing the default; the helper makes the wrong thing harder; the network fence (covered in the main Deep Scan post: non-root, read-only rootfs, default-deny egress, optionally gVisor) makes it impossible for anything the first two miss; and human review makes sure a demonstration is a demonstration, not an exploit in disguise.

Summary🔗

The four principles I’d carry to any agentic-remediation setup:

• Demonstration is a side effect: govern it as deliberately as you govern the fix itself.
• Encode the rule where the agent reads it: a skill file, versioned and loaded every run; not a hopeful sentence in chat.
• Give the agent a safe target: prove the capability against something inert and owned (echo "It runs!"), never the real thing.
• Layer it: contract, tooling, network fence, human review; each catches the previous one’s failure.

The scary payload wasn’t a bug in the agent. It was a missing contract. Writing that contract down (and building the inert target that makes it livable) is what turns “an agent that can patch security” into “an agent I’d let near a real repo”.

The canary endpoints are deliberately boring: echo "It runs!", an IP echo, a method beacon. That’s the point. The entire value is in what they prove (a path exists, then doesn’t), not in what they do. If you spin up your own, keep them inert, keep them on a domain you monitor, and never point a demonstration at a host you don’t own.