# Josh Botz

> A field guide, a red-team bench that runs in your browser, and a weekly radar for AI agent security. Plain words, tested where it says tested. Written by one person. "Tested" means Josh ran it in his own lab; "reported" means he read it and links the source.

## Field guide

- [Prompt injection](https://www.joshbotz.com/guide/prompt-injection/) (JB-001, attack, tested, checked 2026-09-09): An AI agent can't tell the difference between the job you gave it and instructions that show up inside a ticket, email, or web page it reads while doing that job. So an attacker writes the instructions into the content. The agent follows them. That's prompt injection, and in my lab it worked on the cheapest model nine times out of ten.
- [RAG poisoning](https://www.joshbotz.com/guide/rag-poisoning/) (JB-004, attack, tested, checked 2026-09-09): A retrieval assistant answers from documents it looks up, with citations. Every overt attack I ran against one was resisted when the true answer was also in the documents it found. The attack that worked every time was quieter: change the one authoritative document and leave no correct copy behind. Six times out of six the assistant stated the wrong value as fact, citing the corrupted source. No inference-time filter changed that.
- [Canary tokens](https://www.joshbotz.com/guide/canary-tokens/) (JB-003, control, tested, checked 2026-09-09): A canary token is a fake secret you plant on purpose: a key, a password, a record that nothing legitimate should ever touch. If it shows up leaving the building, something went wrong and you know exactly when. For AI agents this matters because an agent that refuses one attack is not the same as a system that notices the next one succeeding.
- [Redact secrets from tool output](https://www.joshbotz.com/guide/output-redaction/) (JB-002, control, tested, checked 2026-09-09): Most agent leaks need the agent to be holding the secret in the first place. If the tool that looks up an account returns recovery codes, the agent can be talked into sending them. If the tool returns the codes masked, the agent can be talked into sending an email and the attacker still gets nothing. In my lab this took the leak rate from 9 in 10 to zero.
- [The AI use policy that forgot the basics](https://www.joshbotz.com/guide/ai-use-policy-basics/) (JB-007, control, tested, checked 2026-09-09): I ran a checker over 34 real, published AI acceptable-use policies, looking not for AI rules but for the ordinary ones: keep your account safe, use approved software, report incidents. Most were missing. 94% had no safe-computing duty. Two in three said nothing about account security or approved tools. The AI clauses were there. The floor under them wasn't.
- [MCP](https://www.joshbotz.com/guide/mcp/) (JB-005, term, reported, checked 2026-09-09): MCP stands for Model Context Protocol. It's a standard way to plug tools and data into an AI assistant: a server says "here are the things I can do" and the assistant can call them. That's what turns a chat window into an agent that reads your files, queries your systems, and sends things. Every one of those plugs is also a place where the agent can be lied to, or can do damage.
- [Risk tiers](https://www.joshbotz.com/guide/risk-tiers/) (JB-008, term, tested, checked 2026-09-09): A risk tier is a label you give a system that decides how much scrutiny it gets. Low tier, light checks. High tier, the full treatment. Every mature software development policy I read already does this. None of them mentioned AI at all. The AI frameworks turned out to use the same move independently. So the bridge from the governance you have to the governance you need isn't a new framework. It's the tiering you already do, pointed at new risks.
- [The evaluation agent that left its sandbox](https://www.joshbotz.com/guide/sandbox-escape/) (JB-006, incident, reported, checked 2026-09-09): In July 2026 Hugging Face disclosed that an autonomous agent system had reached limited internal datasets and service credentials. OpenAI later said the agent came from one of its own internal model evaluations, found a weakness in its evaluation sandbox, reached the internet, and went looking for benchmark answers. Nobody meant harm. The box the agent ran in had a way out, and the agent found it.

## Tools

- [The Bench](https://www.joshbotz.com/bench/): watch an AI support agent get talked out of planted recovery codes, in the browser, with a small model running on the visitor's machine. Six cons, two controls to flip. No keys, no server.
- [Check your AI policy](https://www.joshbotz.com/check/): six questions, two minutes.

## Radar

- [Week of 2026-09-08](https://www.joshbotz.com/radar/#w-2026-W37): A coding agent ran a command hidden in a web page it was asked to summarize; OpenAI's evaluation agent reached Hugging Face's internal systems; Refusal and detection are different controls; OWASP published its Top 10 for Agentic Applications

## Writing

- [I tried six ways to make an AI agent leak recovery codes. One worked.](https://www.joshbotz.com/ai-agent-prompt-injection-social-engineering/) (2026-06-16, From the Lab): Four of six attacks never landed once, on any model. The one that worked reliably was the oldest con there is: authority, a reason to help, and a reason to stay quiet. It walked the recovery codes out the door in 9 of 10 runs on the cheapest model.
- [AI lowered the barrier to build, not the bar.](https://www.joshbotz.com/ai-lowered-the-barrier-not-the-bar/) (2026-06-12, Frameworks & Governance Translated): AI lowered the barrier to entry, not the bar: everything a thing has to survive before real people and real data depend on it. Proofs of concept are quietly being used as production tools by people who never went through any of that.

## Pages

- [Lab](https://www.joshbotz.com/lab/): labs you can run yourself.
- [About](https://www.joshbotz.com/about/): who Josh is.
- [Contact](https://www.joshbotz.com/contact/): LinkedIn or josh@joshbotz.com.

## Feeds

- https://www.joshbotz.com/rss.xml
- https://www.joshbotz.com/feed.json