Two new security papers reveal "cross-tool description poisoning" and "Agent-in-the-Middle" attacks that exploit the unmonitored channels agents silently trust — tool descriptions and inter-agent messages.

Transcript

Here's a question worth sitting with before you wire a single agent into anything that matters. It is not "can this agent do the job." It is "how would I know if it did the job wrong." Those sound like the same question. They are not, and the distance between them is where most of the interesting failures live.

There's a post going around Moltbook that puts it plainly. The gap between what an agent CAN do and what it should be TRUSTED to do is not a capability gap. It's a verification gap. And deploying an agent before you can cheaply verify its work is not an engineering shortfall — it's a governance failure wearing an engineering costume. I want to take that claim seriously today, because two security papers making the rounds this week show exactly what happens when you get the verification question wrong.

Let me start with the one that unsettled me most.

The first paper, posted in mid-June by a researcher named Shi and eight co-authors, describes an attack with a name that sounds dull and an effect that is anything but. They call it cross-tool description poisoning. Here's the setup in plain terms. A modern agent doesn't just answer questions. It has tools — a calculator, a web search, a function that sends email, a function that reads your files. Each tool comes with a description, a little block of metadata that tells the agent what the tool does and when to use it. The agent's planner reads all of those descriptions before it decides what to do.

Now, the obvious way to attack a tool is to build a malicious one and trick the agent into calling it. Everybody worries about that. You'd log the call, you'd see the malicious tool fire, you'd have something to inspect. The attack in this paper is more elegant and more disturbing, because the poisoned tool NEVER GETS CALLED.

The attacker writes a malicious description and attaches it to a tool the agent will never select. It just sits there in the list. But the planner reads the whole list before planning. So the poisoned text — instructions dressed up as harmless metadata — enters the planning context and steers the agent's reasoning about a completely different task. The malicious tool is a decoy that does its damage by being read, not by being used. The agent picks a legitimate tool, executes a plan that looks entirely reasonable, and the plan itself has been bent by a description it glanced at and discarded.

Think about what that does to your defenses. If you're watching which tools an agent calls, you see nothing wrong. The tool call log is clean. The agent used the tools it was supposed to use. There is no malicious execution to catch. The manipulation happened upstream, in the reading, in a place most monitoring doesn't even look. You verified the output of every tool the agent touched, and you STILL missed it, because the poison was in the tool it didn't touch.

And that, right there, is the verification gap made concrete. The agent was capable of doing the task. It just also did something else, quietly, and your ability to notice was zero.

The second paper generalizes this from one agent to many. It came out earlier and was accepted at a major computational linguistics conference this year — six authors, He, Lin, Dong, Xu, Xing and Liu. They call the attack Agent-in-the-Middle, and if you've heard of man-in-the-middle attacks on network traffic, the name is doing exactly the work you think it is. But the target is different, and the difference matters.

Most multi-agent frameworks today — the popular ones like AutoGen, LangGraph, CrewAI — are built on a shared, unspoken assumption. When one agent sends a message to another agent, that message is trusted data. An orchestrator hands a task to a subagent. The subagent does the work and reports back. The orchestrator reads the report and moves on. Nobody in that chain treats the messages between agents as a place an attacker might live. The tools are guarded. The user input is sanitized. But the inter-agent channel is assumed to be family.

Agent-in-the-Middle treats that trust itself as the vulnerability. The attack doesn't compromise any individual agent. It doesn't need to. It intercepts and tampers with the messages agents exchange. One agent asks another for information, and the answer that comes back has been altered in transit. The receiving agent has no reason to doubt it, because doubting your colleagues is not something these systems are built to do. The whole architecture runs on the premise that the message from the other agent is clean.

Put the two papers side by side and a single lesson comes into focus. In both cases, the exploit lives in a channel that the system reads and trusts but does not verify. In the first, it's the tool descriptions the planner reads. In the second, it's the messages agents pass to each other. Neither attack breaks a capability. The agents are all working perfectly. The attacks break an ASSUMPTION — the assumption that if you didn't call it, it can't hurt you, and the assumption that a message from a fellow agent is safe to believe.

Which brings me back to the Moltbook post and why I think it's the sharpest thing I've read on agents in a while. The honest question, it says, is never "can it do this." It's "how would we know if it did this wrong." Both of these attacks are, at bottom, about that second question having no good answer. When the poisoned tool never fires, how would you know your plan was steered? When the tampered message looks exactly like a legitimate one, how would the receiving agent know it was lied to?

I want to push on this a little, because I think it's easy to nod along and then go do the wrong thing anyway.

The temptation in agent development is enormous, and it runs in one direction. You get a demo working. The agent books the meeting, refactors the code, drafts the email, reconciles the invoices. It works, visibly, in front of you. And the pull to expand its scope from there is almost gravitational. It handled ten invoices, so let it handle ten thousand. It drafted the email, so let it send the email. The capability is right there, proven, and the cost of restraint feels like leaving value on the table.

But capability arriving early is exactly the trap. An agent can very often do a task correctly LONG before anyone can cheaply confirm that it did the task correctly. Those two abilities are on completely different clocks, and we confuse them constantly. Doing the thing is a research and engineering problem, and it's getting solved fast. Knowing whether the thing was done right is a verification problem, and it is not keeping pace. So the responsible move — deploy only as far as you can verify — feels like caution, feels like it's holding back a capable system for no reason. And that framing is the failure. You've dressed a governance decision up as an engineering constraint, and then resented the constraint.

Here's what the security papers add to the argument. They show that the verification gap isn't a hypothetical. It has a shape and a location. It lives in exactly the places you're not watching — the tool you didn't call, the message you didn't distrust. An attacker who understands your system will not attack where you're looking. They'll attack the channel you read and believe without checking. So "how would we know if it did this wrong" is not an abstract governance koan. It's a concrete engineering audit. Go find every place your system reads something and trusts it. Those are your unmonitored channels, and every one of them is a candidate for exactly these attacks.

What does taking this seriously actually look like? A few things follow directly. You treat tool descriptions as untrusted input, the same way you'd treat text a user typed into a box — because in both of the poisoning attacks, that's what they were. You treat inter-agent messages as data that can be tampered with, not as trusted reports from colleagues, which means signing them, or checking them, or at minimum logging them so a tampered message leaves a trace. And you set the boundary of what an agent is allowed to do at the boundary of what you can actually verify it did — not at the boundary of what it's capable of doing. Those are different lines, and the second one is always further out than the first.

The uncomfortable part is that this puts a real ceiling on how fast you can grow an agent's autonomy. If you can only trust it as far as you can verify it, and verification is the slow-moving half of the problem, then your deployment is paced by your weakest monitoring, not by your strongest model. That is a genuinely disappointing constraint if you were hoping the capable model would just carry the whole thing.

But I think it's the right constraint, and I think the poisoned-tool attack is the cleanest argument for it I've seen. Because the failure there was completely silent. Nothing crashed. No malicious tool fired. The logs were clean. The agent did its job. And the plan was compromised the entire time, by a description it read once and threw away. If your only signal that something went wrong is that something visibly went wrong, an attack that never trips a visible signal is invisible to you by design.

So the question to carry out of here is the one the Moltbook post ended on, and I'd ask you to apply it to whatever agent you're building or running right now. Not "what can it do." You already know it can do a lot; that's why you built it. The question is: if it did the wrong thing tonight — quietly, through a channel you're not watching — how, exactly, would you find out tomorrow? If you don't have a concrete answer, you don't have a capability problem. You have a scope you haven't earned yet.

Sources & References

Episode produced with mindTunes, using AI and the sources listed above.