The lethal trifecta
Three capabilities that are harmless alone and dangerous together — and the three places I keep finding all three switched on at once.
An AI assistant becomes dangerous at the moment it holds three things at once: access to private data, exposure to untrusted content, and the ability to communicate outward. Simon Willison named this combination the lethal trifecta. The name has stuck because it describes something teams keep assembling without deciding to.
Each capability is unremarkable on its own. An assistant that reads your documents is useful. One that summarises a web page is useful. One that files a ticket is useful. The failure is not in any of them. It is in the intersection, and the intersection is rarely anyone’s design decision. It accumulates.
I find it helps to think of these systems as a prodigy intern. Brilliant, tireless, quick to learn, and completely unable to distinguish an instruction you gave from an instruction someone else left lying around for it to find. The intern does not lack competence at the task. It lacks the concept of provenance. Every sentence it encounters carries equal authority, whether it came from you, from a customer, or from a stranger who anticipated that a machine would read it.
There is no boundary between data and instructions, because the model has no concept of a boundary.
What follows are the three places I keep finding the full trifecta. They are not three different problems. They are the same problem at three levels of visibility — and the visibility drops sharply as you go down the list.
Where it starts: the prompt
The version everyone imagines is the direct one. Someone types something clever into a chat box and the model does something it should not.
That framing is comfortable because it implies an attacker who is present, typing, and identifiable. The uncomfortable version is indirect. The attacker never touches your system. They write the instruction into a document, a web page, an email signature, a calendar invitation, a support ticket — anywhere your assistant will eventually read. When it does, the text arrives as part of the conversation. Not as data about the conversation. As part of it.
This is why input filtering disappoints. You are not trying to catch a known pattern of malicious text. You are trying to enforce a distinction the architecture does not make. The model is not being tricked into ignoring the rules. It is following instructions faithfully. It simply cannot tell whose instructions they are.
Where it hides: the click-enabled feature
The riskiest control in most organisations right now is a small button in a SaaS product labelled something like Summarise with AI.
Consider what happens when someone presses it on a customer record. The feature reads content the organisation did not author. It runs with the permissions of the person who clicked. And in most current implementations it can act on what it finds — drafting a reply, updating a field, calling another integration. That is the complete trifecta behind a single click.
Nobody prompted anything. Nobody deployed anything. The feature arrived in a minor release, enabled by default, and was never reviewed because from a change-management perspective nothing changed. Your architecture review process is looking for projects. This was a checkbox.
The governance gap here is procurement, not engineering. If your vendor security questionnaire predates the current generation of assistants, it does not ask whether a feature reads untrusted content, what identity it runs as, or what it is permitted to do with the result. Those three questions are the trifecta, restated as procurement.
Where it scales: patch automation
The third place is the one I would look at first, because it is where the blast radius stops being theoretical.
Vulnerability management is an obvious candidate for automation. The data is structured, the volume is punishing, and the triage decisions are repetitive. So teams begin feeding scanner output to a model: exports containing hostnames, software versions, CVE references, and a precise inventory of every unpatched path in the estate. The model reads ticket text alongside it. And because the point is automation, it can write back — open changes, update records, trigger a pipeline.
Read that as three properties rather than one workflow. Private data: a map of your weakest assets, which is close to the most sensitive document your organisation produces. Untrusted content: scanner output includes fields populated from banners, certificates, HTTP responses, and other strings an attacker can influence, plus ticket text written by whoever filed it. Outbound capability: write access to the remediation pipeline.
Scanner output was never designed to be read as instructions. It is being read that way now. The assumption that a vulnerability export is inert data is inherited from a decade when the only thing consuming it was a dashboard.
What I would actually do
The trifecta is useful precisely because it is a removal exercise. You do not have to solve prompt injection. You have to ensure that no single context holds all three properties at once.
- Cut the outbound channel first. It is the one people defend hardest and the one that converts a disclosure into an action. An assistant that proposes a change for a human to approve retains most of the value and loses most of the risk.
- Separate the sessions. A context that reads untrusted content should not be the same context that holds privileged data. This is unglamorous architecture, and it works.
- Give the assistant its own identity. Running as the person who clicked means every permission that person has accumulated over six years is now reachable by anything the model reads.
- Add three questions to vendor review. What untrusted content does this feature ingest, what identity does it run as, and what can it do with the output. Those three answers tell you whether you have bought a trifecta.
None of this requires a new control category or a procurement cycle. It requires treating model-adjacent data flows as data flows, which is a discipline most security teams already have and have simply not applied here yet.
The prodigy intern is not going to develop judgment about provenance. That was never on the roadmap. What can change is how much you hand it in a single sitting.