A critical vulnerability in Microsoft Copilot allowed attackers to steal users' two-factor authentication codes — the very tokens that are supposed to be your last line of defence. Researchers dubbed it "SearchLeak," and the exploit worked by manipulating Copilot's web search functionality to exfiltrate sensitive data from a victim's session without any unusual-looking interaction on the user's end.

The mechanics are straightforward in a grim way. Copilot's search integration, when handling certain crafted prompts, could be coaxed into surfacing live authentication codes from a user's email or messaging context and routing them back to an attacker. No elaborate social engineering required — just the right query, and 2FA essentially became a delivery mechanism for the attacker rather than a barrier.

Microsoft has patched the issue, but the researchers' conclusion carries more weight than the fix itself. The industry keeps treating LLM security as a feature to ship later, rather than a first-principles design constraint. Copilot has deep access to emails, calendars, files, and messages — the kinds of integrations that make it genuinely useful. That same depth makes each new prompt-injection or data-exfiltration vector disproportionately dangerous.

This isn't the first time. The pattern is familiar: a new capability ships, researchers probe the edges, a serious exfiltration bug surfaces, a patch follows, and the industry declares it solved until the next integration lands. "SearchLeak shows why the industry's approach to LLM security fails over and over again," is how Ars Technica framed it — and there's not much to argue with there.

For Australian organisations, Copilot for Microsoft 365 is already broadly deployed across enterprise, government, and education. The Australian Signals Directorate's guidance on AI in government explicitly flags data handling and access scope as risk areas — and this vulnerability is precisely the scenario those warnings describe. Any organisation that took an "it's Microsoft, it'll be fine" approach to Copilot's permission boundaries should revisit that assumption now.

What makes this structurally significant is what it reveals about the AI integration model. When an LLM can read your inbox, schedule your meetings, and query your files, the attack surface isn't just the model — it's everything the model can touch. A conventional app vulnerability affects the app. An LLM vulnerability with broad permissions affects the entire data estate the model was trusted with.

The fix here is a combination of tighter output filtering, better prompt-injection defences, and — most importantly — the principle of least privilege applied to AI assistant integrations. Copilot probably doesn't need write access to everything, and it definitely shouldn't be able to surface authentication tokens in a query response.

Watch for: whether Microsoft publishes a detailed post-mortem on the isolation boundary failures here, and whether other AI assistants with similar email/calendar integrations (Google Gemini, Apple Intelligence) receive equivalent scrutiny from researchers off the back of this finding.