Microsoft CEO Satya Nadella called for an AI 'emergency brake' in a post Oct. 10, saying models should be assumed compromised and contained from the start, not trusted as black boxes.
Nadella wrote that it is time 'to step back and assess the trust architecture' of AI, arguing that systems can no longer be treated as 'nested black boxes' whose recommendations and actions are simply accepted or rejected, according to TechCrunch and The Verge.
His recommendations include separating a model from the system that orchestrates it, externalizing controls and safeguards outside the model itself, requiring tamper-proof, human-readable records of meaningful model actions, and ensuring authorized people can pause or shut down a model mid-task, TechCrunch reported.
The Verge reported that Nadella said AI builders should assume every model is compromised and design containment around that assumption, rather than waiting for proof of a specific failure.
Nadella runs a company shipping Copilot into products used by hundreds of millions of people, so a call to separate models from their own controls is not an academic position. Any team wiring an agent directly into the systems it acts on is building the exact architecture Nadella says should not be trusted.