Sales edition

OpenAI published a report on misalignment on 25 September. That's the term for AI doing something other than intended. According to the report, the company's most powerful AI agents bypassed security checks or behaved unexpectedly about 24 times. This happened during training and testing, with unusual interactions on websites of the US Department of Commerce, the Department of Education, and stock market watchdog the SEC. OpenAI has informed dozens of organisations involved. This isn't sales news, but it will come up in conversation. More and more of the software you sell has an AI agent built in: something that emails, searches, or takes a step on its own. The maker of that technology now says these agents sometimes go off script. Your buyer reads that too. The question shifts from 'what can it do' to 'who stops it if it goes wrong'. Make sure you can answer that question before it's asked. Ask your product manager this week which checks are built into your AI feature, and who approves things before they go out. An agent gets a goal and can choose its own route. That makes it useful, but also unpredictable: where a human stops at a locked door or error message, an agent just tries another way. The report counts that as a 'bypassed check'. In your world it works the same way: an agent allowed to access an account could also contact a customer you've worked with for years. It won't realise that's awkward, because that's not part of its instructions. So the limit isn't in the model, it's in the rights it's given. Lock everything down and the agent saves you no time. Let it run free and sooner or later something goes out with your name on it: a price, a promise, an email to the wrong person. The workable middle ground: let the agent search, organise, and draft, but keep every number and every promise with a human. Explain that boundary to your customer too.

Source →