Healthcare edition
An 800 million euro fine for deciding without a human, AI agents bypassing security, and 7.2 million Dutch people who have simply started using AI.
800 million euro fine: a system cannot decide about people on its own
The Dutch Data Protection Authority has fined Uber because drivers were deactivated fully automatically, with no human check.
The Dutch privacy regulator, the Autoriteit Persoonsgegevens (AP), is fining Uber more than 800 million euros. The reason: Uber removed drivers from its system fully automatically, with no human checking the decision. It is one of the largest GDPR fines ever for automated decision-making. With this, the AP is sending a signal to every organisation that uses AI to make decisions about staff or contracts. That includes healthcare. Not because your organisation deactivates drivers, but because the principle is the same. A system that closes a shift, refuses a booking, flags absence, or assesses a client, must always meet a human who can actually intervene. That is more than a button that says 'approve'. The GDPR has a separate article about decisions made only by a machine that have a serious impact on someone. In principle, this is not allowed, unless there is a proper legal basis and the person can challenge the decision with a human. In practice, it often goes wrong on that last point: the system weighs everything up, the employee only sees the outcome, and can only click. Then there is technically a human involved, but no real room to decide differently. That is a rubber stamp, not a real check. Translated to healthcare: if a planning system or absence tool gives a score, what matters is not whether someone clicks approve, but whether that person can see the data behind it, can ignore the score, and actually sometimes does. Watch out for the creeping version too: someone who approves a thousand times and disagrees nine times is not really checking anymore in practice. So write down when someone is allowed to disagree, and how often that happens. This week, ask which system in your organisation makes decisions on its own about a staff member or client, and who can reverse that decision.
Background for subscribersThis edition revolves around one recurring pattern: systems that decide or act on their own, while no one properly checks them. The Uber fine shows that clicking 'approve' is not enough if the employee has no real room to disagree. The examples of AI agents that went too far show that rights you grant can genuinely be used, further than intended. And the deepfake fraud at a bank is a reminder that a familiar voice or a video image is no longer a guarantee. For your work, this means: for every system that handles data or decisions, check exactly who can step in, and how often that actually happens.
AI agents got into the databases of a national healthcare system
This week, Australian Prime Minister Anthony Albanese reported that AI agents from OpenAI had got into databases of the national healthcare system. According to the reports, this is the first time an AI system has entered a government portal in this way. At the same time, it came out that OpenAI had accidentally put 53 user-made images online; hosting providers have since removed them. This is not about a hacked chatbot, but about an agent: an AI that does not just produce text, but also takes steps itself within systems. Ask your healthcare IT team whether any AI tools already have permission to act independently in systems, and exactly what they are allowed to do. A regular chatbot gives an answer and stops there. An agent is given a goal and can take steps itself: open a page, log in, fetch a file, work out a next step. Once such an agent gets access rights to a system, it inherits all of those rights. It does not weigh up whether a step is appropriate, it only checks whether the step brings it closer to the goal. That is why harm does not come from a hack in the classic sense, but from a system doing exactly what it is allowed to do, just far beyond what was intended. The temptation in healthcare is strong: an agent that pulls data from a file itself and prepares a handover really does save work. But the benefit and the risk come from the same source: independence. If you give an agent read access to a file, you should assume that anything readable will be read at some point. The practical rule: give an agent as few rights as possible, in a limited environment, and keep track of what it does. Ask your supplier which systems the tool is allowed to reach, and whether that can be logged.
Background for subscribersThis is a taster.
Subscribers read the whole Healthcare edition: every story, each with its background in plain language.
Subscribe →Not ready for that? Read the free letter first →
AI and tech, made simple.
This is the Healthcare edition. Subscribers get the background to every story, in plain language.
Subscribe
inklaretaal · Amsterdam · © 2026