Such failures occur daily. They’ve definitely been taking place with AI since I began coping with it in 1985. Methods misbehave, controls change into misconfigured, somebody decides a examine isn’t mandatory, and an incident follows. Swap “AI agent” for “script,” “batch job,” “integration course of,” or “unpatched server,” and you’ve got incidents which have occurred repeatedly all through the historical past of computing. The distinction now could be that the misbehaving system can improvise, and the paperwork it leaves behind reads just like the opening of a thriller.
If a human worker used an organization bank card to purchase a scraping instrument, stashed information on a private cloud drive, or coordinated with coworkers over an unauthorized message board, we wouldn’t conclude that people are an existential risk. We’d conclude that governance, entry controls, and monitoring had been insufficient. That’s precisely what occurred right here, with brokers as an alternative of workers. Unstable fashions had been the set off, however weak safety and governance had been the trigger. Mannequin misbehavior that stays contained is an engineering drawback. Mannequin misbehavior that runs undetected for months throughout the general public web is a governance failure (a well-recognized one) and one we all know repair.
The half that pursuits me
The a part of this story I discover genuinely attention-grabbing isn’t in regards to the fashions in any respect. It’s about how folks latch onto these types of incidents and use them to help their very own beliefs. The reviews are ambiguous, nuanced paperwork. OpenAI explicitly states that not one of the incidents brought on important hurt and that none signifies how usually misalignment happens throughout its fashions. However nuance doesn’t journey. The second these paperwork hit the web, they had been stripped of their caveats and repurposed as proof texts for a story that was already in place.
