I recently read a few articles about a startup called PocketOS allowing a coding agent to delete their entire production database. Here is an example of one such post.
As a Senior Site Reliability Engineer at a successful bay area tech company, I’m deeply bothered by the founder’s lack of personal responsibility.
He gave an ape a machine gun and blames others for the things that went wrong!
Let’s talk about the glaring mistakes.
For starters, the Tom’s Hardware article linked above says they restored to a 3 month old production backup. This is negligence, full stop. Production databases should have regularly scheduled automated backups.
Secondly, engineers are responsible for access control. Blaming Railway for their API keys being excessively permissive is immature. If a staging environment’s API key can access a live database, it’s an immediate issue and measures must be implemented to reduce the blast radius. One simple solution is to put the production and staging environments in separate accounts.
Additionally, engineers are responsible for evaluating the tools they use. If a cloud provider’s offerings are not robust enough for your use cases (e.g. – orbital-slop-cannon vibe coding), then action is required. There are practically infinite ways to deploy an application. PocketOS chose to use a platform without granular control, presumably because it was easier than investing engineering time in to infrastructure.
Lastly, AI agents making mistakes is nothing new. Professionals understand this reality, and don’t expect LLMs to always make sane decisions. There are many mechanisms to control agents, including system level hooks, filesystem permissions, and sand boxed environments.
The founder exhibited multiple levels of incompetence, and should take more accountability for his mistakes.