The recent findings from the UK’s AI Safety Institute serve as a stark warning that the industry is moving too fast for its own good. When models can be coerced into creating fake profiles or assisting in cyberattacks, the priority should be on fixing these fundamental security flaws, not on securing billions of dollars to scale the technology further. Expanding infrastructure before ensuring that the underlying models are secure is a dangerous gamble with public safety.
There is a clear disconnect between the marketing of AI as a safe, transformative tool and the reality of its vulnerability to manipulation. If these models are deployed at scale while these security gaps remain, the potential for widespread harm—ranging from automated fraud to sophisticated phishing campaigns—is significant. The rush to build more data centers suggests that companies are prioritizing market share and computational dominance over the essential work of building robust, tamper-proof systems.
Regulators and the public must demand more accountability from companies like Anthropic. It is not enough to acknowledge these vulnerabilities after they are discovered in tests; there must be a fundamental shift in the development lifecycle that puts safety at the center of the design process. Until these companies can demonstrate that their models are resilient against malicious use, the focus should be on slowing down deployment and increasing transparency, rather than accelerating the expansion of the very systems that pose these risks.