Skeptics and safety researchers argue that the current industry practice of self-regulation is fundamentally flawed. When companies are responsible for both the development and the safety testing of their own products, there is an inherent conflict of interest. The pressure to beat competitors to market often leads to the prioritization of speed over the thorough vetting of potential harms, such as bias, security vulnerabilities, or the potential for mass-scale manipulation.
Critics point to the lack of transparency as a major concern. Without independent, third-party audits, the public has no way of knowing how a model was trained or what guardrails are actually in place. This opacity makes it impossible for researchers to understand the long-term societal impacts of these models until after they have already been integrated into critical systems like finance, healthcare, or public communication.
There is also the risk of catastrophic failure. As models become more autonomous, the potential for them to act in ways that their creators did not intend increases. Relying on voluntary commitments from a handful of corporations is not a substitute for enforceable, legally binding safety standards that hold companies accountable for the outcomes of their technology.
For the public, the stakes are high. If AI models are deployed without rigorous, standardized safety benchmarks, society may face irreversible damage to trust in information and the stability of digital infrastructure. The call for caution is not about stopping progress, but about ensuring that the development of powerful technology is guided by public interest rather than just corporate profit margins.