While the open weights letter promotes noble ideals, it glosses over serious safety concerns that have troubled AI researchers and policymakers. Releasing model weights to the public means anyone can download and deploy these powerful systems, including malicious actors. Without tight controls, the same technology that helps doctors diagnose diseases could be weaponised to create disinformation, automate cyberattacks, or generate fraudulent content.
For the UK, which is positioning itself as a global AI hub, this presents a regulatory headache. The government has pledged to foster innovation while managing risks, but open weights make enforcement nearly impossible. Once weights are public, they can be used without any oversight, bypassing built-in guardrails. The letter offers no concrete plans for monitoring or accountability measures.
This is not a hypothetical worry. Past incidents with open-source language models have shown how easily they can be manipulated. Critics argue that the companies signing the letter are prioritising community goodwill over due diligence. A more prudent approach would be to maintain some degree of central control until robust safety mechanisms are proven. The rush to openness could undermine public trust and invite stricter, less flexible regulations down the line.