How strict should safeguards be?
The more capable AI models become, the more protective mechanisms their providers build in. These are called safeguards. There is broad agreement that they make sense. How strict they should be is contested. For companies the question is more practical than it sounds: it helps decide which tools actually help in daily work.
Why safeguards exist
Many capabilities of AI models cut both ways. A model that finds security vulnerabilities helps defenders fix them and attackers exploit them. The same goes for questions from biology and chemistry. The major providers respond in several ways:
- Classifiers detect sensitive requests. The model then declines or answers more cautiously; some providers route such requests to a less capable model.
- Access restrictions reserve the strongest models for selected partners, such as security firms and software vendors.
- Identity verification requires proof of who is using the model for certain features.
- Longer retention of inputs and outputs for especially capable models, so misuse can be investigated afterwards.
The case against
Critics consider many of these restrictions too broad. Analyst Ben Thompson argues in his newsletter Stratechery (“Who’s Afraid of Chinese Models?”) that strict cyber restrictions mainly hit the wrong people. Anyone who wants to attack switches to freely available models, of which there are now many, a good number of them from China. Rather than holding capabilities back, he says, the US should strengthen its own open models.
Part of this criticism applies to everyday work too. A classifier cannot reliably tell an attacker from an administrator checking her own systems. People who get weaker answers for legitimate work lose time, or they turn to a tool with less oversight. Both are the opposite of what safeguards set out to achieve.
Where we stand
We think safeguards are right where serious, concrete harm is at stake. For instructions on biological or chemical weapons, the price of a rule that is too loose is higher than that of one that is too strict. And we think it reasonable that especially strong models for offensive tooling are first made available only to vetted defenders.
At the same time, restrictions that are too broad have real costs. They slow down precisely the people trying to make systems safer, and they make AI less useful for many professional tasks. Good safeguards are therefore precise: they target what is actually dangerous and explain why a request is treated differently. Where sensitive capabilities are concerned, checking who is asking is better than refusing everyone the same answer.
Open models are part of the answer here. If you run a model yourself, you can operate it on your own infrastructure in the EU and keep control of your data. That control comes with responsibility: the safeguards a provider would otherwise build in now have to be organised by whoever operates the model.
What this means for your company
- Know the rules of your tools. Ask your provider which requests are restricted, how long data is kept and who can see it.
- Expect limits. General-purpose AI services in particular may answer IT security tasks cautiously. Plan for that rather than letting your team fall back on private accounts.
- Look at the alternative. A model operated for you on servers in the EU gives you more control. Make sure someone is responsible for securing it.
The debate is open, and providers’ rules will keep changing. We follow it and will report here when something significant happens.