Why the Best General AI Model Keeps Losing to Narrower Ones on Security

Anthropic didn't stumble into a weak cybersecurity score with Claude Opus 5. It capped the model's offensive security capability deliberately, a trade-off meant to limit misuse risk. The consequence is that specialized systems like Sakana AI's Fugu-Cyber and Google's Gemini 3.5 Flash Cyber now outperform Opus 5 at penetration testing, not because they are smarter overall, but because they were built for exactly one job, with no safety ceiling holding them back.

That's an unusual dynamic in an industry that spent years pitching breadth as the whole point: one model, competent at everything, no toolbox required. Security is where that pitch breaks down first. The trait that makes a model good at finding vulnerabilities, aggressive and exhaustive probing of a system's weak points, is the same behavior a safety team has every reason to constrain once that model ships to millions of users. Anthropic's choice isn't a failure of engineering. It's a legible trade-off: broad safety versus narrow capability, and Opus 5 chose the former. The pattern isn't new: Google's fine-tune beat Claude Opus 4.6 at vulnerability discovery.

The other side of the same coin

Alibaba's response to a related problem is instructive precisely because it isn't about model capability at all. Its in-session code review catches vulnerabilities in AI-generated code before they land in a repository, rather than relying on the coding model itself to avoid writing insecure code in the first place. That's an architectural answer to the same underlying issue: don't expect a general-purpose model to also be a reliable security gate. Bolt a dedicated reviewer onto the pipeline instead. Whether developers actually let that reviewer run on every commit, rather than disabling it under deadline pressure, is the real open question Alibaba's rollout has to answer. Even Anthropic now ships a separate scanner, Claude Security, that reviews codebases and suggests patches rather than trusting Opus 5 to police its own output, per our write-up of the tool.

The threat side is moving too. A newly documented zero-query attack lets adversaries extract proprietary models from cloud APIs using only timing and memory side-channel data, without sending a single query the provider could log or flag. That puts it in a different class from the prompt-injection and jailbreak concerns that have dominated AI security coverage. It targets infrastructure, not the model's judgment, and no amount of capability capping on the model itself does anything to stop it.

What this adds up to

Put those three together and the direction is clear: AI security is fragmenting into specialized layers rather than consolidating into one trustworthy general model. Offensive security testing is moving to purpose-built systems free of the constraints a consumer product needs. Defensive code review is moving to dedicated tools bolted onto the pipeline rather than trusted to the coding model's own judgment. Infrastructure-level threats like model theft need yet another defense, one that has nothing to do with how capable or safe the model itself is.

For any enterprise renewing an AI security contract on the assumption that a leading general model is automatically the leading security tool, that assumption no longer holds. The practical move is testing narrow, purpose-built alternatives against the specific job at hand: penetration testing, code review, threat detection, rather than assuming the smartest model in the room is also the best at the job. Some offerings are already built that way, including a Devin-powered service that promises to clear 80% of CVE backlogs.