Open models close the gap in cybersecurity

In this article you will see:

• How AISI measured the gap between open and closed models
• Which models lead in each category
• Why the cost difference matters as much as performance
• What this means for defenders and attackers

How AISI measured the gap

AISI used two methodologies. The first, called Narrow Cyber Tasks, evaluates 70 tasks at four difficulty levels, covering vulnerability research, reverse engineering, web exploitation and cryptography. The second, Cyber Ranges, tests autonomous capabilities in simulated networks, such as the scenario "The Last Ones", which simulates a 32 step attack on a corporate network with four subnets and about 20 hosts.

In Narrow Cyber Tasks, GLM-5.2, released in June 2026, performed at the level of Opus 4.6, released in February 2026. This puts the open model about 4 months behind the closed one. DeepSeek V4-Pro performed at the level of Opus 4.5, released in November 2025, that is, 5 months ago. In the Cyber Ranges band, the gap is larger, about 7 months, but AISI treats this comparison as weaker evidence because it comes from fewer scenarios.

The comparison also uses Cyber ECI, a cybersecurity version of the Epoch Capabilities Index. GLM-5.3, the open model with the highest score, has Cyber ECI 160, above GPT-5.4 (March 2026), but below Claude Mythos Preview and GPT-5.5 (April 2026). This places the open frontier where the closed frontier was 6 to 7 months earlier.

Which models lead

Among closed models, GPT-6 Sol, part of OpenAI's restricted cybersecurity program Daybreak, posted the best result in the Vals test, with 80.8% of objectives. DeepSeek V4.1 Flash 1017, an open-weight freely available model, came close, with 72.8%, at a fraction of the cost: about $3 per run versus $22 for GPT-6 Sol.

In AISI's table, GPT-6 Sol Daybreak leads with 27 of 31 objectives in Lighthouse, followed by DeepSeek V4.1 Flash with 21. Meta Wonder appears with 28 in Lighthouse but only 7 in Nova. Kimi K3.1 Pro scores 20 in Lighthouse and 13 in Nova.

Most models do not make significant progress until they pass 100 million tokens. DeepSeek scored zero in the first 25 million tokens and made most of its progress only after 100 million. Short tests therefore underestimate the real potential of open models.

Why cost matters as much as performance

AISI measured the cost of a 100 million token run in Cyber Range: $85 with Opus 4.5 or 4.6, $46 with GLM-5.2 and only $1.19 with DeepSeek V4-Pro. For individual tasks that both models solve reliably, Opus 4.6 costs $15 per task, GLM-5.2 $6, Opus 4.5 $12.50 and DeepSeek V4-Pro 28 cents.

This makes attacks with cheap open models easy to scale. In addition, the safety measures on open models proved largely ineffective. DeepSeek V4-Pro sometimes refused reverse engineering tasks, but it was enough to try again to bypass the restriction. Techniques like refusal training are easily reversible with access to the weights.