Vals CyberOffense: Open Models Breach Corporate Networks

In this article you will see:

• How CyberOffense works and why it is different from previous tests
• The numbers for GPT-6 Sol and Deepseek V4.1 Flash on the benchmark
• Why long runtimes favor open models
• What the result means for those defending corporate networks

What is CyberOffense, the benchmark that tests real attacks

CyberOffense simulates real corporate networks to test AI agents. Each agent operates alone on an isolated attack machine, starting with only a public web address and no human help. The objective is to compromise the network, steal credentials and move laterally between companies.

The test is different from traditional code benchmarks. Instead of solving isolated exercises, the agent needs to chain vulnerability discovery, exploitation, privilege escalation and lateral movement. It is the scenario a real attacker faces.

Vals AI justifies the new test by pointing to saturation of existing benchmarks. When models can run, verify and correct their own code, they reach more than 99% on those tests, and differences between models can almost not be measured.

The numbers: GPT-6 Sol leads, Deepseek V4.1 Flash surprises

GPT-6 Sol, part of OpenAI's restricted Daybreak program, obtained the best result: 80.8% of objectives completed. Deepseek V4.1 Flash 1017, an open model available for free, was not far behind, with 72.8%, and costs about $3 per run.

In the hardest test, called Lighthouse, Daybreak took administrative control of the first company, stole passwords and used those credentials to invade the second company. It completed 27 of 31 objectives in less than three hours. Deepseek achieved 21 objectives at one tenth of the cost.

The 8 percentage point difference between the proprietary leader and the open model is smaller than the cost difference. An attack with Deepseek costs about $3, while the same attack with GPT-6 Sol costs significantly more.

Why long runtimes favor open models

Long execution times proved critical. Deepseek scored zero in the first 25 million tokens and made most of its progress only after 100 million tokens. Short tests therefore underestimate the real potential of open models.

This pattern has a practical implication. Anyone who runs short benchmarks to evaluate security models may conclude open models are harmless, when in fact they just need more time to find the path. Vals AI recommends tests with a high token budget to measure real risk.

What the AISI analysis says about the gap

The results align with a recent analysis from the UK AI Security Institute, July 2026. The gap between open and frontier proprietary models in cyber capabilities shrank to four to seven months. For most of 2025, that gap was still six to ten months.

Open models are also much cheaper. One run in the cyber-range with Deepseek V4-Pro costs only $1.19 according to AISI, versus $85 for Anthropic's Opus 4.5. Safety measures on open models proved largely ineffective.

AISI also notes that the same findings create opportunity for defenders, because the same models that exploit vulnerabilities can help companies find and fix them first.

What this means for network defenders

Real network attacks will likely increase in the coming months. The combination of cheap open models, growing capability and ineffective security measures lowers the barrier to entry for attackers.

For security teams, the message is clear: defenses based on obscurity or on assuming attackers use expensive tools no longer work. It is necessary to assume an attacker has access to open models capable of chaining complex attacks.

Vals AI points out that models that exploit vulnerabilities can be used to find and fix flaws before attackers. The same technology serves both sides. Security teams that adopt these models for red teaming will have an advantage over those who still rely on traditional tools.