Skip to content
Now

Tech · AI exploit testing

OpenAI previews a 77% exploit score—but leaves the comparison unproven

The unnamed model reportedly leads GPT-6 Cyber by 24.4 percentage points on ExploitGym. Missing test conditions and access rules limit what that result tells defenders.

Written 5 min read · 13 sources

OpenAI’s latest cybersecurity preview points to a model much better at turning software weaknesses into working exploits. But the most important part of the comparison is still missing: enough information to tell whether the models faced equivalent tests. A reported 77.0% score is a substantial claim, not yet a reproducible demonstration of how much stronger the system is.⁠1⁠2

BigGo Finance and Gizmodo Japan report that OpenAI showed the result for an unnamed, unreleased model, placing it above the existing cybersecurity specialist GPT-6 Cyber, at 52.6%. An approximately 38% result for Anthropic’s Mythos 5.2 was reportedly mentioned aloud as a reference value, rather than displayed on the chart. These are secondary accounts of presented results, not an independently reviewed evaluation.⁠1⁠2

77.0%Unreleased model’s reported ExploitGym score 12
52.6%GPT-6 Cyber’s reported ExploitGym score 12
~38%Mythos 5.2’s spoken reference score 12

The presentation initially compared the general-purpose GPT-6 Astra with GPT-6 Cyber, according to Gizmodo Japan. Then a red line appeared above them for the next-generation cybersecurity model. No numeric result for GPT-6 Astra is established in the material available for this report; the 77.0% figure belongs to the unreleased system, not GPT-6 Cyber.⁠2

The gap between 77.0% and 52.6% is 24.4 percentage points. That is not a 24.4% relative improvement, and calling it a roughly 25% gain confuses two different measures. Likewise, the roughly two-to-one ratio against the approximate Mythos reference concerns these particular scores. It does not establish that OpenAI’s new model is twice as effective at cybersecurity generally.⁠1⁠2

The event’s timing also needs care. Gizmodo Japan places the presentation in the early hours of November 13, 2026, in its Japanese-language account. The linked video is titled “Cyber at the Frontier,” rather than the reversed title used in that coverage. Its November 10 publication metadata does not establish when the presentation occurred, and no readable transcript was available to verify the benchmark passages directly.⁠2⁠11

From a bug to an attack

ExploitGym tests a narrower—and more consequential—skill than simply spotting faulty code. An agent receives a program input that triggers a vulnerability, then tries to extend it into a working exploit. Success means a concrete security impact, such as accessing a file without permission or running unauthorized code. The benchmark’s authors distinguish the vulnerability from the attack it might enable.⁠12⁠13

That distinction matters to defenders. Finding a weakness leaves a practical question: what could someone actually do with it? Demonstrating exploitation can help establish which repairs deserve urgent attention. The same capability can also reduce the work an attacker needs to do. Nothing in this preview establishes that an organization was attacked; the concern is what a more capable exploit-building system could make possible.⁠13

The original ExploitGym description was submitted on May 11, 2026, as a preprint and has not been peer reviewed. It describes 898 instances drawn from real-world vulnerabilities across ordinary applications, Google’s V8 JavaScript engine and the Linux kernel, the operating system’s core. Researchers varied the security protections applied to instances and packaged the configurations in reproducible software containers.⁠12⁠13

Those design choices are not incidental. Exploitation depends on the protections and runtime conditions surrounding a bug, as well as the agent’s ability to reason about it. The original evaluation found that exploitation remained challenging, but frontier agents could exploit a meaningful fraction of tested vulnerabilities and retained some success with widely used defenses enabled.⁠13

In that earlier evaluation, the strongest configurations for Claude Mythos Preview and GPT-5.5 produced 157 and 120 working exploits, respectively. Those are different models from the ones in the new comparison. Neither those counts nor the original benchmark’s 898 instances establishes the denominator behind November’s percentages. Turning them into a progress chart would require first matching the tasks and conditions.⁠12⁠13

Practical claims need practical proof

The accompanying accounts also credit GPT-6 Cyber with finding 81 previously unknown, high-severity Firefox vulnerabilities and examining roughly 1,000 Linux vulnerabilities in a single run, with patch generation. These are claims about the existing specialist, not verified outcomes for the unreleased model. Examining Linux vulnerabilities does not mean discovering 1,000 new flaws.⁠1⁠2

Nor does generating a patch mean a repair has been validated, accepted or deployed. The available reporting does not provide corresponding vulnerability records or patch outcomes that would let defenders check those claims. A proposed fix still has to survive engineering review and testing before it can count as a successful repair.⁠1⁠2

Gizmodo Japan’s separate account of OpenAI’s defensive-tool demonstration describes that human checkpoint: AI proposes changes, a person reviews them, and the system applies the approved fixes. That is a useful distinction between producing security work and establishing that the work is sound. It is not independent confirmation of the Firefox or Linux totals.⁠4

What would make the lead convincing

The missing evaluation details go well beyond a model name. The new comparison does not establish the task count, benchmark version, vulnerability selection or scoring rules. It also leaves unclear the precise model configurations, prompts, tools, attempt limits and time or compute budgets—and whether those conditions were equivalent across models.⁠1⁠2

Repeated runs and uncertainty estimates would help show whether the reported gap is stable. Contamination controls would help establish whether prior exposure to test material affected performance. Without those details, the defensible conclusion is a substantial reported benchmark lead, not a measured across-the-board advantage in real-world security.

The unnamed model remains unreleased. No public release date, access policy or specific safeguards are established in the available evidence. The next consequential announcement is therefore not necessarily a larger score: it is an evaluation that others can scrutinize, alongside access arrangements that explain how OpenAI intends to manage a capability valuable to defenders and attackers alike.⁠1⁠2

Sources12

Of the sources cited here, ギズモード・ジャパン published first, 18 hours before this report aired. 4 more outlets followed.

7 outlets with no date:antenna.jpar5ivcool3c.comCuriousCatsIT業界あるあるまとめUnnamed channel

Every source, by outlet

13 sources from 12 outlets· Most cited first

As aired
  1. OpenAI previewed an unreleased AI model that turns software weaknesses into working exploits.
  2. The model reportedly scored 77.0 percent on ExploitGym, which tests whether AI can turn a supplied vulnerability into an exploit. That ability could help defenders prioritize repairs, and make attacks easier. How much stronger is it under comparable conditions, and who will get access?
  3. OpenAI's preview put an unnamed next-generation model above its existing cybersecurity specialist. The presentation first compared the general-purpose GPT-6 Astra with GPT-6 Cyber. Then a red line for the unreleased model appeared above them. The new model's 77.0 percent score is the figure driving the comparison.
  4. The presented scores were 77.0 percent for the unreleased model and 52.6 percent for GPT-6 Cyber. Anthropic's Mythos 5.2 was given an approximate 38 percent reference value verbally, separate from the displayed chart.
  5. The gap over GPT-6 Cyber is 24.4 percentage points. Calling that a 25 percent improvement would mix up two different measures. And the roughly two-to-one ratio against Mythos applies to these particular scores. It tells defenders about performance on this exploitation test, while broader cybersecurity work involves many other tasks.
  6. ExploitGym gives an agent an input that triggers a software vulnerability. The agent then tries to build a working exploit from it. Success means producing a concrete security impact, such as accessing a file without permission or running unauthorized code.
  7. A working exploit can show defenders which weakness needs fixing urgently. Finding a bug leaves a practical question about what someone could actually do with it. Demonstrating exploitation helps resolve that question. But the same capability can lower the barrier for attackers, so access controls become part of the security stakes.
  8. Subscribe to this channel as we follow the model's evaluation details and who will be allowed to use it.
  9. The original ExploitGym benchmark contained 898 instances drawn from real-world vulnerabilities. They covered ordinary applications, Google's V8 JavaScript engine, and the Linux kernel, the core of the operating system. Its description was submitted on May 11, 2026, as a preprint.
  10. Researchers varied the security protections around the vulnerable programs and packaged the configurations in reproducible software containers. Those are controlled environments that others can run again. Exploitation depends on the protections and runtime conditions surrounding a bug, as well as the agent's reasoning. The original study hasn't been peer reviewed.
  11. In that earlier evaluation, the strongest configurations produced 157 working exploits for Claude Mythos Preview and 120 for GPT-5.5. Those were different models. Comparing those counts with November's percentages would require matching the tasks and conditions first.
  12. The original 898 instances don't tell us how many tasks sit behind the new percentages. In the May evaluation, exploitation remained challenging, but frontier agents succeeded on a meaningful fraction of vulnerabilities. They also retained some success with widely used defenses enabled. That combination explains both the defensive promise and the risk.
  13. GPT-6 Cyber reportedly found 81 previously unknown, high-severity Firefox vulnerabilities. It also examined roughly 1,000 Linux vulnerabilities in one run and generated patches. Those claims concern the existing model, and examination of Linux vulnerabilities doesn't mean discovery of that many new flaws.
  14. Generating a patch still leaves engineers with work to do. They need to validate the fix before accepting and deploying it. The practical claims about GPT-6 Cyber don't come with vulnerability records or patch outcomes that would let defenders check those results. The unreleased model's stronger benchmark score also needs its own evaluation details.
  15. The new comparison leaves the test setup unclear. Defenders need to know which tasks counted, how success was scored, and whether models had equivalent tools and attempt limits. Repeated runs would help show how much the results vary.
  16. The presented figures show a substantial benchmark lead. Whether that translates into a comparable real-world defensive advantage is still unresolved. OpenAI also hasn't announced access arrangements or specific safeguards for this unnamed model. It remains unreleased, with no public release date.

Aired recently