GenXR-Bench: Claude Opus 5 scores 50.9% on XR

The v2 version of GenXR-Bench was updated on arXiv on October 22, 2026. The paper gathers 112 tasks for coding agents that build extended reality, or XR, applications and reports 50.9% Pass@1 for the strongest configuration evaluated.

That configuration was Claude Opus 5. The result is specific to the benchmark presented by the authors, not a position in a universal programming ranking.

In this article you will see:

• How GenXR-Bench organizes XR tasks
• What the 50.9% Pass@1 result means
• What failures appear in spatial apps
• How to interpret the paper's limits

GenXR-Bench covers 112 XR development tasks

XR brings together virtual, augmented and mixed reality experiences in which the application reacts to movement and a three dimensional environment. An agent needs to understand coordinates, sensors and interactions that do not appear in a common web page.

The paper splits the 112 tasks into three groups: 24 SDK feature implementations, 21 bug fixes and 67 full application creation tasks. The authors say the set covers examples from education, health, daily life, design and entertainment.

Each task combines a natural language specification, an editable project and a separate evaluation suite. This way the benchmark can test not only whether the code compiles, but also whether the observed behavior matches the request.

The chosen base is XR Blocks, an open source WebXR framework used by the authors to build and run the applications. This makes tests reproducible inside that environment, but it does not automatically represent projects made with all XR engines or SDKs.

To validate the suite, eight specialists evaluated 124 candidate applications on consumer XR hardware. Across 372 evaluation runs, the automated system reached 96.5% average accuracy compared to human evaluations, with 4.0% instability.

That 96.5% measures the agreement of the automated suite with the specialists, not the success rate of the agents. Separating these two metrics avoids confusing test reliability with model capability.

Claude Opus 5 reached 50.9% Pass@1

The authors evaluated ten model configurations. The paper table identifies claude-opus-5 as the strongest configuration in the set, with 50.9% Pass@1 on the overall result.

Pass@1 indicates the proportion of tasks completed correctly on the first evaluated attempt. In this benchmark, performance dropped to 29.2% on SDK feature implementation tasks, while the table shows 85.7% on repair and 47.8% on application creation.

The difference between categories shows that the aggregate result hides specific difficulties. An agent can repair a known behavior more often than implement a new capability that depends on internal SDK details.

The paper also does a failure analysis. Among the recurring problems are coordinate transformations, asynchronous sensor initialization and interaction semantics that require two hands.

The figure shows different ways to verify the same task: semantic assertions, scripted actions, an adaptive agent and vision based checks. This design tries to approximate the evaluation of what a person observes inside the XR experience.

What the GenXR-Bench results do not demonstrate

The result for Claude Opus 5 applies to the tasks, tools and conditions described in the paper. It does not allow concluding that the model will be the best for any XR project, nor compare it directly with results from traditional programming benchmarks.

Version v2 is a preprint on arXiv. The numbers are the outcome of the authors' study and still need to be interpreted alongside sampling details, the framework used and how the tasks were executed.

The 50.9% score also does not mean that half of all XR code generated in the world is correct. It summarizes first attempt performance across the 112 tasks that make up this benchmark.

The set can help test capabilities that common code evaluations leave out, but it still represents a slice of spatial development. XR applications can vary in hardware, sensors, tools and physical safety requirements.

To see another type of benchmark for interactive software creation, check the article on Code2Video-AA and its evaluation of videos generated from code. To compare agentic programming tools, see also the guide to Codex, Copilot and Claude Code.

How to use GenXR-Bench to evaluate an agent

Start by separating tasks by skill. A bug fix, an SDK implementation and a full application require different capabilities, so a single number can hide where the agent fails.

Use observable criteria before running the agent. Combine code tests with visual and interactive checks when the outcome depends on movement, sensors or position in space.

Record the model, the available tools, execution limits and the number of attempts. Without this information, comparing two setups can mix model differences with environment differences.

Then review errors by category. If coordinate failures or sensor loops repeat, turn each one into a specific test and check whether a change fixes the problem without breaking other behaviors.

The paper includes code and benchmark materials to reproduce the evaluation. Before using the results as a purchasing criterion, confirm the published version, your project's compatibility and whether the tasks resemble the work your team actually delivers.