Cartesia Managed Agents leads Speech Agent Arena

The difference matters because the ranking measures two distinct things: human preference for a conversation and the correct completion of a tool task. See the result published by Artificial Analysis on X.

In this article you will see:

• How the Speech Agent Arena works
• What Elo 1.115 and Task Success Rate of 91.7% mean
• Why preference and task completion are not equivalent
• What limits to consider before choosing a voice agent

How the Speech Agent Arena evaluates voice agents

The Speech Agent Arena compares speech to speech systems in live conversations. Artificial Analysis explains that human participants compare two hidden systems in the same scenario and then choose which conversation they preferred.

The evaluation separates 15 agentic scenarios, which require tool calling, from 20 scenarios without tool calling. The set includes situations like ordering food and asking about opening hours, according to the official launch article of the Arena.

The preference vote feeds an Elo score. Elo is a relative scale used to rank participants from comparisons, not a percentage of accuracy. Therefore, 1.115 Elo does not mean 1.115 percentage points nor that the agent completes 1.115 tasks.

For agentic scenarios, Task Success Rate measures the proportion of eligible conversations in which the system completed the requested action with the correct tool call. Cases with participant deviations or unverifiable results are excluded from this calculation.

The launch chart from Artificial Analysis places Elo on the vertical axis and task success on the horizontal axis. It illustrates why a system can sound convincing without necessarily completing the requested action.

Cartesia Managed Agents scored Elo 1.115

In the October 27 post, Artificial Analysis reported Elo 1.115 for Cartesia Managed Agents and said the result was 98 points above Cartesia Line, the company's previous voice agent product.

The same post reports Task Success Rate of 91.7% for Managed Agents, versus 77.5% for Cartesia Line. The publication does not present in this text the absolute Elo score for Cartesia Line, so the comparison should be limited to the 98 point difference disclosed by Artificial Analysis.

These numbers are a snapshot of the ranking at the time of the post, not a guarantee that the position will remain the same. The Arena can change as new systems and conversations are added.

The result also refers to the configuration called Managed Agents. It is not correct to transfer it automatically to all voice synthesis by Cartesia or to any integration built with its models.

Elo and task success answer different questions

Elo tells you which conversation participants preferred in a head to head comparison. That perception can involve naturalness, pacing, clarity and a sense of appropriate response, although the publication does not provide a detailed breakdown of the score by attribute.

Task Success Rate tells you whether the action was actually completed by the expected tool call. An agent can sound pleasant but still fail to book a time slot, update a record or complete another task connected to an external system.

For a support product, these metrics need to be viewed together. If the speech experience is good but the task fails, the user still does not get the outcome they asked for. If the task is completed but the conversation feels choppy or slow, the experience can still be poor.

The Artificial Analysis speech to speech Arena also publishes time to first audio and other results. To understand how the benchmark organizes evaluations, check the official Speech Agent Arena dashboard.

What the publication does not allow you to conclude

Artificial Analysis states that most of the prompts, tool schemas and scenario instructions are private to reduce overfitting to the tests. The cited exception is a dental scheduling example.

The organization also says it uses selected, paid third party participants to conduct and evaluate the interactions. That describes the collection, but it does not reveal the size of each sample, the distribution of results by task or the variation between participants in the post about Cartesia.

For that reason, the ranking is useful evidence of performance on that set, not proof of universal superiority. It does not by itself demonstrate behavior in Portuguese, in background noise, with interruptions, in long calls or with tools different from those evaluated.

It is also important to confirm the ranking version and the snapshot date before comparing systems. The position published by Artificial Analysis is dated, and preference rankings can change with new evaluations.

How to use the result when choosing a voice agent

Start with the product's real tasks. If the agent needs to complete a booking, log a request or check an account, prioritize tests that validate the final tool call in addition to speech quality.

Then compare the benchmark result with your own flow. Check latency, error recovery, interruptions, language, accent, privacy and integration with the required tools. The Artificial Analysis post does not replace testing with your own scenarios and users.

For those building an agent, the bridgecode guide on how to build agents with the Cursor Agent SDK shows how to define tools and evaluate behavior. And the article on interactive worlds with native audio in Genie 4 helps separate audio generation from a conversation that needs to execute actions.