Dicas
6 min
7 tips for evaluating AI agent benchmarks
Seven checks for comparing agent benchmarks: task unit, sample size, generalization, test conditions and score limits.
#benchmarks de agentes#avaliação de IA#generalização
Artigos publicados
252 artigos por Marcelo Santiago
Newsletter
5 min
NVIDIA Cosmos 3 releases four models for simulation
NVIDIA adds four Cosmos3-Nano checkpoints for automotive and robotics simulation. See capabilities, requirements, license and limits.
#NVIDIA#Cosmos 3#Cosmos3-Nano
Benchmarks
5 min
ToyBlocks: A Controlled Benchmark for Robots
AI2 introduces ToyBlocks, a simulation environment with four tasks to study generalization and physical transfer in robotics.
#AI2#ToyBlocks#robótica
Newsletter
6 min
Codex 0.167.0 adds an agent hub in the terminal
Codex 0.167.0 adds an agent hub in the terminal, highlights unread results and preserves drafts when switching backends. See what changes.
#OpenAI#Codex#Codex CLI
Benchmarks
6 min
GPT-5.1 tops tau2-Bench Airline with 80%
On the tau2-Bench Airline leaderboard from November 6, GPT-5.1 scored 80.0%. Learn about the method, minimum sample size and limitations before choosing an agent.
#OpenRouter#GPT-5.1#tau2-Bench Airline
Newsletter
5 min
Anthropic analyzes how Claude shows up in scientific research
Anthropic analyzed 207,049 scientific sessions in Claude and Claude Code and separated observed usage from productivity projections.
#Anthropic#Claude#Claude Code
Dicas
6 min
5 tips to give context to agents with NVIDIA Auto Ontology
Learn about NVIDIA Auto Ontology, an Apache 2.0 repository for giving semantic context to agents and the security considerations before adopting it.
#NVIDIA Auto Ontology#ontologia semântica#agentes de IA
Benchmarks
7 min
Free models and new providers of the week (06/11)
Kimi K3.1 Pro and ERNIE 5.5 arrived with public weights; Axion, Shackleton and ORO expanded the open movement and agent infrastructure.
#modelos-abertos#modelos-gratuitos#provedores-de-ia
Newsletter
6 min
Gemini Omni 1.2 Flash generates videos up to 15 seconds long
Google announced Gemini Omni 1.2 Flash on November 5, with video generation up to 15 seconds and sequential edits in the creative flow.
#Gemini Omni 1.2 Flash#Google AI#geração de vídeo
Newsletter
7 min
Baidu releases ERNIE 5.5 Flash Base
Baidu released ERNIE 5.5 Flash Base on November 5. The checkpoint has 280 billion parameters and self reported results on 30 benchmarks across seven categories.
#ERNIE 5.5 Flash Base#Baidu#modelos base
Newsletter
6 min
GPT-6 Cyber limits access to security testing
OpenAI unveiled GPT-6 Cyber on November 5 for approved organizations. See what the tests measure, their limits, and the access conditions.
#GPT-6 Cyber#OpenAI#cibersegurança
Dicas
6 min
5 tips for using Playwright Test Agents
Planner, Generator and Healer split test creation in Playwright. See how to review the results and the caveat about the Docker image v1.64.1.
#Playwright#Playwright Test Agents#testes end-to-end
The biggest AI newsletter in the World
Follow the evolution of AI squads and multi-agent systems.
In-depth technical curation: agentic architectures, real token benchmarks, MCP servers and open-source tools in your inbox and on the blog.