Artificial Analysis, an AI inference evaluation firm, has published the first results of AgentPerf, the industry's first benchmark for measuring agentic AI execution performance. NVIDIA's Blackwell-generation system, the GB300 NVL72, took the top spot, demonstrating that it can run up to 20x more AI agents per megawatt of power than the company's previous-generation system[1]. What sets the benchmark apart is that it measures the workload unique to agents, which differs in nature from chat-style AI, by reproducing actual tasks.
Agentic AI Is a "Relay," Not a "Sprint"
NVIDIA describes the agentic AI workload as fundamentally different from conventional conversational AI. A single chat completion is closer to a "sprint," where one large language model (LLM) call returns one answer. An agent, by contrast, behaves more like a "relay": it breaks a goal into many steps and keeps going until the task is done[1].
As a result, dozens to hundreds of LLM calls are chained together, each passing a growing context to the next. At every handoff, tool calls such as code compilation and execution, database search, and web browsing are inserted. NVIDIA notes that the load does not simply grow by addition but expands multiplicatively[1].
The problem is that many existing AI inference benchmarks measure only a single LLM call. Designed to gauge response speed to a single request or the number of simultaneous requests, they cannot capture the agent-specific load that arises when calls are chained and tool waits and context growth occur[1].
"AgentPerf" Reproduces Real Coding Work
AgentPerf is built on the trajectories of real coding agents. An agent receives a task, reads files, writes and edits code, executes commands, and iterates based on the results. This entire flow is reproduced from actual public repositories spanning more than 12 programming languages[1].
The evaluation measures how many agentic tasks a system can support simultaneously while meeting set thresholds for responsiveness and output rate. The reference output token speed ranges from 20 to 300 tokens per second, and the time to first token (TTFT) is set between 3 and 10 seconds[2]. The tool calls themselves are not executed but are simulated with representative CPU processing time, so the results reflect only the differences in computing platforms[1].
The first round of testing used DeepSeek V4 Pro, a large mixture-of-experts (MoE) model. It is positioned as a representative example of the frontier models powering today's most capable agents[1]. On Artificial Analysis's intelligence index, the model scores 52, placing it second among open-weight reasoning models[2].
GB300 NVL72 Runs 20x More Agents per Megawatt
The GB300 NVL72 recorded the highest performance in this first round. Compared with the previous-generation HGX H200 system, it can run up to 20x more agents per megawatt of power[1]. According to Artificial Analysis's tally, it was able to handle roughly 61,000 concurrent agents per megawatt, a level that corresponds to 20x that of the H200[2].
NVIDIA explains that this gap comes from "codesign," in which hardware and software are engineered together. The GB300 NVL72 connects 72 GPUs into a single rack-scale system, efficiently distributing the execution of large MoE models like DeepSeek V4 Pro. In addition, CUDA kernels overlap communication and computation, and the inference software TensorRT LLM separates input processing from output generation, optimizing so that efficiency is maintained even as the number of concurrently running agents increases[1].
Adoption in Real Services Is Advancing
Several inference providers are already serving agent-oriented workloads using DeepSeek V4 Pro on Blackwell, including Baseten, DeepInfra, and Together AI[1].
As a concrete example, Together AI runs real-time inference for the AI coding platform Cursor on Blackwell. While developers keep working, agents handle bug investigation, feature generation, and refactoring behind the scenes. DeepInfra operates Pam.ai, an AI workforce platform for car dealerships, entirely on Blackwell, with agents handling service appointment bookings, phone responses, and outbound sales activities[1].
NVIDIA states that its next-generation architecture, Vera Rubin, is already in full production and will deliver next-generation processing capacity to meet the growing demand for agentic AI[1].
Summary
AgentPerf is the first benchmark to measure agent-specific loads, such as chained calls and tool waits, in line with actual coding work. In its first results, NVIDIA's GB300 NVL72 took the top spot with 20x more agent processing per megawatt, showing that the Blackwell generation holds an advantage as an execution platform for agentic AI. As more comparison targets emerge, it looks set to take hold as a metric for running AI agents at scale.
Source: https://blogs.nvidia.com/blog/nvidia-blackwell-agentperf-artificial-analysis/
