A persistent problem in reinforcement learning from verifiable rewards (RLVR) is the dead-gradient sample. When every reasoning trace sampled for a given prompt receives the same reward, group-relative advantage estimation produces zero gradient signal. The traces get discarded, even if they differ meaningfully in reasoning quality. That is wasted compute and a missed learning opportunity.
Reasoning Arena is a new adaptive training framework that routes these non-diverse reward groups to a judge system instead of throwing them away. The judge goes beyond checking final answers. It runs trace tournaments: head-to-head comparisons between reasoning traces that expose finer-grained quality differences within the group. Those comparisons get converted into relative reward signals, giving the optimizer something to work with.
The efficiency problem with pairwise comparison is obvious. Exhaustively comparing every trace against every other trace is quadratic. Reasoning Arena avoids this. Each new trace is evaluated against a small, dynamically updated pool of previously generated traces used as anchors. That establishes a relative ranking without full pairwise coverage. A Bradley-Terry model is then fit on the resulting incomplete comparison graph, producing a scalable ranking that integrates cleanly into the RL training loop.
The numbers are meaningful for anyone running training at scale. Reasoning Arena outperforms the RLVR baseline by 7.6% on average across competition mathematics and coding benchmarks. Training accelerates by 27% to 41% depending on the setting. Generation compute drops by nearly 50%, because the framework stops wasting inference budget on samples that would otherwise contribute nothing.
The core insight here is architectural, not just algorithmic. RLVR pipelines currently have a structural blind spot: they only learn from prompts where sampled traces spread across different reward levels. Reasoning Arena treats uniform-reward groups as a signal of their own, a cue to switch evaluation strategy rather than skip the sample entirely. The judge system fills the gap that outcome-based rewards leave open.
For product engineers building or fine-tuning reasoning models, the practical takeaway is concrete. If your RLVR pipeline discards a large fraction of training batches because reward variance collapses to zero, you are leaving both quality and compute efficiency on the table. Reasoning Arena suggests a specific remedy: instrument your training loop to detect zero-advantage groups, route them to a comparative judge, and use a ranking model to recover gradient signal. The Bradley-Terry approach means you do not need exhaustive comparisons to make this work. Start by measuring what fraction of your batches are currently discarded due to reward non-diversity. That number tells you how much headroom this approach might unlock.