A new analysis from IBM Research reveals a critical gap in how AI agent reliability is measured. While a ReAct agent backed by GPT-4.1 reports 77.4% average success on benchmark tasks, it succeeds on all five repeated runs for only 53.0% of those same tasks—a 24.4 percentage point consistency gap. This discrepancy exposes a fundamental problem: standard industry benchmarks report mean performance, which hides a costly unreliability problem in production environments where mission-critical tasks like financial reconciliation or contract validation demand consistent execution.
The inconsistency stems from how language model probability distributions shape decision points in agent workflows. Some decisions emerge from sharp probability distributions where one token choice dominates, producing consistent behavior run after run. Others emerge from flat distributions where multiple token choices carry similar probability mass, causing the agent to "flip" between different actions on identical requests. These flip-prone steps compound across a trajectory of dozens of decisions, creating volatility that no amount of model capability or decoding optimization can fully eliminate.
IBM's solution combines a diagnostic tool called the Consistency Analyzer with a new guideline type that targets reliability directly. By resampling decision points from recorded agent trajectories, the system identifies unstable steps and generates consistency guidelines that stabilize them at inference time. The result: the 24.4-point consistency gap shrinks to 12.0 points without sacrificing average accuracy, bringing practical production reliability in line with reported performance.
Key Points
Standard benchmarks hide inconsistency: 77.4% average success masks only 53% consistent success across repeated runs—a 24.4 percentage point gap
Root cause identified: flat probability distributions in language models cause agent decisions to flip between runs, compounding across multi-step trajectories
Consistency Analyzer diagnoses unstable decision points using single-pass resampling without re-running end-to-end tasks or requiring ground truth labels
Consistency guidelines reduce the reliability gap from 24.4pp to 12.0pp without costing average accuracy, improving production readiness for mission-critical workflows