Your agent aced the task. Will it do it again? Measuring consistency
An average success rate does not show an agent is reliable. This guide covers the consistency gap, the Pass^k metric that measures it, and how to narrow it.
Models
An average success rate does not show an agent is reliable. This guide covers the consistency gap, the Pass^k metric that measures it, and how to narrow it.