Post
4,986
A retry can succeed and still be unjustified.
RetryBench separates whether an agent’s retry is supported by the visible contract from what happens afterward. After a timeout, success alone does not tell us whether retrying was justified.
Try the interactive demo (English / 中文):
uuhugg/retrybench-demo
Read the findings and reproduction guide:
https://huggingface.co/blog/uuhugg/retrybench-retry-permission
How do you evaluate retries in your agents?
RetryBench separates whether an agent’s retry is supported by the visible contract from what happens afterward. After a timeout, success alone does not tell us whether retrying was justified.
Try the interactive demo (English / 中文):
uuhugg/retrybench-demo
Read the findings and reproduction guide:
https://huggingface.co/blog/uuhugg/retrybench-retry-permission
How do you evaluate retries in your agents?
