Finally a good paper testing whether memory-based self-improving agents actually improve. The re-evaluation adds two things prior work skipped, multiple runs to measure variance and randomly shuffled task orders. Both hurt. Agent evaluation is already noisy on multi-step
Memory-Based Agent Evaluation Adds Random Task Order and Repeated Runs
By
–
