Sign up to save your podcasts
Or
Share Repo State Loopholes During Agentic Evaluation · Issue #465 · SWE-bench/SWE-bench
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Repo State Loopholes During Agentic Evaluation · Issue #465 · SWE-bench/SWE-bench
https://github.com/SWE-bench/SWE-bench/issues/465
We've identified multiple loopholes with SWE Bench Verified where agents may look at future repository state (by querying it directly or through a variety of methods), and cases in which future rep...
View all episodes
By VoiceFeedhttps://github.com/SWE-bench/SWE-bench/issues/465
We've identified multiple loopholes with SWE Bench Verified where agents may look at future repository state (by querying it directly or through a variety of methods), and cases in which future rep...