{"ID":6621304,"CreatedAt":"2026-07-15T01:01:48.440468303Z","UpdatedAt":"2026-07-15T03:28:55.185153975Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.12217","arxiv_id":"2607.12217","title":"Good Benchmarks","abstract":"Good tasks are correct, solvable, verifiable, well-specified, and hard for interesting reasons. The best tasks describe a real problem an experienced practitioner would recognize, in language a practitioner would use, with tests that verify the outcome rather than the approach.","short_abstract":"Good tasks are correct, solvable, verifiable, well-specified, and hard for interesting reasons. The best tasks describe a real problem an experienced practitioner would recognize, in language a practitioner would use, with tests that verify the outcome rather than the approach.","url_abs":"https://arxiv.org/abs/2607.12217","url_pdf":"https://arxiv.org/pdf/2607.12217v1","authors":"[\"Ivan Bercovich\"]","published":"2026-07-13T23:32:05Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[]","has_code":false}