{"ID":2876990,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.20410","arxiv_id":"2508.20410","title":"UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools","abstract":"AI text-to-app tools promise high quality applications and websites in minutes, yet no public benchmark rigorously verifies those claims. We introduce UI-Bench, the first large-scale benchmark that evaluates visual excellence across competing AI text-to-app tools through expert pairwise comparison. Spanning 10 tools, 30 prompts, 300 generated sites, and 4,000+ expert judgments, UI-Bench ranks systems with a TrueSkill-derived model that yields calibrated confidence intervals. UI-Bench establishes a reproducible standard for advancing AI-driven web design. We release (i) the complete prompt set, (ii) an open-source evaluation framework, and (iii) a public leaderboard. The generated sites rated by participants will be released soon. View the UI-Bench leaderboard at https://uibench.ai/leaderboard.","short_abstract":"AI text-to-app tools promise high quality applications and websites in minutes, yet no public benchmark rigorously verifies those claims. We introduce UI-Bench, the first large-scale benchmark that evaluates visual excellence across competing AI text-to-app tools through expert pairwise comparison. Spanning 10 tools, 3...","url_abs":"https://arxiv.org/abs/2508.20410","url_pdf":"https://arxiv.org/pdf/2508.20410v3","authors":"[\"Sam Jung\",\"Agustin Garcinuno\",\"Spencer Mateega\"]","published":"2025-08-28T04:20:00Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[]","project_urls":"[\"https://uibench.ai/leaderboard\"]","has_code":false}