{"ID":2828666,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.15798","arxiv_id":"2512.15798","title":"DP-Bench: A Benchmark for Evaluating Data Product Creation Systems","abstract":"A data product is created with the intention of solving a specific problem, addressing a specific business usecase or meeting a particular need, going beyond just serving data as a raw asset. Data products enable end users to gain greater insights about their data. Since it was first introduced over a decade ago, there has been considerable work, especially in industry, to create data products manually or semi-automatically. However, there exists hardly any benchmark to evaluate automatic data product creation. In this work, we present a benchmark, first of its kind, for this task. We call it DP-Bench. We describe how this benchmark was created by taking advantage of existing work in ELT (Extract-Load-Transform) and Text-to-SQL benchmarks. We also propose a number of LLM based approaches that can be considered as baselines for generating data products automatically. We make the DP-Bench and supplementary materials available in https://huggingface.co/datasets/ibm-research/dp-bench .","short_abstract":"A data product is created with the intention of solving a specific problem, addressing a specific business usecase or meeting a particular need, going beyond just serving data as a raw asset. Data products enable end users to gain greater insights about their data. Since it was first introduced over a decade ago, there...","url_abs":"https://arxiv.org/abs/2512.15798","url_pdf":"https://arxiv.org/pdf/2512.15798v1","authors":"[\"Faisal Chowdhury\",\"Sola Shirai\",\"Sarthak Dash\",\"Nandana Mihindukulasooriya\",\"Horst Samulowitz\"]","published":"2025-12-16T19:19:01Z","proceeding":"cs.DB","tasks":"[\"cs.DB\",\"cs.CL\"]","methods":"[\"Large Language Model\"]","has_code":false}