{"ID":2824501,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.23847","arxiv_id":"2512.23847","title":"A Test of Lookahead Bias in LLM Forecasts","abstract":"We develop a statistical test to detect lookahead bias in economic forecasts generated by large language models (LLMs). Using state-of-the-art pre-training data detection techniques, we estimate the likelihood that a given prompt appeared in an LLM's training corpus, a statistic we term Lookahead Propensity (LAP). We formally show that a positive correlation between LAP and forecast accuracy indicates the presence and magnitude of lookahead bias, and apply the test to two forecasting tasks: news headlines predicting stock returns and earnings call transcripts predicting capital expenditures. Our test provides a cost-efficient, diagnostic tool for assessing the validity and reliability of LLM-generated forecasts.","short_abstract":"We develop a statistical test to detect lookahead bias in economic forecasts generated by large language models (LLMs). Using state-of-the-art pre-training data detection techniques, we estimate the likelihood that a given prompt appeared in an LLM's training corpus, a statistic we term Lookahead Propensity (LAP). We f...","url_abs":"https://arxiv.org/abs/2512.23847","url_pdf":"https://arxiv.org/pdf/2512.23847v1","authors":"[\"Zhenyu Gao\",\"Wenxi Jiang\",\"Yutong Yan\"]","published":"2025-12-29T20:20:04Z","proceeding":"q-fin.GN","tasks":"[\"q-fin.GN\",\"cs.LG\",\"q-fin.TR\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}