{"ID":2831221,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.08646","arxiv_id":"2512.08646","title":"QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models","abstract":"We introduce QSTN, an open-source Python framework for systematically generating responses from questionnaire-style prompts to support in-silico surveys and annotation tasks with large language models (LLMs). QSTN enables robust evaluation of questionnaire presentation, prompt perturbations, and response generation methods. Our extensive evaluation (\u003e40 million survey responses) shows that question structure and response generation methods have a significant impact on the alignment of generated survey responses with human answers. We also find that answers can be obtained for a fraction of the compute cost, by changing the presentation method. In addition, we offer a no-code user interface that allows researchers to set up robust experiments with LLMs \\emph{without coding knowledge}. We hope that QSTN will support the reproducibility and reliability of LLM-based research in the future.","short_abstract":"We introduce QSTN, an open-source Python framework for systematically generating responses from questionnaire-style prompts to support in-silico surveys and annotation tasks with large language models (LLMs). QSTN enables robust evaluation of questionnaire presentation, prompt perturbations, and response generation met...","url_abs":"https://arxiv.org/abs/2512.08646","url_pdf":"https://arxiv.org/pdf/2512.08646v2","authors":"[\"Maximilian Kreutner\",\"Jens Rupprecht\",\"Georg Ahnert\",\"Ahmed Salem\",\"Markus Strohmaier\"]","published":"2025-12-09T14:35:26Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.CY\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}