{"ID":2845645,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.03186","arxiv_id":"2511.03186","title":"Adobe Summit Concierge Evaluation with Human in the Loop","abstract":"Generative AI assistants offer significant potential to enhance productivity, streamline information access, and improve user experience in enterprise contexts. In this work, we present Summit Concierge, a domain-specific AI assistant developed for Adobe Summit. The assistant handles a wide range of event-related queries and operates under real-world constraints such as data sparsity, quality assurance, and rapid deployment. To address these challenges, we adopt a human-in-the-loop development workflow that combines prompt engineering, retrieval grounding, and lightweight human validation. We describe the system architecture, development process, and real-world deployment outcomes. Our experience shows that agile, feedback-driven development enables scalable and reliable AI assistants, even in cold-start scenarios.","short_abstract":"Generative AI assistants offer significant potential to enhance productivity, streamline information access, and improve user experience in enterprise contexts. In this work, we present Summit Concierge, a domain-specific AI assistant developed for Adobe Summit. The assistant handles a wide range of event-related queri...","url_abs":"https://arxiv.org/abs/2511.03186","url_pdf":"https://arxiv.org/pdf/2511.03186v1","authors":"[\"Yiru Chen\",\"Sally Fang\",\"Sai Sree Harsha\",\"Dan Luo\",\"Vaishnavi Muppala\",\"Fei Wu\",\"Shun Jiang\",\"Kun Qian\",\"Yunyao Li\"]","published":"2025-11-05T05:05:24Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[]","has_code":false}