{"ID":2830707,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.09610","arxiv_id":"2512.09610","title":"ImageTalk: Designing a Multimodal AAC Text Generation System Driven by Image Recognition and Natural Language Generation","abstract":"People living with Motor Neuron Disease (plwMND) frequently encounter speech and motor impairments that necessitate a reliance on augmentative and alternative communication (AAC) systems. This paper tackles the main challenge that traditional symbol-based AAC systems offer a limited vocabulary, while text entry solutions tend to exhibit low communication rates. To help plwMND articulate their needs about the system efficiently and effectively, we iteratively design and develop a novel multimodal text generation system called ImageTalk through a tailored proxy-user-based and an end-user-based design phase. The system demonstrates pronounced keystroke savings of 95.6%, coupled with consistent performance and high user satisfaction. We distill three design guidelines for AI-assisted text generation systems design and outline four user requirement levels tailored for AAC purposes, guiding future research in this field.","short_abstract":"People living with Motor Neuron Disease (plwMND) frequently encounter speech and motor impairments that necessitate a reliance on augmentative and alternative communication (AAC) systems. This paper tackles the main challenge that traditional symbol-based AAC systems offer a limited vocabulary, while text entry solutio...","url_abs":"https://arxiv.org/abs/2512.09610","url_pdf":"https://arxiv.org/pdf/2512.09610v1","authors":"[\"Boyin Yang\",\"Puming Jiang\",\"Per Ola Kristensson\"]","published":"2025-12-10T12:57:55Z","proceeding":"cs.HC","tasks":"[\"cs.HC\",\"cs.AI\",\"cs.CV\"]","methods":"[]","has_code":false}