{"ID":5552897,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-04T00:55:20.191773895Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.00309","arxiv_id":"2607.00309","title":"A Text-Steerable Instrument for Sketching Procedural Soundscapes via Language Models","abstract":"We present a real-time musical interface that converts natural-language scene descriptions into evolving procedural soundscapes. A performer types a prompt such as \"warm jazz cafe at midnight\" and steers it through direct parameter adjustments - stepping brightness down, switching a rhythm style - each producing a predictable, audible shift without re-prompting. Where GPU-bound text-to-audio systems synthesize monolithic waveforms, our instrument generates human-readable configurations over a categorical schema, enabling fine-grained performer control; most valid combinations are designed to sound musically coherent. Three interchangeable backends - embedding retrieval for sub-second CPU-only use, hosted LLMs via API, and a fine-tuned 270M local model - all emit the same schema. A live generator architecture continuously emits audio while resolving new instructions in the background, crossfading seamlessly when ready; even when an LLM takes 5-12 seconds to respond, the audience hears uninterrupted sound - reframing text-to-music as an ongoing performable stream rather than a one-shot generation. We evaluate text-audio semantic alignment using LAION-CLAP on held-out prompts as a technical proxy, finding that retrieval-based configuration outperforms random valid configurations on this metric, while noting that LAION-CLAP also informed retrieval-map construction. We report performance observations, informal listener feedback, and release materials for the SDK, dataset artifacts, model, and audiovisual performance interface.","short_abstract":"We present a real-time musical interface that converts natural-language scene descriptions into evolving procedural soundscapes. A performer types a prompt such as \"warm jazz cafe at midnight\" and steers it through direct parameter adjustments - stepping brightness down, switching a rhythm style - each producing a pred...","url_abs":"https://arxiv.org/abs/2607.00309","url_pdf":"https://arxiv.org/pdf/2607.00309v1","authors":"[\"Prabal Gupta\"]","published":"2026-07-01T01:21:42Z","proceeding":"cs.SD","tasks":"[\"cs.SD\",\"cs.CL\",\"cs.HC\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}