{"ID":2826454,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.18551","arxiv_id":"2512.18551","title":"Neologism Learning as a Parameter-Efficient Alternative to Fine-Tuning for Model Steering","abstract":"In language modeling, neologisms are new tokens trained to represent a concept not already included in a given model's vocabulary. Neologisms can be used to encourage specific behavior in models, for example by appending prompts with \"Give me a neologism answer.\" Behavioral steering can also be achieved through fine-tuning, albeit with more compute and less flexibility: learning a neologism only trains d parameters and allows the user to still access the model's default behavior. We compare the performance of neologism learning against low-rank adaptation (LoRA) fine-tuning, finding that neologisms outperform fine-tuned models under a matched training setup (same data and hyperparameters). We also investigate self-verbalizations of neologisms, and observe that the model will occasionally make up its own new words when asked about a neologism.","short_abstract":"In language modeling, neologisms are new tokens trained to represent a concept not already included in a given model's vocabulary. Neologisms can be used to encourage specific behavior in models, for example by appending prompts with \"Give me a neologism answer.\" Behavioral steering can also be achieved through fine-tu...","url_abs":"https://arxiv.org/abs/2512.18551","url_pdf":"https://arxiv.org/pdf/2512.18551v1","authors":"[\"Sungjoon Park\",\"Varun Ramamurthi\",\"Owen Terry\"]","published":"2025-12-21T00:45:23Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Language Model\",\"LoRA\"]","has_code":false}