Efficiently Adapting Spoken Language Models for the Singaporean Context
Ng Jia Sheng Jason
Why It Matters
What makes this one worth your time
The study addresses the challenge of adapting language models to specific cultural and linguistic contexts, which is crucial for effective multilingual interaction in diverse environments.
This research adapts spoken language models for multilingual applications in Singapore, achieving significant performance improvements.
Summary
The paper presents an adaptation of an open-source spoken language model (SLM) to the Singaporean context, utilizing techniques like LoRA fine-tuning and a multi-task objective to enhance performance across multilingual speech tasks.
Key contributions
- Development of HTD-multilingual-QA, a large multilingual QA dataset in both text and spoken form.
- Implementation of LoRA fine-tuning for efficient adaptation of spoken language models.
- Demonstration of improved performance metrics, including accent and gender recognition, compared to larger models.
Notable insights
- The use of a surrogate text-QA dataset helps mitigate catastrophic forgetting during model adaptation.
- The adaptation of the CoBa reweighting scheme to speech tasks is a novel approach that enhances model performance.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2607.10092v1 Announce Type: cross Abstract: Spoken language models (SLMs) unify speech perception and reasoning, but adapting them to sensitive domains is underexplored, especially when the original training data is inaccessible and the use case demands multilingual, spoken-query interaction. We adapt an open-source SLM to the Singaporean Home Team context across five speech tasks in Singapore's four official languages, combining LoRA fine-tuning, a surrogate text-QA dataset that guards against catastrophic forgetting, and a multi-task objective that adapts the CoBa reweighting scheme to speech. We also build HTD-multilingual-QA, a 504,853 sample multilingual QA dataset in text and spoken form. The resulting HT-Moonstone (5B) matches or outperforms SLMs up to 7x its size on most tasks, attains the best accent and gender recognition among all models evaluated, and loses under 2\% of its original speech QA ability.