Arabic is the fifth most spoken language in the world, with over 400 million speakers across a vast geographic spread of dialects. Yet Arabic remains significantly underrepresented in speech large language models. Most speech-LLM benchmarks and training corpora are built around English and Chinese. When Arabic is included at all, it is often limited to automatic speech recognition, with little coverage of dialect identification, speech emotion recognition, age recognition, or speech question answering. The few existing Arabic speech-LLM efforts, like the Octopus family, cover only three tasks and rely on predominantly private datasets that restrict reproducibility. Yingzhi Wang, Reem Alhazzani, and Muhammad Alqurishi at Elm Company in Saudi Arabia introduce Nuha-Speech to change this: a comprehensive initiative spanning dataset construction, model training, and systematic evaluation for general-purpose Arabic speech-LLMs.

Why Arabic speech-LLMs face unique infrastructure gaps

The scarcity of Arabic-capable speech-LLMs reflects two concrete problems. First, publicly accessible Arabic speech corpora for non-ASR tasks remain extremely limited. There is a striking absence of Arabic speech datasets of sufficient scale for tasks like speech emotion recognition or speech question answering. Arabic speech instruction-tuning data is almost non-existent: most speech instruction-following corpora are in English, and although some have been translated into Arabic, genuine Arabic speech-instruction pairs are nearly impossible to find. Second, there is no widely adopted evaluation benchmark for Arabic speech LLMs. Existing benchmarks are centered on mainstream languages, with most speech tasks derived from English datasets, making it difficult to assess models' understanding of Arabic.

The Qwen family offers some Arabic support. Qwen2-Audio relies on Whisper-large-v3, which supports Arabic, but its demos and test cases are only in English and Chinese. Qwen2.5-Omni demonstrates speech instruction following in over 29 languages, but its public evaluation focuses on English and Chinese, leaving Arabic ability unverified. Qwen3-Omni explicitly includes Arabic among 19 supported speech-input languages, making it the strongest baseline for Arabic speech understanding among available models.

A 1.5 million sample training corpus across seven tasks

Nuha-Speech constructs a large-scale Arabic Speech Question-Answering corpus comprising over 1.5 million training samples formatted as {speech, instruction, output} tuples. Each task uses a broad instruction set designed with GPT-5 to support zero-shot generalization and avoid overfitting to narrow prompts.

ASR provides the foundation. Four large-scale public datasets are used: MGB-2 (278K samples), MASC (263K), SADA (171K), and Common Voice (78K), with segments outside the 1 to 10 second range removed. This yields roughly 790K ASR training samples spanning multiple Arabic dialects.

AST (Arabic-to-English speech translation) combines 5K samples from CoVoST-v2 with 150K samples from the ASR dataset whose transcriptions were translated into English using Qwen3-32B. This addresses the severe shortage of Arabic-to-English speech translation data.

SQA (speech question answering) targets the core conversational ability. Using MGB-2 transcriptions as input, the authors generate 150K closed-ended QA pairs (focused on structured query handling and exact content retrieval) and 150K open-ended QA pairs (promoting cross-sentence reasoning and expressive language generation) using Qwen3-32B.

DI (dialect identification) draws 150K samples from ADI-17, which classifies 17 Arabic dialects. This task is crucial because Arabic's dialectal variation significantly affects downstream performance in multilingual and multi-dialectal settings.

SER (speech emotion recognition) addresses a long-standing data scarcity problem. Using ElevenLabs' emotion-conversion TTS, the authors synthesize 20K emotional samples across four balanced categories (angry, happy, sad, neutral), with each category synthesized by 8 separate speakers to reduce speaker-specific bias. Style exaggeration and stronger speaker similarity control are activated during synthesis.

AR (age recognition) defines three groups: young (under 18), adult (18 to 60), and elder (above 60). The authors extract all available young and elder samples from SADA and Common Voice, add balanced adult samples for 10K total, then synthesize another 10K samples using ElevenLabs with 10 distinct speakers per group.

GR (gender recognition) uses 51K samples from SADA with equal male and female distribution.

Multi-SQA adds multi-turn speech question answering. First, 5K samples merge gender, age, and ASR annotations from SADA into speech-analytics-style multi-turn queries. Second, 10K emotion reasoning QA pairs are generated using Qwen3-32B, where the model first recognizes speech emotion, then transcribes content, then infers emotion correlation between modalities.

Two-stage training curriculum on Qwen-Omni models

The authors use three Qwen-Omni model variants as baselines: Qwen2.5-Omni-3B (Whisper-large-v3 encoder plus Qwen2.5 LLM), Qwen2.5-Omni-7B (same architecture at larger scale), and Qwen3-Omni-30B (AuT audio encoder plus Qwen3 MoE architecture). The choice reflects both the state of Arabic speech-LLM infrastructure and the need for systematic comparison across model scales.

Training follows a two-stage curriculum. In Stage 1, the model trains only on ASR (790K samples, 3 epochs) to establish audio-text mapping as the foundation for subsequent tasks. In Stage 2, all tasks are trained together, with ASR reduced to a 200K subset to preserve ASR performance while improving task balance (2 epochs). The audio encoders are frozen in both stages, and only the LLM is trained with LoRA (rank 8, alpha 32, all linear layers) to stabilize training.

For Qwen3-Omni-30B, Stage 1 is skipped because the model already demonstrates competitive ASR performance on the Arabic ASR leaderboard. The MoE router's load balancing loss coefficient is set to 1e-3, batch size to 8, and learning rate to 1e-4. For the smaller models, batch size is 16 with the same learning rate.

The resulting models are named Nuha-Speech-3B, Nuha-Speech-7B, and Nuha-Speech-30B.

Results: dramatic gains on paralinguistic tasks

The benchmark evaluates all seven tasks using task-appropriate metrics. ASR uses average WER on the Open Universal Arabic ASR Leaderboard (47K test samples across 6 datasets). AST uses BLEU and Gemma-based sentence similarity on CoVoST-v2 Ar-En (2.3K samples). SQA uses LLM-as-Judge scoring (LLaMA-3.3-70B-Instruct rating relevance, correctness, and conciseness on a 0 to 3 scale) on 1.2K synthesized utterances. DI, SER, AR, and GR use accuracy on their respective test sets.

The results show clear improvements across all tasks after fine-tuning, with the most dramatic gains on paralinguistic tasks where baseline models started near random guessing. Dialect identification jumps from 9.70% to 57.83% for the 3B model, from 13.66% to 72.65% for 7B, and from 16.78% to 76.02% for 30B. Age recognition goes from 41.75% to 83.00% for 3B, from 33.33% to 73.00% for 7B, and from 42.67% to 85.75% for 30B. These are multi-fold increases from baseline performance that was close to random.

For speech emotion recognition, baselines already showed adequate initial performance (51.93% for 3B, 72.14% for 7B, 80.23% for 30B), and fine-tuning pushes all models to excellent levels (88.14%, 86.64%, 84.88% respectively). Gender recognition reaches near-perfect accuracy: 98.85% for 3B, 99.50% for 7B, and 94.30% for 30B.

For semantic tasks, the improvements are more modest but consistent. ASR WER drops from 74.74% to 34.91% for the 7B model and from 30.71% to 29.66% for 30B. AST BLEU improves from 39.27 to 48.32 for 3B, from 42.08 to 50.60 for 7B, and from 48.71 to 51.73 for 30B. SQA scores improve across all models, with Nuha-Speech-7B marginally outperforming the others at 2.66.

The pattern is clear: paralinguistic tasks exhibited more substantial performance gains than semantic tasks after fine-tuning, suggesting that Arabic speech paralinguistics was under-represented in the pre-training phase of the baseline models. The 30B model showed smaller gains because it had already acquired strong pre-trained semantic capabilities, while the 3B and 7B models benefited more from the diverse training tasks.

What the baseline comparison reveals

An important detail: the authors also tested Qwen2-Audio-7B-Instruct as a baseline, but after fine-tuning on the same corpus, it consistently underperformed the fine-tuned Qwen2.5-Omni-7B across all tasks. They excluded it from the final benchmark to maintain a competitive set of baseline models. This suggests that the Qwen2.5-Omni architecture, with its Whisper-large-v3 encoder, provides a better starting point for Arabic speech-LLM fine-tuning than the earlier Qwen2-Audio architecture.

The 3B model's results are particularly notable. Starting from near-random dialect identification (9.70%) and mediocre age recognition (41.75%), fine-tuning pushes these to 57.83% and 83.00% respectively, while ASR WER drops from unreported to 36.83%. The 3B model achieves 98.85% gender recognition accuracy. This demonstrates that even small models can acquire substantial Arabic speech understanding from the 1.5 million sample corpus, making deployment on resource-constrained devices feasible.

Limitations the authors acknowledge

The training corpus relies heavily on synthesized data for SER and AR tasks, which introduces a domain gap between synthesized and natural speech. The emotion conversion TTS, while effective, produces speech that may not fully represent the distribution of natural emotional expression in Arabic. The SQA evaluation uses synthesized speech passages from a textual QA dataset, which may not capture the acoustic characteristics of natural conversational speech.

The benchmark evaluates only text outputs, not speech outputs. A truly general-purpose Arabic speech-LLM would need to generate Arabic speech responses, not just text. The current work establishes the speech-understanding foundation but leaves speech generation to future work.

The instruction sets are designed using GPT-5, which introduces a dependency on a proprietary model for data curation. While the training data itself is largely from public sources, the instruction design pipeline is not fully reproducible without access to GPT-5.

The evaluation covers seven tasks but does not include cross-lingual transfer evaluation, code-switching (mixing Arabic with English or French in speech), or long-form speech understanding. These are realistic scenarios for Arabic speakers that the current benchmark does not capture.

What this means for Arabic NLP and speech technology

Nuha-Speech establishes the first systematic Arabic speech multi-task benchmark, providing a standard for evaluating future Arabic speech-LLMs. The 1.5 million sample training corpus, constructed primarily from public datasets with transparent curation strategies, makes the work reproducible by the research community. The two-stage training curriculum demonstrates a practical approach to building Arabic speech-LLMs under resource constraints.

The dramatic gains on dialect identification are particularly significant for Arabic NLP. Arabic dialectal variation is a major challenge: a model that cannot identify whether speech is Egyptian, Levantine, Gulf, or Maghrebi will struggle with downstream tasks. Moving from near-random to 76% accuracy on 17-dialect classification with the 30B model, and to 58% with the 3B model, provides a foundation for dialect-aware Arabic speech systems.

The use of ElevenLabs TTS for data augmentation on SER and AR tasks demonstrates a practical approach to the Arabic data scarcity problem. Rather than waiting for large-scale annotated Arabic speech datasets to appear, the authors synthesize balanced training data using commercially available TTS with emotion and age control. This strategy is not limited to Arabic: it provides a template for other under-resourced languages where natural speech data with paralinguistic annotations is scarce.

For developers building Arabic voice assistants, customer service systems, or accessibility tools, the Nuha-Speech models provide a starting point. The 3B model is small enough for on-device deployment while achieving reasonable performance across all tasks. The 30B model provides stronger performance for cloud-based applications. The systematic evaluation across seven tasks gives developers a clear picture of what each model can and cannot do, rather than relying on anecdotal demonstrations.

The broader lesson is that multilingual speech-LLM development requires dedicated infrastructure for each language family. Arabic's dialectal diversity, right-to-right script, and unique phonological properties mean that English-centric training pipelines do not transfer cleanly. Nuha-Speech demonstrates that building this infrastructure, while resource-intensive, is feasible with public data, transparent methods, and systematic evaluation.

Read the paper on arXiv