{"work":{"id":"83956045-536a-41ff-af02-b80e2a614eab","openalex_id":null,"doi":null,"arxiv_id":"2503.01743","raw_key":null,"title":"Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs","authors":null,"authors_text":"Microsoft: Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao","year":2025,"venue":"cs.CL","abstract":"We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-quality web and synthetic data, significantly outperforming recent open-source models of similar size and matching the performance of models twice its size on math and coding tasks requiring complex reasoning. This achievement is driven by a carefully curated synthetic data recipe emphasizing high-quality math and coding datasets. Compared to its predecessor, Phi-3.5-Mini, Phi-4-Mini features an expanded vocabulary size of 200K tokens to better support multilingual applications, as well as group query attention for more efficient long-sequence generation. Phi-4-Multimodal is a multimodal model that integrates text, vision, and speech/audio input modalities into a single model. Its novel modality extension approach leverages LoRA adapters and modality-specific routers to allow multiple inference modes combining various modalities without interference. For example, it now ranks first in the OpenASR leaderboard to date, although the LoRA component of the speech/audio modality has just 460 million parameters. Phi-4-Multimodal supports scenarios involving (vision + language), (vision + speech), and (speech/audio) inputs, outperforming larger vision-language and speech-language models on a wide range of tasks. Additionally, we experiment to further train Phi-4-Mini to enhance its reasoning capabilities. Despite its compact 3.8-billion-parameter size, this experimental version achieves reasoning performance on par with or surpassing significantly larger models, including DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B.","external_url":"https://arxiv.org/abs/2503.01743","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-25T05:55:25.037579+00:00","pith_arxiv_id":"2503.01743","created_at":"2026-05-10T05:51:10.232784+00:00","updated_at":"2026-05-25T05:55:25.037579+00:00","title_quality_ok":true,"display_title":"Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs","render_title":"Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs"},"hub":{"state":{"work_id":"83956045-536a-41ff-af02-b80e2a614eab","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":67,"external_cited_by_count":null,"distinct_field_count":11,"first_pith_cited_at":"2024-12-31T07:32:35+00:00","last_pith_cited_at":"2026-05-21T18:00:22+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-05-30T15:51:11.496249+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":10},{"context_role":"baseline","n":4},{"context_role":"dataset","n":1},{"context_role":"other","n":1}],"polarity_counts":[{"context_polarity":"background","n":9},{"context_polarity":"baseline","n":4},{"context_polarity":"unclear","n":2},{"context_polarity":"use_dataset","n":1}],"runs":{"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T18:09:45.713162+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":11},{"title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","work_id":"008df105-2fdd-45d8-857a-8e35868aecb6","shared_citers":9},{"title":"Gemma 3 Technical Report","work_id":"f93e08bf-9e96-409b-8ac6-b8385fd17fd7","shared_citers":8},{"title":"Qwen2.5-Omni Technical Report","work_id":"438f105c-fa9b-44aa-ad52-43acb8045cda","shared_citers":8},{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":7},{"title":"Qwen2-Audio Technical Report","work_id":"c249e63c-cf40-408f-a4ff-fdf68e8cbeb8","shared_citers":6},{"title":"GPT-4o System Card","work_id":"f37bf1c7-4964-4e56-9762-d20da8d9009f","shared_citers":5},{"title":"Language Models (Mostly) Know What They Know","work_id":"8ca58a10-da41-4f70-baae-7e449512e345","shared_citers":4},{"title":"LoRA: Low-Rank Adaptation of Large Language Models","work_id":"0426219a-789e-4964-adc8-a04538510818","shared_citers":4},{"title":"Qwen Technical Report","work_id":"bb1fd52f-6b2f-437c-9516-37bdf6eb9be8","shared_citers":4},{"title":"gpt-oss-120b & gpt-oss-20b Model Card","work_id":"178c1f7e-4f19-4392-a45d-45a6dfa88ead","shared_citers":3},{"title":"Mmau-pro: A challenging and comprehensive benchmark for holistic evaluation of audio general intelligence","work_id":"34fbbc93-cd79-4b11-a99d-32d597b71a37","shared_citers":3},{"title":"Qwen2.5 Technical Report","work_id":"d8432992-4980-4a81-85c7-9fa2c2b87f85","shared_citers":3},{"title":"Qwen3-Omni Technical Report","work_id":"ae43e594-8bab-4471-b6af-92a300f6a048","shared_citers":3},{"title":"Voxtral.arXiv preprint arXiv:2507.13264","work_id":"d8af79fd-31af-429f-b7c7-b98cea5e46ed","shared_citers":3},{"title":"Audio flamingo 3: Advancing audio intelligence with fully open large audio language models","work_id":"67c2892d-8e27-4da1-8198-71c48e673e96","shared_citers":2},{"title":"Audiomarathon: A comprehensive benchmark for long-context audio understanding and efficiency in audio llms","work_id":"09d1fac2-ebc7-4758-8efd-e47325ca4f45","shared_citers":2},{"title":"Available: https://arxiv.org/abs/2203.15591","work_id":"08f20a51-8071-4da8-aeb8-740367682349","shared_citers":2},{"title":"Available: https://arxiv.org/abs/2505.08699","work_id":"ae35a2e9-11a4-4654-a808-1a57efb893aa","shared_citers":2},{"title":"Bradley Efron and Robert J Tibshirani.An introduction to the bootstrap, volume","work_id":"414143a1-235c-49ec-a733-5cff2faefa32","shared_citers":2},{"title":"Building a taiwanese mandarin spoken language model: A first attempt","work_id":"124b430d-5f85-4a49-a1a2-0e0ff371dbcb","shared_citers":2},{"title":"Constitutional AI: Harmlessness from AI Feedback","work_id":"faaaa4e0-2676-4fac-a0b4-99aef10d2095","shared_citers":2},{"title":"DAPO: An Open-Source LLM Reinforcement Learning System at Scale","work_id":"64019d00-0b11-4bbd-b173-b46c8fad0157","shared_citers":2},{"title":"DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models","work_id":"c5006563-f3ec-438a-9e35-b7b484f34828","shared_citers":2}],"time_series":[{"n":33,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T18:10:13.055892+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T18:09:55.835920+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs","claims":[{"claim_text":"We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-quality web and synthetic data, significantly outperforming recent open-source models of similar size and matching the performance of models twice its size on math and coding tasks requiring complex reasoning. This achievement is driven by a carefully curated synthetic data recipe emphasizing high-quality math and coding datasets. Compared to its predecessor, Phi-3.5-Mini, Phi-4-Mini features an expanded vocabulary size of","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T18:09:36.163259+00:00"}},"summary":{"title":"Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs","claims":[{"claim_text":"We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-quality web and synthetic data, significantly outperforming recent open-source models of similar size and matching the performance of models twice its size on math and coding tasks requiring complex reasoning. This achievement is driven by a carefully curated synthetic data recipe emphasizing high-quality math and coding datasets. Compared to its predecessor, Phi-3.5-Mini, Phi-4-Mini features an expanded vocabulary size of","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":11},{"title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","work_id":"008df105-2fdd-45d8-857a-8e35868aecb6","shared_citers":9},{"title":"Gemma 3 Technical Report","work_id":"f93e08bf-9e96-409b-8ac6-b8385fd17fd7","shared_citers":8},{"title":"Qwen2.5-Omni Technical Report","work_id":"438f105c-fa9b-44aa-ad52-43acb8045cda","shared_citers":8},{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":7},{"title":"Qwen2-Audio Technical Report","work_id":"c249e63c-cf40-408f-a4ff-fdf68e8cbeb8","shared_citers":6},{"title":"GPT-4o System Card","work_id":"f37bf1c7-4964-4e56-9762-d20da8d9009f","shared_citers":5},{"title":"Language Models (Mostly) Know What They Know","work_id":"8ca58a10-da41-4f70-baae-7e449512e345","shared_citers":4},{"title":"LoRA: Low-Rank Adaptation of Large Language Models","work_id":"0426219a-789e-4964-adc8-a04538510818","shared_citers":4},{"title":"Qwen Technical Report","work_id":"bb1fd52f-6b2f-437c-9516-37bdf6eb9be8","shared_citers":4},{"title":"gpt-oss-120b & gpt-oss-20b Model Card","work_id":"178c1f7e-4f19-4392-a45d-45a6dfa88ead","shared_citers":3},{"title":"Mmau-pro: A challenging and comprehensive benchmark for holistic evaluation of audio general intelligence","work_id":"34fbbc93-cd79-4b11-a99d-32d597b71a37","shared_citers":3},{"title":"Qwen2.5 Technical Report","work_id":"d8432992-4980-4a81-85c7-9fa2c2b87f85","shared_citers":3},{"title":"Qwen3-Omni Technical Report","work_id":"ae43e594-8bab-4471-b6af-92a300f6a048","shared_citers":3},{"title":"Voxtral.arXiv preprint arXiv:2507.13264","work_id":"d8af79fd-31af-429f-b7c7-b98cea5e46ed","shared_citers":3},{"title":"Audio flamingo 3: Advancing audio intelligence with fully open large audio language models","work_id":"67c2892d-8e27-4da1-8198-71c48e673e96","shared_citers":2},{"title":"Audiomarathon: A comprehensive benchmark for long-context audio understanding and efficiency in audio llms","work_id":"09d1fac2-ebc7-4758-8efd-e47325ca4f45","shared_citers":2},{"title":"Available: https://arxiv.org/abs/2203.15591","work_id":"08f20a51-8071-4da8-aeb8-740367682349","shared_citers":2},{"title":"Available: https://arxiv.org/abs/2505.08699","work_id":"ae35a2e9-11a4-4654-a808-1a57efb893aa","shared_citers":2},{"title":"Bradley Efron and Robert J Tibshirani.An introduction to the bootstrap, volume","work_id":"414143a1-235c-49ec-a733-5cff2faefa32","shared_citers":2},{"title":"Building a taiwanese mandarin spoken language model: A first attempt","work_id":"124b430d-5f85-4a49-a1a2-0e0ff371dbcb","shared_citers":2},{"title":"Constitutional AI: Harmlessness from AI Feedback","work_id":"faaaa4e0-2676-4fac-a0b4-99aef10d2095","shared_citers":2},{"title":"DAPO: An Open-Source LLM Reinforcement Learning System at Scale","work_id":"64019d00-0b11-4bbd-b173-b46c8fad0157","shared_citers":2},{"title":"DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models","work_id":"c5006563-f3ec-438a-9e35-b7b484f34828","shared_citers":2}],"time_series":[{"n":33,"year":2026}],"dependency_candidates":[]},"authors":[]}}