Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Specialized LLMs now consistently outperform general models, survey argues

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A survey of 24 specialized LLMs (2022-2025) claims a shift from domain fine-tuning to native architectures, but the synthesis is undermined by citation errors and selection bias.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Plausible survey narrative, but the paper's own evidence doesn't support its core claims—needs a serious rewrite before I'd trust it. the 5 major comments →

arxiv 2508.19667 v1 pith:R37M6P2D submitted 2025-08-27 cs.CL cs.AI

Survey of Specialized Large Language Model

Chenghan Yang , Ruiyu Zhao , Yang Liu , Ling Jiang This is my paper
classification cs.CL cs.AI
keywords specialized large language modelsdomain adaptationdomain-native architecturemixture-of-expertsparameter efficiencydomain-specific benchmarksmultimodal LLMse-commerce customer service
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey of 48 specialized large language models released between 2022 and 2025 argues that domain specialization has become a paradigm shift in AI: instead of merely fine-tuning general models, builders are now designing domain-native systems from the ground up. The paper organizes the field into four modular axes—specialized datasets, training architectures, evaluation standards, and components such as retrieval, tools, and memory—and reads the recent wave of healthcare, finance, legal, math, and multimodal models as consistent evidence that task-aligned designs beat general-purpose counterparts on domain benchmarks. It matters because it gives practitioners a map of where specialization works and what techniques carry the gains, and it identifies e-commerce customer service as an area where the recipe has not yet been applied.

Core claim

On its own terms, the paper establishes a developmental narrative: specialized LLMs moved through three phases—continued pretraining on domain corpora (BioGPT), architecture-level domain adaptation (Med-PaLM 2, BloombergGPT), and hybrid, agent-oriented systems with sparse expert routing, retrieval, tool use, and explicit memory (BLADE, Self-MoE, GLM-4.5). Across these phases the paper claims specialized models now consistently outperform general-purpose LLMs in professional domains by combining task-aligned architectures, domain-adaptive training, and efficient inference, and that smaller carefully designed models can beat much larger general ones.

What carries the argument

The organizing device is a modular taxonomy of specialization: dataset specialization (synthetic and multimodal corpora), training-architecture specialization (parameter-efficient tuning, mixture-of-experts, quantization and compression, reasoning in the forward pass), evaluation specialization (domain benchmarks like MedBench, HumanEval, and perplexity-based filters), and component-level specialization (retrieval, tool calling, memory layers). This four-part framework is what converts a list of 48 models into an argument that specialization is a coherent, accelerating design space rather than isolated fine-tuning efforts.

Load-bearing premise

The load-bearing premise is that the performance numbers and efficiency figures the survey quotes from secondary sources are correctly attributed and comparable; if any of those figures is misquoted or taken out of context, the case for a systematic specialization advantage loses its evidentiary foundation.

What would settle it

Compile the five most-cited specialized models and their general-purpose baselines, run them on one shared domain benchmark per field with matched inference budgets, and check whether the specialized models still win; a single field where the general model wins by a wide margin, or a documented misquotation in the cited efficiency numbers (e.g., the 70% KV-cache reduction), would undercut the 'consistently outperform' claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the trend holds, procurement of LLMs for medicine, law, finance, and engineering should favor domain-native models over general models with prompting, because the reported gap is systematic, not incidental.
  • Parameter-efficiency results (7.3x activation-memory reduction with Mixture-of-LoRAs, near-lossless 3-bit SpQR, 25% parameter deletion via SliceGPT) imply that small specialized models can be deployed on modest hardware without sacrificing domain performance.
  • Evaluation practice will shift from general metrics to layered domain-specific checks: perplexity as a cheap first-pass filter, then domain benchmarks, then expert and interactive assessment.
  • The e-commerce pipeline proposed in Section 4.2—fine-tune a general model on high-quality domain corpora, validate with perplexity-like metrics, and benchmark with Ecom-Bench pass@k—is a concrete, low-cost entry path for the gap the paper identifies.
  • Future specialized models are expected to be lightweight, continually updated, multimodal, and embedded in agent systems that take actions and make decisions autonomously, with interpretability and safety as constraints.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The survey's 'consistently outperform' claim is a synthesis of heterogeneous leaderboard numbers; a fair test would be a single uniform benchmark administered to both specialized and general models across domains, which the paper itself does not provide.
  • The cited efficiency numbers (e.g., 70% KV-cache memory reduction, 7.3x activation memory cut) come from separate papers with different baselines; combining them into one narrative overstates the coherence of the evidence until re-measured in a common setting.
  • If the e-commerce route proposed in Section 4.2 works, the same recipe—domain corpus, cheap validation, efficient fine-tuning, domain benchmark—should transfer to other underserved verticals such as education and industrial maintenance, where the paper lists models but few benchmarks.
  • A testable extension would be measuring perplexity drift on domain corpora before and after continued pretraining: the paper implies PPL is a sensitive first-pass signal, which predicts that domain fine-tuning should lower PPL on held-out domain text while leaving general PPL roughly unchanged.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This survey aims to chart the evolution of specialized large language models between 2022 and 2025, arguing that the field has moved from simple domain fine-tuning to domain-native architectures and efficient, agent-oriented designs. It organizes the material into a timeline, a tabular overview of 24 models, and thematic sections on datasets, training architectures, evaluation, retrieval augmentation, tool use, and memory. The authors conclude that specialized LLMs now consistently outperform general-purpose counterparts, and they identify e-commerce customer service as an under-explored application area with a roadmap for future work.

Significance. If the factual basis were reliable, the survey would be a useful synthesis of a fast-moving area. It identifies several genuine trends—parameter-efficient fine-tuning, sparse MoE, quantization, retrieval-augmented training, tool-use specialization, and memory-augmented agents—and it makes a practical connection to e-commerce applications. However, the paper's evidentiary quality is currently too weak to support its central claims. The claimed model count does not match the table, several key model-to-paper attributions are wrong, quantitative statements are presented without the necessary context or traceable sources, and the headline conclusion about consistent outperformance over general-purpose models is not derived from any systematic comparison. The value of the survey depends entirely on accurate citation and faithful reporting of secondary results, so these issues are load-bearing.

major comments (5)
  1. [§1, Table 1] The Introduction states that the survey 'systematically examine[s] 48 cutting-edge models,' but Table 1 contains 24 rows. Moreover, several rows are not specialized LLMs at all (e.g., KV cache OTT is an inference optimization; GLM-4.5 is an agent foundation model), while models discussed in the text (Lawyer LLaMA, FinBERT-QA) are absent from the table. This mismatch makes the scope unverifiable and undermines the 'systematic' characterization. Please reconcile the count or state explicit inclusion/exclusion criteria.
  2. [§4.1, Abstract] The central conclusion that specialized LLMs 'now consistently outperform their general-purpose counterparts' is not supported by the evidence presented. Sections 2 and 3 report individual successes (e.g., Med-PaLM 2 on USMLE, BloombergGPT on finance tasks) but never provide a controlled comparison of a specialized model and a strong general-purpose model on the same benchmark under the same evaluation conditions. The quantitative comparisons in §3 are mostly ablations of techniques (MoE vs. dense, LoRA variants, quantization), not specialized-versus-general model comparisons. Either add a dedicated comparison table with matched benchmarks or substantially qualify the claim.
  3. [§2, References] Several model-to-paper attributions are incorrect. EM-LLM is cited as (Bolton et al., 2024), but that reference is the BioMedLM paper. Self-MoE is cited as (Yang et al., 2024), but the listed reference is 'EmoLLM.' BLADE is cited as (Xu et al., 2024b), which is titled 'SA-MDKIF,' not BLADE. These are not minor typographical issues: the historical narrative and the claimed 'domain-native' trend rest on these attributions. The bibliography needs a full fact-checking pass, and unsupported attributions should be corrected or removed.
  4. [§3.2, §2, §3.1] Quantitative claims are repeatedly presented without the necessary context or source traceability. Examples: 'KV cache OTT ... reduced memory usage by 70%' is stated without model, sequence length, or quantization setting; 'Mixture-of-LoRAs ... cutting activation memory by 7.3× yet preserving 97% of the legal-domain F1' has no accompanying benchmark details; SpQR is described as 'reducing GPU memory by 3.9' without units (presumably 3.9×) and with no dataset for the 'near-lossless perplexity' claim. In §3.1, 'Constitutional-Poly' is attributed to Bai et al. (2022), but that reference is the Constitutional AI paper and does not describe a multi-turn debate among three specialist LLMs or the consistency scores 0.94 vs. 0.71. Every number needs a traceable primary source and the original units/conditions.
  5. [§4.2] The statement that 'few studies have been applied to the field of e-Commerce customer service' is inconsistent with the survey itself: Table 1 lists EcomGPT (2023), and the same section then cites the authors' own MindFlow and Ecom-Bench as the only recent e-commerce works. If the intended claim is that e-commerce customer service is less studied than healthcare or finance, it should be supported by a comparative literature count or an explicit scope statement. The self-citations should also be contextualized rather than appearing to define the only prior work.
minor comments (5)
  1. [Abstract] The phrase 'with specialized models consistently performance gains on domain-specific benchmarks' is ungrammatical; it should be 'consistently achieve performance gains.'
  2. [§2, Table 1] The sentence 'Representative paper and reletive field have been displayed in the table 1' contains a typo ('reletive') and is awkwardly placed. Table 1 also does not include all models mentioned in the text, such as Lawyer LLaMA and FinBERT-QA.
  3. [Figure 1] The timeline includes entries like 'KV-cache OTT' and 'GLM-4.5' that the text itself categorizes as optimization/agent models rather than specialized LLMs. The figure caption should align with the inclusion criteria in Table 1, or the criteria should be explained.
  4. [References] Several reference entries are incomplete or nonstandard, e.g., author lists truncated to 'and 1 others' and repeated initials. Please update to a consistent style and verify all DOIs/arXiv IDs.
  5. [§3.3] The claim that perplexity 'correlates strongly with downstream pass@k gains' is presented as a general fact without citation or qualification. This is a strong empirical assertion and needs a reference or a caveat.

Circularity Check

0 steps flagged

No significant circularity: the survey aggregates external results; the only self-citations (MindFlow, Ecom-Bench) are in a future-work section and are not load-bearing.

full rationale

This is a survey, not a derivation. The central claim that specialized LLMs 'consistently outperform' general-purpose counterparts (Section 4.1) is an interpretation of reported benchmark results from many external sources, not a result derived from the paper's own equations, fitted parameters, or definitions. No quantity is defined in terms of another and then predicted back; no parameter is fitted to a subset of data and then presented as a prediction on a closely related quantity. The only self-citations appear in Section 4.2: MindFlow (Gong et al., 2025) and Ecom-Bench (Wang et al., 2025), both involving overlapping authors. These support the observation that e-commerce customer service is relatively underexplored and that the authors' own deployed model uses a generic LLM; this is a proposal for future work, not a load-bearing assumption for the survey's taxonomy or performance conclusions. There is no imported uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known result as a new contribution. The paper does contain reporting inconsistencies—the Introduction claims 48 models while Table 1 lists only 24 rows, and the 'Self-MoE' citation in the Introduction points to an unrelated Emollm paper—but these affect scope and verifiability, not circularity. The survey is self-contained against external benchmarks in the sense that its evidence base is the cited literature, not its own construction.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The survey contributes no parameters, entities, or formal derivations. Its load-bearing assumptions are about sample representativeness and the reliability of its secondary citations.

axioms (3)
  • domain assumption The 24 listed models are representative of the specialized-LLM landscape
    The survey infers trends from a non-exhaustive, non-random sample selected without stated criteria.
  • domain assumption Performance numbers quoted from cited papers are accurate and comparable
    The survey forwards specific metrics (e.g., 'activation memory cut by 7.3x', 'lifts GSM8K by 14.7%') without re-evaluation, and several citations are mismatched.
  • domain assumption The chronological ordering of models reflects a genuine architectural progression
    The claimed 'evolution' from fine-tuning to native architectures is inferred from release dates of independently developed systems rather than from a controlled comparison.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Survey of Specialized Large Language Model." pith.science (2026). https://pith.science/paper/R37M6P2D

@misc{pith2026250819667,
  author       = {Pith},
  title        = {Pith review of: Survey of Specialized Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R37M6P2D}},
  note         = {Machine review of arXiv:2508.19667}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The rapid evolution of specialized large language models (LLMs) has transitioned from simple domain adaptation to sophisticated native architectures, marking a paradigm shift in AI development. This survey systematically examines this progression across healthcare, finance, legal, and technical domains. Besides the wide use of specialized LLMs, technical breakthrough such as the emergence of domain-native designs beyond fine-tuning, growing emphasis on parameter efficiency through sparse computation and quantization, increasing integration of multimodal capabilities and so on are applied to recent LLM agent. Our analysis reveals how these innovations address fundamental limitations of general-purpose LLMs in professional applications, with specialized models consistently performance gains on domain-specific benchmarks. The survey further highlights the implications for E-Commerce field to fill gaps in the field.

Figures

Figures reproduced from arXiv: 2508.19667 by Chenghan Yang, Ling Jiang, Ruiyu Zhao, Yang Liu.

Figure 1
Figure 1. Figure 1: Evolution of representative specialized LLMs (2022–2025). The timeline highlights the transition from [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training

    cs.LG 2025-11 unverdicted novelty 7.0

    Q-RAG trains embedders via RL for multi-step retrieval and reports state-of-the-art results on BabiLong and RULER benchmarks for contexts up to 10M tokens.

Reference graph

Works this paper leans on

53 extracted references · 21 canonical work pages · cited by 1 Pith paper · 6 internal anchors

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Avinash Anand, Raj Jaiswal, Abhishek Dharmadhikari, Atharva Marathe, Harsh Popat, Harshil Mital, Ashwin R Nair, Kritarth Prasad, Sidharth Kumar, Astha Verma, and 1 others. 2024. Geovqa: A comprehensive multimodal geometry dataset for secondary education. In 2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR), p...

    work page 2024

  4. [4]

    Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024

    Pith/arXiv arXiv 2024

  5. [5]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

    Pith/arXiv arXiv 2022

  6. [6]

    Elliot Bolton, Abhinav Venigalla, Michihiro Yasunaga, David Hall, Betty Xiong, Tony Lee, Roxana Daneshjou, Jonathan Frankle, Percy Liang, Michael Carbin, and 1 others. 2024. Biomedlm: A 2.7 b parameter language model trained on biomedical text. arXiv preprint arXiv:2403.18421

    Pith/arXiv arXiv 2024

  7. [7]

    Jiaqi Cao, Jiarui Wang, Rubin Wei, Qipeng Guo, Kai Chen, Bowen Zhou, and Zhouhan Lin. 2025. Memory decoder: A pretrained, plug-and-play memory for large language models. arXiv preprint arXiv:2508.09874

    arXiv 2025

  8. [8]

    Krrish Chawla, Aryan Sahai, Mario DePavia, Sudharsan Sundar, Brando Miranda, Elyas Obbad, and Sanmi Koyejo. 2025. Quantifying the importance of data alignment in downstream model performance. arXiv preprint arXiv:2501.08496

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Junling Chen, Peilin Wang, Kai Ji, and 1 others. 2023. Huatuogpt-ii, one-stage training for medical adaption of llms. arXiv preprint arXiv:2311.09774

    Pith/arXiv arXiv 2023

  10. [10]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

    Pith/arXiv arXiv 2021

  11. [11]

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413

    Pith/arXiv arXiv 2025

  12. [12]

    Michael Han Daniel Han and Unsloth team. 2023. http://github.com/unslothai/unsloth Unsloth

    2023

  13. [13]

    Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. 2023. Spqr: A sparse-quantized representation for near-lossless llm weight compression. arXiv preprint arXiv:2306.03078

    Pith/arXiv arXiv 2023

  14. [14]

    Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. 2024. Mixture-of-loras: An efficient multitask tuning for large language models. arXiv preprint arXiv:2403.03432

    Pith/arXiv arXiv 2024

  15. [15]

    Ming Gong, Xucheng Huang, Chenghan Yang, Xianhan Peng, Haoxin Wang, Yang Liu, and Ling Jiang. 2025. Mindflow: Revolutionizing e-commerce customer support with multimodal llm agents. arXiv preprint arXiv:2507.05330

    Pith/arXiv arXiv 2025

  16. [16]

    Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. 2024. mplug-docowl2: High-resolution compressing for ocr-free multi-page document understanding. arXiv preprint arXiv:2409.03420

    Pith/arXiv arXiv 2024

  17. [17]

    Quzhe Huang, Mingxu Tao, Chen Zhang, Zhenwei An, Cong Jiang, Zhibin Chen, Zirui Wu, and Yansong Feng. 2023. Lawyer llama technical report. arXiv preprint arXiv:2305.15062

    Pith/arXiv arXiv 2023

  18. [18]

    Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh-Thang Luong, and Orhan Firat. 2021. Beyond distillation: Task-level mixture-of-experts for efficient inference. arXiv preprint arXiv:2110.03742

    Pith/arXiv arXiv 2021

  19. [19]

    Mengtian Li, Jinshu Chen, Wanquan Feng, Bingchuan Li, Fei Dai, Songtao Zhao, and Qian He. 2025. Hyperlora: Parameter-efficient adaptive generation for portrait synthesis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 13114--13123

    work page 2025

  20. [20]

    Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Richard James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, and 1 others. 2023. Ra-dit: Retrieval-augmented dual instruction tuning. In The Twelfth International Conference on Learning Representations

    work page 2023

  21. [21]

    Mianxin Liu, Weiguo Hu, Jinru Ding, Jie Xu, Xiaoyang Li, Lifeng Zhu, Zhian Bai, Xiaoming Shi, Benyou Wang, Haitao Song, and 1 others. 2024. Medbench: A comprehensive, standardized, and reliable benchmarking system for evaluating chinese medical large language models. Big Data Mining and Analytics, 7(4):1116--1128

    work page 2024

  22. [22]

    Renqian Luo, Liai Sun, Yingce Xia, and 1 others. 2022. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6):bbac409

    work page 2022

  23. [23]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. Codegen: An open large language model for code with multi-turn program synthesis

    work page 2023

  24. [24]

    Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2024. Gorilla: Large language model connected with massive apis. Advances in Neural Information Processing Systems, 37:126544--126565

    work page 2024

  25. [25]

    Zishang Qiu, Xinan Chen, Long Chen, and Ruibin Bai. 2025. Mela: A metacognitive llm-driven architecture for automatic heuristic design. arXiv preprint arXiv:2507.20541

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  26. [26]

    Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. 2022. Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale. In International conference on machine learning, pages 18332--18346. PMLR

    work page 2022

  27. [27]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316--1331

    2023

  28. [28]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539--68551

    2023

  29. [29]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, and 1 others. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172--180

    work page 2023

  30. [30]

    Yi Su, Yuechi Zhou, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, and Min Zhang. 2025. Accurate kv cache quantization with outlier tokens tracing. arXiv preprint arXiv:2505.10938

    Pith/arXiv arXiv 2025

  31. [31]

    Arpita Vats, Rahul Raja, Vinija Jain, and Aman Chadha. 2024. The evolution of mixture of experts: A survey from basics to breakthroughs

    work page 2024

  32. [32]

    Haoxin Wang, Xianhan Peng, Xucheng Huang, Yizhe Huang, Ming Gong, Chenghan Yang, Yang Liu, and Ling Jiang. 2025. Ecom-bench: Can llm agent resolve real-world e-commerce customer support issues? arXiv preprint arXiv:2507.05639

    arXiv 2025

  33. [33]

    Ke Wang, Huan Ren, Aojun Zhou, and 1 others. 2023. Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning. arXiv preprint arXiv:2310.03731

    Pith/arXiv arXiv 2023

  34. [34]

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560

    Pith/arXiv arXiv 2022

  35. [35]

    Jason Weston and Sainbayar Sukhbaatar. 2023. System 2 attention (is something you might need too). arXiv preprint arXiv:2311.11829

    Pith/arXiv arXiv 2023

  36. [36]

    Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yanfeng Wang. 2024. Pmc-llama: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, 31(9):1833--1843

    2024

  37. [37]

    Shijie Wu, Ozan Irsoy, Steven Lu, and 1 others. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564

    Pith/arXiv arXiv 2023

  38. [38]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024 a . Wizardlm: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations

    work page 2024

  39. [39]

    Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang. 2023. Protst: Multi-modality learning of protein sequences and biomedical texts. In International Conference on Machine Learning, pages 38749--38767. PMLR

    work page 2023

  40. [40]

    Tianyang Xu, Zhiqiang Hu, Lei Chen, and 1 others. 2024 b . Sa-mdkif: A scalable and adaptable medical domain knowledge injection framework for large language models. arXiv preprint arXiv:2402.00474

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    Qian Yang, Mao Ye, Bo Du, and 1 others. 2024. Emollm: Multimodal emotional understanding meets large language models. arXiv preprint arXiv:2406.16442

    Pith/arXiv arXiv 2024

  42. [42]

    Yi Yang, Yixuan Tang, and Kar Yan Tam. 2023 a . Investlm: A large language model for investment using financial domain instruction tuning. arXiv preprint arXiv:2309.13064

    Pith/arXiv arXiv 2023

  43. [43]

    Zhiyuan Yang, Xin Liu, Lin Li, and 1 others. 2023 b . Fingpt: Open-source financial large language models. arXiv preprint arXiv:2306.06031

    arXiv 2023

  44. [44]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809--11822

    2023

  45. [45]

    Zhouliang Yu, Ruotian Peng, Keyi Ding, Yizhe Li, Zhongyuan Peng, Minghao Liu, Yifan Zhang, Zheng Yuan, Huajian Xin, Wenhao Huang, and 1 others. 2025. Formalmath: Benchmarking formal mathematical reasoning of large language models. arXiv preprint arXiv:2505.02735

    Pith/arXiv arXiv 2025

  46. [46]

    Bithiah Yuan. 2025. Finbert-qa: Financial question answering with pre-trained bert language models. arXiv preprint arXiv:2505.00725

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  47. [47]

    Li Yunxiang, Li Zihan, Zhang Kai, and 1 others. 2023. Chatdoctor: A medical chat model fine-tuned on llama model using medical domain knowledge. arXiv preprint arXiv:2303.14070

    Pith/arXiv arXiv 2023

  48. [48]

    Aohan Zeng, Xiao Liu, Zhengxiao Du, and 1 others. 2025. Glm-4.5: An early multimodal foundation model for agents. arXiv preprint arXiv:2508.06471

    Pith/arXiv arXiv 2025

  49. [49]

    Jiawei Zhang, Xiaohan Wang, Zhiqiang Li, and 1 others. 2024 a . Wts: Way-to-specialist--multi-stage expert tuning for medical diagnosis. arXiv preprint arXiv:2403.09876

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  50. [50]

    Ningyu Zhang, Yue Chen, Shumin Xiang, and 1 others. 2024 b . Lawllm: Enhancing large language models for legal services in china. arXiv preprint arXiv:2402.17258

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  51. [51]

    Shuai Zhang, Jian Li, Xiaoqing Wang, and 1 others. 2024 c . Medinstruct-200k: A large-scale, verified clinical instruction dataset. Nature Machine Intelligence

    work page 2024

  52. [52]

    Xiaodong Zhang, Hao Li, Jing Wang, and 1 others. 2025. Evaluating large language models in dental implantology: A multi-dimensional expert assessment. Journal of Dental Research, 104(2):123--135

    work page 2025

  53. [53]

    Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M Dai, Quoc V Le, James Laudon, and 1 others. 2022. Mixture-of-experts with expert choice routing. Advances in Neural Information Processing Systems, 35:7103--7114

    work page 2022

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.