Pith. sign in

REVIEW 4 major objections 7 minor 41 references

Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that quantizing a reliable large language model yields a small model that is more trustworthy than one trained from scratch, and that quantization preserves trustworthiness better than pruning.

desk verdict Useful empirical benchmark, but the headline 'compressed beats pre-trained' claim needs matched-efficiency measurements before it becomes a deployment rule. read the letter →

arxiv 2608.11981 v1 pith:2DTI2ZRE submitted 2026-08-12 cs.CL

Haokun Lin , Kaijie Zhu , Haobo Xu , Yichen Wu , Zhichao Lu , Qingfu Zhang , Zhenan Sun This is my paper · ORCID
classification cs.CL
keywords smalllanguagemodelstrustworthinessmodelcompressionquantizationpruningknowledgedistillationfairnessrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to answer how to build a small language model (SLM) that is trustworthy as well as efficient. It argues that the standard path of training compact models from scratch produces models with weak trustworthiness scores, while quantizing a larger, already-aligned model keeps most of that model's reliability while shrinking memory use by about four times. It also argues that pruning, especially semi-structured 2:4 pruning, degrades trustworthiness, making quantization the safer compression route. If the paper is right, practitioners can treat model size and trustworthiness as separable: pick a strong large model, compress it with 4-bit quantization, and optionally distill from a trustworthy teacher to improve reliability further.

What carries the argument

The load-bearing object is the direct comparison between pre-trained SLMs and quantized larger models, with the 4-bit GPTQ quantization of Qwen2.5-1.5B as the decisive case. The quantization formula $\hat{x} = Q_b(x) = s \cdot \Pi_{\Omega(b)}(x/s)$ projects weights onto a low-bit integer set with error compensation, and that low-loss projection is what allows a compressed model to keep the ethical, privacy, robustness, and fairness behaviors of its full-precision teacher. The comparison is measured by TrustLLM's four averaged subtask groups, which supply the single overall score that drives all three headline conclusions.

What would settle it

Run the same four trustworthiness dimensions on an independent battery of safety, fairness, privacy, and robustness tasks not drawn from TrustLLM, and check whether Qwen2.5-1.5B-INT4-GPTQ still beats all four sub-1B pre-trained SLMs; a reversal on such held-out tasks would show the paper's conclusion is benchmark-specific.

Watch

Extended reading notes

Core claim

The paper's central claim is that compression can be a better route to trustworthy SLMs than training small models from scratch. On the TrustLLM benchmark, the four pre-trained SLMs under 1 billion parameters score around 45.5 to 51.9 percent overall, whereas Qwen2.5-1.5B quantized to 4 bits with GPTQ scores about 63.9 percent, and larger quantized models lose less than 2 percent trustworthiness compared with their full-precision versions. Pruning, by contrast, consistently damages trustworthiness, with semi-structured sparsity causing larger drops than unstructured pruning. The paper further claims that knowledge distillation from a more trustworthy teacher (Qwen2.5-7B to Qwen2.5-3B) raises the student's score on all four dimensions, making it a complementary improvement on top of compression.

Load-bearing premise

The whole recommendation rests on TrustLLM's four averaged subtask groups being a valid and sufficient measure of real-world trustworthiness; if that benchmark misses important failure modes, the advice to quantize rather than prune or train from scratch may not carry over to deployment.

Editorial extensions

If this is right

  • If the central claim holds, deployment teams should prefer a 4-bit quantized larger model over a sub-1B model trained from scratch when both fit the same memory budget, because the quantized model preserves the larger model's trustworthiness.
  • Pruning methods, particularly semi-structured 2:4 pruning, should be avoided when trustworthiness matters, since they consistently lower ethics, robustness, and fairness scores in these experiments.
  • Quantization bit-width can be used as a trustworthiness-preserving efficiency dial: the paper finds GPTQ at 3, 4, and 8 bits on Qwen2.5 models keeps overall trustworthiness nearly flat, so lower bits can be chosen for speed without expecting a reliability cliff.
  • Knowledge distillation from a stronger, more trustworthy teacher can serve as an additional step to raise a small model's trustworthiness across all four measured dimensions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's memory and latency discussion implies a testable extension: measure trustworthiness per unit of deployment cost across the full range of bit-widths and model sizes, rather than comparing only at fixed parameter counts.
  • Because every headline result is a TrustLLM score, the ranking may shift under a different trustworthiness battery, such as jailbreak resistance, stereotype transfer, or data-memorization probes; re-running the same comparison on those tasks would show how general the 'quantize, don't train small' rule is.
  • The distillation evidence is limited to one teacher-student pair (7B to 3B); whether even stronger teachers can transfer trustworthiness to sub-1B students, and whether distillation plus quantization compounds, are natural next tests the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript reports an empirical evaluation of the trustworthiness of small language models (SLMs), built either by pre-training compact models or by compressing larger models. Using the TrustLLM benchmark restricted to four dimensions (ethics, privacy, robustness, fairness), it compares pruning (SparseGPT, Wanda) and quantization (GPTQ, AWQ) across several open model families, and then compares four sub-1B pre-trained SLMs with a quantized Qwen2.5-1.5B model. The main claims are that quantization preserves trustworthiness better than pruning, that compressing a larger aligned model via quantization yields SLMs with better trustworthiness than pre-trained small models, and that knowledge distillation from a trustworthy teacher further improves SLM trustworthiness. The evidence consists of average accuracy tables (Tables I-V).

Significance. If substantiated, the central finding would give practitioners a concrete recipe: quantize a large aligned model rather than train a small model from scratch to obtain a trustworthy small model. The paper covers multiple compression methods and model families, and it addresses an underexplored question. The strength of the design is the breadth of the comparison, and the paper directly engages with a practically important trade-off. However, the headline comparisons mix model scale with compression efficiency, omit measured deployment costs, use a subset of TrustLLM dimensions, and report single-point scores without uncertainty estimates, leaving the main practical recommendation not yet fully established.

major comments (4)
  1. [IV.D, Table IV] Section IV.D, Table IV: The central claim that compressing a reliable large model via quantization (Qwen2.5-1.5B-INT4-GPTQ, overall 63.86) yields SLMs with superior trustworthiness to pre-trained sub-1B models (45.46-51.92) is not established on a matched-efficiency basis. The 1.5B model has roughly three times the parameters of the 0.5B baselines, and the only bridge to 'small model' status is the unmeasured Memory and Latency Discussion, which asserts that INT4 reduces weight-memory traffic by ~4x and that latency is typically within ~1.2-1.5x of a 0.5B FP16 model. No runtime, memory, throughput, or energy measurements are reported, so the advantage may simply reflect the larger base model rather than the compression recipe. The same paragraph also claims 'adaptability' via bit-width flexibility without operationalizing or evaluating it. Please add direct efficiency measurements or explicitly rescope the conclusion to 'higher TrustLLM score per parameter budget.'
  2. [IV.B and IV.C, Tables I-II] Section IV.B and Section IV.C, Tables I-II: The recommendation that quantization is more effective than pruning at preserving trustworthiness rests on comparisons across different base models and scales. Pruning is evaluated on Gemma-1.1-7B, Llama-3.1-8B, and Qwen2.5-7B, while quantization is evaluated on Llama-3.2-1B/3B and Qwen2.5-0.5B/1.5B/3B/7B. This confounds the compression method with model family and size (e.g., a 2:4-pruned Llama-3.1-8B scores 55.29 overall, while an INT4-GPTQ Qwen2.5-0.5B scores 52.93). A matched comparison, in which the same base models are subject to both pruning and quantization at comparable compression ratios, is needed before the 'quantization over pruning' advice can be supported.
  3. [IV.A, Trustworthiness Measurement] Section IV.A, Trustworthiness Measurement: The paper adopts TrustLLM but evaluates only four of its six dimensions (ethics, privacy, robustness, fairness); truthfulness and safety are omitted without justification. The abstract and conclusion use the unqualified term 'trustworthiness,' which is broader than what is measured. The recommendations may not transfer to the omitted dimensions, which are central to the trustworthiness construct in TrustLLM. Please either justify the reduced dimension set or explicitly limit the claims to the four measured dimensions.
  4. [Tables I-IV] Tables I-IV: All scores are reported as single percentages with no standard deviations, confidence intervals, or significance tests. Each cell averages an unspecified number of subtasks, so the two-decimal precision is not interpretable without variance estimates; several conclusions depend on small differences (e.g., Llama-3.2-1B AWQ 65.67 vs GPTQ 63.67 in Table II; Qwen2.5-3B INT3 54.78 vs INT8 54.71 in Table III), which may be noise. Please report per-subtask spreads or multiple runs, and check whether the ordinal claims survive a simple significance test.
minor comments (7)
  1. [IV.B, Table I] The statement that 2:4 sparsity is uniformly worse than 4:8 sparsity is contradicted by LLaMA-3.1-8B 2:4-Wanda fairness (72.04) exceeding 4:8-Wanda (46.19); qualify the claim with 'in most cases' or discuss the exceptions.
  2. [IV.D, first paragraph] The claim that quantized LLMs outperform pre-trained SLMs 'across all trustworthiness dimensions' is contradicted by Table IV, where SmolLM2-360M-Instruct has higher privacy (48.59) than Qwen2.5-1.5B-INT4-GPTQ (40.38); the text should say 'on average' or acknowledge the trade-off.
  3. [Tables II and IV] The label 'A WQ' appears in several table headers and should be 'AWQ'; also unify 'Sparsegpt' vs 'SparseGPT' in Table I.
  4. [IV.D] The abstract promises 'adaptability,' but Section IV.D only mentions bit-width flexibility without defining or evaluating adaptability; either define a measurable notion or remove the term from the claims.
  5. [IV.E] The distillation claim is based on a single source-target pair (Qwen2.5-7B distilled to Qwen2.5-3B on Alpaca); the conclusion that knowledge distillation can further enhance the reliability of SLMs is broader than this one configuration supports.
  6. [IV.A, Table IV] The set of pre-trained SLMs includes Qwen2.5-1.5B-Instruct alongside sub-1B models; clarify whether this model is a control for scale or part of the SLM comparison, since the text says 'under 1B parameters.'
  7. [Reference [38]] Reference [38] cites Bommasani's foundation models report, but the text uses it for the Alpaca dataset; verify the citation or add an Alpaca-specific reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark study whose conclusions are supported by external, independently published compression methods and an external trustworthiness benchmark.

full rationale

The paper's central claims are empirical comparisons, not derivations: (1) quantization preserves trustworthiness better than pruning, (2) quantized Qwen2.5-1.5B exceeds pre-trained sub-1B SLMs on TrustLLM, and (3) distillation from Qwen2.5-7B improves Qwen2.5-3B. None of these conclusions is obtained by defining a quantity in terms of the target result, fitting a parameter to the benchmark being predicted, or importing a load-bearing uniqueness theorem from the authors' own prior work. The compression methods evaluated are external and public (GPTQ, AWQ, SparseGPT, Wanda), and the trustworthiness measure is the independently published TrustLLM benchmark; no parameter in the evaluation is fitted to TrustLLM outcomes. The paper contains several self-citations, including the first author's DuQuant paper, but these citations are not used as evidence for the headline findings: DuQuant is mentioned only in related work and is not among the methods tested, and the other self-cited works are not invoked to justify the paper's conclusions. The distillation experiment is a straightforward empirical intervention with a reported before/after comparison on the same benchmark. The main weaknesses are scientific-correctness concerns rather than circularity: the pre-trained-versus-compressed comparison is not made on a matched-efficiency basis, since runtime and memory equivalence are asserted without a measurement table, and the validity of TrustLLM as a comprehensive trustworthiness measure is assumed rather than justified. These are limitations of evidence, not cases of the paper's predictions reducing by construction to its inputs. Under the hard rules requiring a quoted reduction or a fitted parameter renamed as a prediction, no circular step can be identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities are needed. The claims depend on the benchmark construct, the model sample, the compression implementations, and an assumed efficiency equivalence, each recorded as a domain assumption.

assumptions (4)
  • domain assumption TrustLLM scores are a valid operationalization of trustworthiness.
    The paper uses TrustLLM as the primary framework and states no justification or limitation; all findings are conditional on this construct.
  • domain assumption The four selected pre-trained models are representative of sub-1B SLMs.
    The paper draws general conclusions about pre-trained SLMs from h2o-danube3-500m, MobiLlama-500m, SmolLM2-360M, and Qwen2.5-0.5B, a small convenience sample.
  • domain assumption Standard pruning and quantization implementations are used with appropriate calibration.
    The paper states SparseGPT, Wanda, GPTQ, AWQ and WikiText-v2 calibration but does not give hyperparameters; reproduction assumes the authors ran them correctly.
  • domain assumption The 1.5B INT4 model is practically comparable in efficiency to sub-1B models.
    Section IV.D asserts this with a memory-bound argument, not with measurements; the pre-trained-versus-compressed comparison depends on this equivalence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed." pith.science (2026). https://pith.science/paper/2DTI2ZRE

@misc{pith2026260811981,
  author       = {Pith},
  title        = {Pith review of: Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DTI2ZRE}},
  note         = {Machine review of arXiv:2608.11981}
}
read the original abstract

Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Language Models (LLMs), offering promising potential in resource-constrained scenarios. Existing approaches to building SLMs typically follow two paths: training compact models from scratch, or compressing larger pre-trained models using methods such as pruning, quantization, or distillation. As language models become increasingly integrated into real-world applications, ensuring their trustworthiness has become a critical concern. However, how to build trustworthy SLMs remains an underexplored question. In this work, we present a comprehensive evaluation of SLM trustworthiness across multiple dimensions, including fairness, robustness, privacy, and ethics. We first examine the effects of pruning and quantization, and find that quantization is significantly more effective in preserving trustworthiness compared to pruning. More importantly, we demonstrate that compressing a reliable large model via quantization can produce SLMs with superior trustworthiness and adaptability compared to using small models trained from scratch. Furthermore, knowledge distillation from trustworthy teacher models can further enhance the reliability of SLMs. We hope our findings provide practical guidance and a foundation for future research into the development and deployment of trustworthy small language models.

Figures

Figures reproduced from arXiv: 2608.11981 by the authors.

Figure 1
Figure 1. Our evaluation framework for assessing the trustworthiness of SLMs, including state-of-the-art pruning and quantization [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 15 canonical work pages

  1. [1]

    Gemma: Open models based on gemini research and technology,

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Juliette Love, et al., “Gemma: Open models based on gemini research and technology,”arXiv preprint arXiv:2403.08295, 2024

    arXiv 2024

  2. [2]

    Qwen2.5: A party of foundation models,

    Qwen Team, “Qwen2.5: A party of foundation models,” September 2024

    2024

  3. [3]

    The llama 3 herd of models,

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  4. [4]

    Mobillama: Towards accurate and lightweight fully transparent gpt,

    Omkar Thawakar, Ashmal Vayani, Salman Khan, Hisham Cholakal, Rao M Anwer, Michael Felsberg, Tim Baldwin, Eric P Xing, and Fahad Shahbaz Khan, “Mobillama: Towards accurate and lightweight fully transparent gpt,”arXiv preprint arXiv:2402.16840, 2024

    arXiv 2024

  5. [5]

    Smollm2: When smol goes big–data-centric training of a small language model,

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, et al., “Smollm2: When smol goes big–data-centric training of a small language model,” arXiv preprint arXiv:2502.02737, 2025

    arXiv 2025

  6. [6]

    Gptq: Accurate post-training quantization for generative pre-trained transformers,

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh, “Gptq: Accurate post-training quantization for generative pre-trained transformers,”arXiv preprint arXiv:2210.17323, 2022

    arXiv 2022

  7. [7]

    Sparsegpt: Massive language models can be accurately pruned in one-shot,

    Elias Frantar and Dan Alistarh, “Sparsegpt: Massive language models can be accurately pruned in one-shot,” inInternational conference on machine learning. PMLR, 2023, pp. 10323–10337

    work page 2023

  8. [8]

    Awq: Activation-aware weight quantization for llm compression and acceleration,

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han, “Awq: Activation-aware weight quantization for llm compression and acceleration,”arXiv preprint arXiv:2306.00978, 2023

    arXiv 2023

Show all 41 references
  1. [9]

    Duquant: Distributing outliers via dual transformation makes stronger quantized llms,

    Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei, “Duquant: Distributing outliers via dual transformation makes stronger quantized llms,”Advances in Neural Information Processing Systems, vol. 37, pp. 87766–87800, 2024

    2024

  2. [10]

    Efficient diffusion language models: A comprehensive survey,

    Haokun Lin, Xinle Jia, Shaozhen Liu, Shujun Xia, Weitao Huang, Haobo Xu, Junyang Li, Yicheng Xiao, Xingrun Xing, Ziyu Guo, et al., “Efficient diffusion language models: A comprehensive survey,”Authorea Preprints, 2026

    2026

  3. [11]

    A survey on evaluation of large language models,

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al., “A survey on evaluation of large language models,”ACM transactions on intelligent systems and technology, vol. 15, no. 3, pp. 1–45, 2024

    2024

  4. [12]

    Trustllm: Trustworthiness in large language models,

    Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al., “Trustllm: Trustworthiness in large language models,”arXiv preprint arXiv:2401.05561, 2024

    2024 arXiv

  5. [13]

    Promptbench: Towards evaluating the robustness of large language models on adversarial prompts,

    Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Wei Ye, Yue Zhang, Neil Zhenqiang Gong, et al., “Promptbench: Towards evaluating the robustness of large language models on adversarial prompts,”arXiv e-prints, pp. arXiv–2306, 2023

    2023

  6. [14]

    Security and privacy challenges of large language models: A survey,

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu, “Security and privacy challenges of large language models: A survey,”ACM Computing Surveys, vol. 57, no. 6, pp. 1–39, 2025

    2025

  7. [15]

    Exploiting llm quantization,

    Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, and Martin Vechev, “Exploiting llm quantization,”Advances in Neural Information Processing Systems, vol. 37, pp. 41709–41732, 2024

    2024

  8. [16]

    Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression,

    Junyuan Hong, Jinhao Duan, Chenhui Zhang, Zhangheng Li, Chulin Xie, Kelsey Lieberman, James Diffenderfer, Brian Bartoldson, Ajay Jaiswal, Kaidi Xu, et al., “Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression,”arXiv preprint arXiv:24...

    2024 arXiv

  9. [17]

    Assessing safety risks and quantization-aware safety patching for quantized large language models,

    Kejia Chen, Jiawen Zhang, Jiacong Hu, Yu Wang, Jian Lou, Zunlei Feng, and Mingli Song, “Assessing safety risks and quantization-aware safety patching for quantized large language models,” inForty-second International Conference on Machine Learning

  10. [18]

    Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,

    Song Han, Huizi Mao, and William J Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,”arXiv preprint arXiv:1510.00149, 2015

    2015 arXiv

  11. [19]

    Efficientllm: Scalable pruning-aware pretraining for architecture-agnostic edge lan- guage models,

    Xingrun Xing, Zheng Liu, Shitao Xiao, Boyan Gao, Yiming Liang, Wanpeng Zhang, Haokun Lin, Guoqi Li, and Jiajun Zhang, “Efficientllm: Scalable pruning-aware pretraining for architecture-agnostic edge lan- guage models,”arXiv preprint arXiv:2502.06663, 2025

    2025 arXiv

  12. [20]

    Prune as you generate: Online rollout pruning for faster and better rlvr,

    Haobo Xu, Sirui Chen, Ruizhong Qiu, Yuchen Yan, Chen Luo, Monica Cheng, Jingrui He, and Hanghang Tong, “Prune as you generate: Online rollout pruning for faster and better rlvr,”arXiv preprint arXiv:2603.24840, 2026

    2026

  13. [21]

    A simple and effective pruning approach for large language models,

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter, “A simple and effective pruning approach for large language models,”arXiv preprint arXiv:2306.11695, 2023

    2023 arXiv

  14. [22]

    Llm-pruner: On the structural pruning of large language models,

    Xinyin Ma, Gongfan Fang, and Xinchao Wang, “Llm-pruner: On the structural pruning of large language models,”Advances in neural information processing systems, vol. 36, pp. 21702–21720, 2023

    2023

  15. [23]

    Dopq-vit: Towards distribution-friendly and outlier- aware post-training quantization for vision transformers,

    Lianwei Yang, Haisong Gong, Haokun Lin, Yichen Wu, Zhenan Sun, and Qingyi Gu, “Dopq-vit: Towards distribution-friendly and outlier- aware post-training quantization for vision transformers,”arXiv preprint arXiv:2408.03291, 2024

    2024 arXiv

  16. [24]

    Quantization meets dllms: A systematic study of post-training quantization for diffusion llms,

    Haokun Lin, Haobo Xu, Yichen Wu, Ziyu Guo, Renrui Zhang, Zhichao Lu, Ying Wei, Qingfu Zhang, and Zhenan Sun, “Quantization meets dllms: A systematic study of post-training quantization for diffusion llms,”arXiv preprint arXiv:2508.14896, 2025

    2025

  17. [25]

    Lrq-dit: Log-rotation post-training quantization of diffusion transformers for image and video generation,

    Lianwei Yang, Haokun Lin, Tianchen Zhao, Yichen Wu, Hongyu Zhu, Ruiqi Xie, Zhenan Sun, Yu Wang, and Qingyi Gu, “Lrq-dit: Log-rotation post-training quantization of diffusion transformers for image and video generation,”arXiv preprint arXiv:2508.03485, 2025

    2025

  18. [26]

    Quantvla: Scale-calibrated post-training quantization for vision-language-action models,

    Jingxuan Zhang, Yunta Hsieh, Zhongwei Wang, Haokun Lin, Xin Wang, Ziqi Wang, Yingtie Lei, and Mi Zhang, “Quantvla: Scale-calibrated post-training quantization for vision-language-action models,”arXiv preprint arXiv:2602.20309, 2026

    2026 arXiv

  19. [27]

    Duquant++: Fine-grained rotation enhances microscaling fp4 quantization,

    Haokun Lin, Xinle Jia, Haobo Xu, Bingchen Yao, Xianglong Guo, Yichen Wu, Zhichao Lu, Ying Wei, Qingfu Zhang, and Zhenan Sun, “Duquant++: Fine-grained rotation enhances microscaling fp4 quantization,”arXiv preprint arXiv:2604.17789, 2026

    2026 arXiv

  20. [28]

    Dapq-dit: Distribution-aware post-training quantization for efficient generative tasks in diffusion transformers,

    Lianwei Yang, Haokun Lin, Yichen Wu, Zhenan Sun, and Qingyi Gu, “Dapq-dit: Distribution-aware post-training quantization for efficient generative tasks in diffusion transformers,” inProceedings of the 2026 International Conference on Multimedia Retrieval, 2026, pp. 2371–2380

    2026

  21. [29]

    Reshape and rotate: Adaptive weight reshaping and fine-grained rotation for ultra-low-bit diffusion transformers quantization,

    Lianwei Yang, Haokun Lin, Yichen Wu, Caifeng Shan, Zhenan Sun, and Qingyi Gu, “Reshape and rotate: Adaptive weight reshaping and fine-grained rotation for ultra-low-bit diffusion transformers quantization,” Neurocomputing, p. 133830, 2026

    2026

  22. [30]

    Distilling the knowledge in a neural network,

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  23. [31]

    Slog: An inductive spectral graph neural network beyond polynomial filter,

    Haobo Xu, Yuchen Yan, Dingsu Wang, Zhe Xu, Zhichen Zeng, Tarek F Abdelzaher, Jiawei Han, and Hanghang Tong, “Slog: An inductive spectral graph neural network beyond polynomial filter,” inForty-first International Conference on Machine Learning, 2024

    2024

  24. [32]

    Image-level memorization detection via inversion-based inference perturbation,

    Yue Jiang, Haokun Lin, Yang Bai, Bo Peng, Zhili Liu, Yueming Lyu, Yong Yang, and Jing Dong, “Image-level memorization detection via inversion-based inference perturbation,” inInternational Conference on Learning Representations, 2025, vol. 2025, pp. 47960–47979

    2025

  25. [33]

    Medrek: Retrieval-based editing for medical llms with key-aware prompts,

    Shujun Xia, Haokun Lin, Yichen Wu, Yinan Zhou, Zixuan Li, Zhongwei Wan, Xingrun Xing, Yefeng Zheng, Xiang Li, Caifeng Shan, et al., “Medrek: Retrieval-based editing for medical llms with key-aware prompts,”arXiv preprint arXiv:2510.13500, 2025

  26. [34]

    Iv-cot: Implicit visual chain-of-thought for structure-aware text-to-image generation,

    Zixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, et al., “Iv-cot: Implicit visual chain-of-thought for structure-aware text-to-image generation,” arXiv preprint arXiv:2606.24849, 2026

    2026 arXiv

  27. [35]

    Mac-splat: Multi-attribute consis- tency for high-fidelity sparse-view reconstruction,

    Jinqian Yang, Yichen Wu, Wanhua Li, Haokun Lin, Renzhen Wang, Xiangchu Feng, and Xixi Jia, “Mac-splat: Multi-attribute consis- tency for high-fidelity sparse-view reconstruction,”arXiv preprint arXiv:2607.10792, 2026

    2026 arXiv

  28. [36]

    A survey on knowledge distillation of large language models,

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Can Xu, Dacheng Tao, and Tianyi Zhou, “A survey on knowledge distillation of large language models,”arXiv preprint arXiv:2402.13116, 2024

    2024 arXiv

  29. [37]

    H2o-danube3 technical report,

    Pascal Pfeiffer, Philipp Singer, Yauhen Babakhin, Gabor Fodor, Nischay Dhankhar, and Sri Satish Ambati, “H2o-danube3 technical report,”arXiv preprint arXiv:2407.09276, 2024

    2024 arXiv

  30. [38]

    On the opportunities and risks of foundation models,

    Rishi Bommasani, “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  31. [39]

    Aligning ai with shared human values,

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt, “Aligning ai with shared human values,”arXiv preprint arXiv:2008.02275, 2020

    2008 arXiv

  32. [40]

    Social chemistry 101: Learning to reason about social and moral norms,

    Maxwell Forbes, Jena D Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi, “Social chemistry 101: Learning to reason about social and moral norms,”arXiv preprint arXiv:2011.00620, 2020

    2011 arXiv

  33. [41]

    Adversarial glue: A multi- task benchmark for robustness evaluation of language models,

    Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li, “Adversarial glue: A multi- task benchmark for robustness evaluation of language models,”arXiv preprint arXiv:2111.02840, 2021

    2021 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.