Pith. sign in

REVIEW 4 major objections 6 minor 96 references

AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AmoebaLLM claims that one fine-tuned LLM can instantly produce subnets of any shape and still sit on the accuracy-efficiency frontier.

desk verdict A well-built one-for-all fine-tuning framework whose headline MMLU results are partly selected on the test metric; the commonsense and ablation evidence suggest the core mechanism is sound, but the SOTA claims need stronger experimental hygiene. read the letter →

arxiv 2411.10606 v1 pith:DWE6AKGT submitted 2024-11-15 cs.LG cs.AI

Yonggan Fu , Zhongzhi Yu , Junwei Li , Jiayi Qian , Yongan Zhang , Xiangchi Yuan , Dachuan Shi , Roman Yakunin
show 1 more author
Yingyan Celine Lin
This is my paper · ORCID
classification cs.LGcs.AI
keywords largelanguagemodelcompressionone-for-alltrainingdynamicprogramminglayerpruningstructuredmixtureofLoRAsin-placedistillationany-shapesubnetefficientdeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes AmoebaLLM, a way to turn a pre-trained large language model into a collection of smaller models (subnets) of many different depths and widths after a single fine-tuning pass. The stated goal is that a developer can pick any subnet shape matching a target device or latency budget and extract it immediately, without any per-configuration retraining. The authors argue that three components make this work: a dynamic-programming layer-selection strategy plus an importance-based neuron selection that preserves pre-trained knowledge; a shape-aware mixture of LoRAs that reduces gradient conflicts during joint fine-tuning; and a loss-magnitude-balanced distillation objective. If correct, the approach would replace per-platform compression pipelines with one model that serves many deployment points. The paper reports that on LLaMA2 7B and Vicuna 7B, the extracted subnets match or beat separately fine-tuned compression baselines at 80%, 65%, and 50% of the original size.

What carries the argument

Three mechanisms carry the argument. The first is a dynamic program over decoder layers, with table $D[n][m]$ storing the best calibration metric when exactly $m$ layers are removed from the first $n$ layers; it assumes that, thanks to residual connections, layer-selection sub-problems are approximately independent, so the optimal solution for removing $m$ layers extends the best solution for $m-1$ layers. The second is the shape-aware mixture of LoRAs (SMoL), a gating function that takes the subnet's shape mask as input and sparsely activates and averages a small set of low-rank adapters, so that subnets sharing similar shapes share similar gradient updates and the adapters can be merged away at deployment. The third is the fine-tuning objective, which samples subnets by sandwich sampling and distills each sampled subnet from the largest subnet while normalizing loss magnitudes so no subnet dominates the update.

What would settle it

Take a small transformer (or a smaller random seed of LLaMA), run the DP layer selection, and compare its chosen layer sets against exhaustive search over all subsets for each removal count on the calibration metric; if the DP-selected sets are meaningfully worse than the exhaustive optimum, or if the gains of jointly removed layers diverge from the sum of individual gains, the independence assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that a given LLM can be endowed, through one one-time fine-tuning, with the ability to produce subnets of arbitrary depth and width that lie on the accuracy-efficiency frontier, meaning no separately tuned compressed model is needed for each target configuration. The discovery is framed as a complete workflow: first, a calibration-guided dynamic program selects which layers to keep for every depth remaining ratio, and an importance metric selects which neurons to keep for every width ratio; second, a shape-aware mixture of LoRAs adapts the model while preventing gradient conflicts among subnets; third, an in-place distillation loss with a loss-magnitude balancing term keeps the largest subnet from being dragged down by the higher losses of smaller subnets. The paper's evidence is that instantly extracted subnets outperform or match strong width- and layer-pruning baselines on MMLU and seven commonsense reasoning tasks, and that the per-subnet fine-tuned upper bound is close.

Load-bearing premise

The whole approach rests on the key hypothesis in Section 3.2 that, because transformer layers are residual, choosing which layers to remove can be treated as a set of approximately independent sub-problems; if layer contributions interact strongly, the dynamic program's choices are not near-optimal and every subnet derived from them inherits the error.

Editorial extensions

If this is right

  • A single fine-tuned AmoebaLLM model can serve many deployment targets: after the one-time fine-tuning, any depth/width combination within the design space can be extracted without additional training, so device-specific or battery-dependent latency budgets can be met on the fly.
  • Because SMoL's LoRAs are selected by shape rather than by input, they can be merged into the model weights at deployment, so the any-shape capability adds no per-input routing overhead.
  • The DP depth-shrinking component on its own is claimed to be a new state-of-the-art layer-pruning method, improving on single-layer importance and fixed heuristic pruning rules, which means it can be used as a standalone compression tool.
  • If factual knowledge is hard to restore after compression, as the calibration-dataset ablations suggest, future compression pipelines should choose calibration metrics and datasets that track the knowledge they care about rather than perplexity alone.
  • AmoebaLLM† (per-subnet fine-tuning of the selected subnets) forms an upper bound, so the one-for-all subnets are reported to land close to separately tuned models, quantifying the cost of instant extraction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test the DP independence hypothesis directly, but it could be probed by comparing DP-selected layer sets against exhaustive search on a small transformer, or by checking whether the improvement from removing two layers together equals the sum of the individual improvements.
  • Because the framework's selection and adapter mechanisms are not tied to LLaMA architecture specifics, the same three-stage recipe could plausibly transfer to other residual transformer LLMs or to multimodal models that use transformer stacks, provided calibration data and LoRA-style adapters are available.
  • The loss-magnitude balancing insight likely extends beyond AmoebaLLM: any joint training of models at very different capacities (for example, any-precision or any-length variants) faces the same domination problem, so normalizing loss scales before gradient averaging could improve other one-for-all training schemes.
  • Combining the shape-conditional gating idea with other parameter-efficient methods such as quantization or pruning-aware training could be a natural next step, since SMoL only touches the adapter pathway and leaves the base weight structure fixed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes AmoebaLLM, a framework to fine-tune a given LLM once so that subnets of arbitrary depth/width combinations can be extracted instantly at deployment time. The framework has three components: (1) a subnet selection strategy using DP-based depth shrinking and importance-based width shrinking; (2) a shape-aware mixture of LoRAs (SMoL) that sparsely activates LoRA experts based on the subnet shape; and (3) an in-place distillation objective with loss-magnitude balancing. Experiments on LLaMA2 7B and Vicuna 7B v1.5 compare AmoebaLLM with LLM-Pruner, FLAP, and Shortened LLaMA at 80%/65%/50% remaining ratios, reporting MMLU, seven commonsense reasoning tasks, perplexity, and device latency. The paper claims the resulting subnets achieve state-of-the-art accuracy-efficiency trade-offs after one fine-tuning, and also that the per-subnet fine-tuned variant achieves new SOTA compression effectiveness.

Significance. If validated, the one-for-all paradigm with instant subnet extraction would be practically valuable: a single fine-tuned model could replace per-configuration compression pipelines. The paper makes several concrete, falsifiable design choices and releases code, which is a strength. The profiling across two devices and three deployment flows is a useful empirical contribution, and the SMoL design is a sensible approach to reducing gradient conflicts among subnets. However, the central empirical evidence is currently weakened by the overlap between calibration and evaluation on MMLU, and by the absence of error bars. The commonsense-reasoning averages provide some independent support, but the differences are small and their statistical reliability is not established. The paper therefore contains a promising framework, but its main claims need substantially strengthened evidence before they can be accepted as stated.

major comments (4)
  1. [Sec. 4.4 and Sec. 3.4, Tables 1-2] The headline MMLU results are compromised by selection-on-evaluation circularity. Section 4.4 states that 'we adopted MMLU as the calibration dataset throughout the previous experiments,' and Section 3.4 uses 40 MMLU samples to select the final (depth, width) subnets reported in Tables 1 and 2. The MMLU column in those tables is therefore not an independent measure of the delivered subnets: the DP layer-selection strategy and the reported subnet shapes were both chosen using MMLU, while the baselines were not given the same metric-specific selection. The large MMLU advantages (e.g., 42.40 vs 40.21 at 80% on LLaMA2 and 48.30 vs 43.50 at 80% on Vicuna) could largely reflect selection bias. This is load-bearing because the abstract and conclusion claim an 'accuracy-efficiency frontier' and 'knowledge preservation' primarily on the basis of MMLU. The authors should either calibrate on a disjoint metric or dataset and evaluate MMLU as a held-out task, or explicitly present the MMLU results as calibration-tuned and support the frontier claim with independent metrics.
  2. [Sec. 4.2, Tables 1-2; NeurIPS Checklist item 7] No error bars, confidence intervals, or repeated-seed results are reported, and the checklist explicitly answers 'No' to statistical significance. This matters because several of the claimed advantages over the strongest baselines are small on the commonsense average (e.g., 62.29 vs 60.98 at 80% on LLaMA2; 56.96 vs 56.12 at 65% on LLaMA2), and individual task differences are often within 1-2 points. Without variance estimates, the claims that AmoebaLLM subnets are 'better or comparable' and achieve a 'state-of-the-art trade-off' are not quantitatively supported. The authors should report multiple runs or bootstrap confidence intervals for the main tables, and state which reported differences are outside noise.
  3. [Sec. 3.2, Eq. (1)-(2)] The DP-based depth shrinking relies on the 'Key hypothesis' that the layer-selection problem can be decomposed into approximately independent subproblems and that the optimal substructure holds. This assumption is central to the correctness of the DP recurrence, but it is not validated. Table 3 compares the DP against two heuristic baselines, which shows that the DP improves on those baselines, but it does not test whether the DP's solution is near-optimal or whether layer interactions are weak. The independence assumption could be tested by comparing the DP-selected layer sets against exhaustive search on a smaller model or a reduced set of layers, or against a randomized/beam-search selection procedure. Without such a validation, the depth-shrinking component's contribution is not established as principled rather than heuristic.
  4. [Sec. 4.4, Table 5] Table 5 demonstrates that the choice of calibration metric has a very large effect on post-fine-tuning MMLU accuracy: at 20 layers, MMLU-calibrated subnets reach 41.0% MMLU while Wikitext2-calibrated subnets reach 23.6%, and BookCorpus-calibrated subnets reach 27.0%. This sensitivity reinforces the concern that the reported MMLU advantages are driven by calibrating to MMLU itself. The paper should address this directly by reporting at least one set of results where the calibration metric is not MMLU and MMLU is evaluated as an unseen task, to show that the method preserves factual knowledge independently of the calibration choice.
minor comments (6)
  1. [Sec. 4.1] The fine-tuning setting lists the number of Alpaca samples, LoRA rank, learning rate, and iteration count, but omits batch size, sequence length, and the number of GPUs used for the one-for-all fine-tuning. These details should be added for reproducibility.
  2. [Sec. 4.2, Tables 1-2] The column labeled 'Average' appears to be the average over the seven commonsense reasoning tasks, excluding MMLU. This should be stated explicitly in the table caption or the evaluation setup.
  3. [Eq. (3)] The loss-normalization terms use the norm of cross-entropy losses; the type of norm (L2 over tokens and vocabulary, or something else) and the numerical treatment of zero-norm cases should be clarified.
  4. [NeurIPS Checklist] The checklist's Limitations answer cites 'Sec. 5.5', but the Limitations section is Sec. 4.5 in the manuscript. The cross-reference should be corrected.
  5. [Sec. 3.2] The importance-score formula F_i^l is dense and the notation is not fully introduced before the equation; a short verbal description of each symbol (k, t, j, X-bar) in the main text would improve readability.
  6. [Fig. 3] The latency-accuracy plot would benefit from labeled markers for each method and from indicating the specific subnet shapes used, since the reader cannot otherwise verify which operating points correspond to the 80%/65%/50% ratios in Tables 1-2.

Circularity Check

1 steps flagged · score 6.0 of 10

MMLU results are selected-on-MMLU and then reported as independent evidence; the reported subnet shapes and layer choices are optimized for the same metric, so the MMLU advantage is partly a fitting artifact, while commonsense results provide partial independence.

  1. fitted input called prediction [Sec. 4.4 (calibration choice), Sec. 3.4 (final subnet search), Tables 1-2]
    "Specifically, we use accuracy on the training set of MMLU [49] and PPL on the training sets of Wikitext2 [50]/BookCorpus [51] as target metrics. ... As such, we adopted MMLU as the calibration dataset throughout the previous experiments. ... we first perform a coarse grid search across uniformly spaced depth and width settings based on a small calibration set (e.g., 40 samples from the MMLU dataset) to identify subnets that meet the efficiency constraint with maximized accuracy."

    The DP layer selection (Eqs. 1-2) and final grid search both use MMLU accuracy as their optimization signal, and the paper states that MMLU was the calibration dataset for all main experiments. The subnets in Tables 1-2 are outputs of this MMLU-based search, and MMLU is then reported as the headline factual-knowledge result, with the conclusion that AmoebaLLM subnets 'achieve higher MMLU accuracy compared to all baselines.' Even with the stated MMLU train/test split, the delivered shapes and layer choices were selected to maximize the same benchmark, so the reported MMLU column is the optimized objective rather than an independent measure; the MMLU advantage is partly a selection effect. The seven commonsense tasks were not used in this calibration loop and Fig.

full rationale

The framework's mechanism (DP layer selection, importance-based width shrinking, SMoL, and loss-magnitude-balanced distillation) is internally consistent and is not a renamed known result; there are no load-bearing self-citations or imported uniqueness theorems. The one concrete circular element is the MMLU calibration/evaluation overlap: both the layer-selection stage and the final subnet search are explicitly optimized against MMLU, and MMLU is then presented as the main evidence for knowledge preservation and SOTA compression effectiveness. This makes the headline MMLU numbers partly fitted inputs rather than independent measurements. The commonsense reasoning results and the accuracy-latency frontier in Fig. 3 are not calibrated on those tasks, so the central one-for-all claim retains independent support; however, no error bars are reported and the MMLU selection bias is not acknowledged. The unvalidated independence hypothesis for the DP is a correctness and fairness risk rather than a circularity. Overall score 6 reflects that one headline evaluation quantity reduces by construction to the calibration objective, while the rest of the derivation chain is self-contained.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several domain assumptions about how LLM knowledge is organized and recoverable, plus hand-chosen design-space parameters. The most consequential choice is calibrating the subnet selection on MMLU, which feeds directly into the headline MMLU evaluation.

free parameters (4)
  • Layer selection vector s from DP = Binary decisions per layer; e.g., 12 layers removed at 50% ratio
    Chosen by dynamic programming to maximize MMLU accuracy on MMLU training calibration data; directly determines subnet composition and affects all downstream results.
  • Width masks M_l and bias B_l = Per-neuron binary masks
    Computed from FLAP importance scores over calibration data; determines which neurons remain in each block.
  • Final subnet shapes at 80/65/50% ratios = (30, 0.875), (28, 0.75), (22, 0.75)
    Selected by grid search on 40 MMLU samples (Sec. 3.4); these exact subnets are then evaluated on MMLU in Tables 1-2.
  • Design space and SMoL hyperparameters = Depth 20-32, width ratios {1, 7/8, 3/4, 5/8, 1/2}, T=5 LoRAs, k=2, rank 64, K=4 subnets per iteration, LR 2e-4, 10K…
    Hand-chosen settings that bound the achievable accuracy-efficiency trade-off and the capacity of the adapter.
assumptions (4)
  • domain assumption LLM layer contributions are approximately independent due to residual connections, so layer selection decomposes into independent subproblems.
    Invoked in Sec. 3.2 'Key hypothesis' to justify DP optimal substructure; if false, the DP-derived strategies are suboptimal.
  • domain assumption FLAP neuron importance scores computed on calibration features identify neurons whose removal least harms task performance.
    Used in Sec. 3.2 for width shrinking; underlying assumption from prior work [8] adopted without re-validation.
  • domain assumption Fine-tuning on 50K Alpaca samples can restore language modeling and knowledge lost by pruning.
    Assumed throughout the fine-tuning objective; the paper's own Sec. 4.5 limitation says limited data may limit achievable trade-offs.
  • domain assumption Distilling from the largest subnet to smaller subnets improves all subnets, with balanced loss magnitudes.
    Used in Eq. (3); relies on sandwich sampling and in-place distillation from BigNAS [11].
invented entities (1)
  • SMoL (shape-aware mixture of LoRAs)
    purpose: Sparsely activate different LoRA experts depending on subnet shape to mitigate gradient conflicts during one-for-all fine-tuning.
    A new module introduced in Sec. 3.3; its efficacy is supported only by in-paper ablations (Table 4), not by external benchmarks or independent replications.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment." pith.science (2026). https://pith.science/paper/DWE6AKGT

@misc{pith2026241110606,
  author       = {Pith},
  title        = {Pith review of: AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWE6AKGT}},
  note         = {Machine review of arXiv:2411.10606}
}
read the original abstract

Motivated by the transformative capabilities of large language models (LLMs) across various natural language tasks, there has been a growing demand to deploy these models effectively across diverse real-world applications and platforms. However, the challenge of efficiently deploying LLMs has become increasingly pronounced due to the varying application-specific performance requirements and the rapid evolution of computational platforms, which feature diverse resource constraints and deployment flows. These varying requirements necessitate LLMs that can adapt their structures (depth and width) for optimal efficiency across different platforms and application specifications. To address this critical gap, we propose AmoebaLLM, a novel framework designed to enable the instant derivation of LLM subnets of arbitrary shapes, which achieve the accuracy-efficiency frontier and can be extracted immediately after a one-time fine-tuning. In this way, AmoebaLLM significantly facilitates rapid deployment tailored to various platforms and applications. Specifically, AmoebaLLM integrates three innovative components: (1) a knowledge-preserving subnet selection strategy that features a dynamic-programming approach for depth shrinking and an importance-driven method for width shrinking; (2) a shape-aware mixture of LoRAs to mitigate gradient conflicts among subnets during fine-tuning; and (3) an in-place distillation scheme with loss-magnitude balancing as the fine-tuning objective. Extensive experiments validate that AmoebaLLM not only sets new standards in LLM adaptability but also successfully delivers subnets that achieve state-of-the-art trade-offs between accuracy and efficiency.

Figures

Figures reproduced from arXiv: 2411.10606 by the authors.

Figure 1
Figure 1. The latency of LLaMA2 7B with scaled depth/width on various devices/deployment flows. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An overview of our AmoebaLLM framework: (a) Stage 1: Generate the subnet selection [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Benchmark AmoebaLLM’s achieved accuracy-latency trade-offs with SOTA LLM com￾pression methods on an NVIDIA A5000 GPU. Benchmark accuracy-latency trade-offs on real devices. We further benchmark the achieved trade-off between average common￾sense reasoning accuracy and measured latency of LLaMA2 7B using MLC-LLM and PyTorch as the deployment flows on an NVIDIA A5000 GPU, following the settings in Sec. 2. For our meth… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

96 extracted references · 30 canonical work pages

  1. [1]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    arXiv 2023

  2. [2]

    Introducing Meta Llama 3: The most capable openly available LLM to date, 2024

    Meta AI. Introducing Meta Llama 3: The most capable openly available LLM to date, 2024

    2024

  3. [3]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

    arXiv 2024

  4. [4]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  5. [5]

    SparseGPT: Massive language models can be accurately pruned in one-shot, 2023

    Elias Frantar and Dan Alistarh. SparseGPT: Massive language models can be accurately pruned in one-shot, 2023

    2023

  6. [6]

    A simple and effective pruning approach for large language models, 2023

    Mingjie Sun, Zhuang Liu, Anna Bair, and Zico Kolter. A simple and effective pruning approach for large language models, 2023

    2023

  7. [7]

    Llm-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702–21720, 2023

    2023

  8. [8]

    Fluctuation-based adaptive structured pruning for large language models

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865–10873, 2024

    2024

Show all 96 references
  1. [9]

    Shortened llama: A simple depth pruning for large language models

    Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, and Hyoung-Kyu Song. Shortened llama: A simple depth pruning for large language models. arXiv preprint arXiv:2402.02834, 2024

    2024 arXiv

  2. [10]

    Sheared llama: Accelerating language model pre-training via structured pruning

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. Sheared llama: Accelerating language model pre-training via structured pruning. arXiv preprint arXiv:2310.06694, 2023

    2023 arXiv

  3. [11]

    Bignas: Scaling up neural architecture search with big single-stage models

    Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, and Quoc Le. Bignas: Scaling up neural architecture search with big single-stage models. In Computer Vision–ECCV 2020: 16th European Conference...

    2020

  4. [12]

    Attentivenas: Improving neural architecture search via attentive sampling

    Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural architecture search via attentive sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6418–6427, 2021

    2021

  5. [13]

    Alphanet: Improved training of supernets with alpha-divergence

    Dilin Wang, Chengyue Gong, Meng Li, Qiang Liu, and Vikas Chandra. Alphanet: Improved training of supernets with alpha-divergence. In International Conference on Machine Learning, pages 10760–10771. PMLR, 2021

    2021

  6. [14]

    Nasvit: Neural architecture search for efficient vision transformers with gradient conflict-aware supernet training

    Chengyue Gong and Dilin Wang. Nasvit: Neural architecture search for efficient vision transformers with gradient conflict-aware supernet training. ICLR Proceedings 2022, 2022

    2022

  7. [15]

    Once-for-all: Train one network and specialize it for efficient deployment

    Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment. arXiv preprint arXiv:1908.09791, 2019

    1908 arXiv

  8. [16]

    Gradient surgery for multi-task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824–5836, 2020

    2020

  9. [17]

    Conflict-averse gradient descent for multi-task learning

    Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34:18878–18890, 2021. 11

    2021

  10. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  11. [19]

    Tensorrt-llm, 2024

    NVIDIA. Tensorrt-llm, 2024

    2024

  12. [20]

    MLC-LLM, 2023

    MLC team. MLC-LLM, 2023

    2023

  13. [21]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  14. [22]

    The theory of dynamic programming

    Richard Bellman. The theory of dynamic programming. Bulletin of the American Mathematical Society, 60(6):503–515, 1954

    1954

  15. [23]

    Does fine-tuning llms on new knowledge encourage hallucinations? arXiv preprint arXiv:2405.05904, 2024

    Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning llms on new knowledge encourage hallucinations? arXiv preprint arXiv:2405.05904, 2024

    2024 arXiv

  16. [24]

    A comprehensive study of knowledge editing for large language models

    Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, et al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286, 2024

    2024 arXiv

  17. [25]

    Knowledge neurons in pretrained transformers

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696, 2021

    2021 arXiv

  18. [26]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913, 2020

    2012 arXiv

  19. [27]

    What does bert learn about the structure of language? In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics, 2019

    Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. What does bert learn about the structure of language? In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics, 2019

    2019

  20. [28]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems , 35:17359–17372, 2022

    2022

  21. [29]

    How large language models encode context knowledge? a layer-wise probing study

    Tianjie Ju, Weiwei Sun, Wei Du, Xinwei Yuan, Zhaochun Ren, and Gongshen Liu. How large language models encode context knowledge? a layer-wise probing study. arXiv preprint arXiv:2402.16061, 2024

    2024 arXiv

  22. [30]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  23. [31]

    Editing large language models: Problems, methods, and opportunities

    Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. arXiv preprint arXiv:2305.13172, 2023

    2023 arXiv

  24. [32]

    Easyedit: An easy-to-use knowledge editing framework for large language models

    Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, et al. Easyedit: An easy-to-use knowledge editing framework for large language models. arXiv preprint arXiv:2308.07269, 2023

    2023 arXiv

  25. [33]

    The unreasonable ineffectiveness of the deeper layers

    Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A Roberts. The unreasonable ineffectiveness of the deeper layers. arXiv preprint arXiv:2403.17887, 2024

    2024 arXiv

  26. [34]

    Flexible group-level pruning of deep neural networks for on-device machine learning

    Kwangbae Lee, Hoseung Kim, Hayun Lee, and Dongkun Shin. Flexible group-level pruning of deep neural networks for on-device machine learning. In 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 79–84. IEEE, 2020

    2020

  27. [35]

    Efficient joint optimization of layer-adaptive weight pruning in deep neural networks

    Kaixin Xu, Zhe Wang, Xue Geng, Min Wu, Xiaoli Li, and Weisi Lin. Efficient joint optimization of layer-adaptive weight pruning in deep neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17447–17457, 2023

    2023

  28. [36]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017

    2017 arXiv

  29. [37]

    Mole: Mixture of lora experts

    Xun Wu, Shaohan Huang, and Furu Wei. Mole: Mixture of lora experts. In The Twelfth International Conference on Learning Representations, 2023. 12

    2023

  30. [38]

    Llava-mole: Sparse mixture of lora experts for mitigating data conflicts in instruction finetuning mllms

    Shaoxiang Chen, Zequn Jie, and Lin Ma. Llava-mole: Sparse mixture of lora experts for mitigating data conflicts in instruction finetuning mllms. arXiv preprint arXiv:2401.16160, 2024

    2024 arXiv

  31. [39]

    Mixlora: Enhancing large language models fine-tuning with lora based mixture of experts

    Dengchun Li, Yingzi Ma, Naizheng Wang, Zhiyuan Cheng, Lei Duan, Jie Zuo, Cal Yang, and Mingjie Tang. Mixlora: Enhancing large language models fine-tuning with lora based mixture of experts. arXiv preprint arXiv:2404.15159, 2024

    2024 arXiv

  32. [40]

    Stanford alpaca: An instruction-following llama model

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, et al. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/ stanford_alpaca, 2023

    2023

  33. [41]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023

    2023

  34. [42]

    Language model evaluation harness (package version caaf9ab)

    EleutherAI. Language model evaluation harness (package version caaf9ab). https://github. com/EleutherAI/lm-evaluation-harness , 2023

    2023

  35. [43]

    BoolQ: Exploring the surprising difficulty of natural yes/no questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In NAACL, 2019

    2019

  36. [44]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In AAAI, 2020

    2020

  37. [45]

    Hellaswag: Can a machine really finish your sentence? In ACL, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In ACL, 2019

    2019

  38. [46]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641, 2019

    1907 arXiv

  39. [47]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  40. [48]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP, 2018

    2018

  41. [49]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  42. [50]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In ICLR, 2017

    2017

  43. [51]

    Aligning books and movies: Towards story-like visual explanations by watching movies and reading books

    Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on computer ...

    2015

  44. [52]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  45. [53]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  46. [54]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020

    2020

  47. [55]

    Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Hait...

    2022 arXiv

  48. [56]

    Scaling transnormer to 175 billion parameters

    Zhen Qin, Dong Li, Weigao Sun, Weixuan Sun, Xuyang Shen, Xiaodong Han, Yunshen Wei, Baohong Lv, Fei Yuan, Xiao Luo, et al. Scaling transnormer to 175 billion parameters. arXiv preprint arXiv:2307.14995, 2023

    2023 arXiv

  49. [57]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  50. [58]

    Pythia: A suite for analyzing large language models across training and scaling

    Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In Intern...

    2023

  51. [59]

    Glm: General language model pretraining with autoregressive blank infilling

    Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360, 2021

    2021 arXiv

  52. [60]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  53. [61]

    Bloom: A 176b-parameter open-access multilingual language model

    BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili´c, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100, 2022

    2022 arXiv

  54. [62]

    Specializing smaller language models towards multi-step reasoning

    Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. Specializing smaller language models towards multi-step reasoning. In International Conference on Machine Learning, pages 10421–10430. PMLR, 2023

    2023

  55. [63]

    Distilling step-by-step! outperform- ing larger language models with less training data and smaller model sizes

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. Distilling step-by-step! outperform- ing larger language models with less training data and smaller model sizes. arXiv preprint arXiv...

    2023 arXiv

  56. [64]

    Optq: Accurate quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Optq: Accurate quantization for generative pre-trained transformers. In The Eleventh International Conference on Learning Representations, 2022

    2022

  57. [65]

    LLM.int8(): 8-bit matrix multiplication for transformers at scale

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems, 2022

    2022

  58. [66]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, 2023

    2023

  59. [67]

    GPTQ: Accurate post-training compression for generative pretrained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training compression for generative pretrained transformers. In International Conference on Learning Representations, 2023

    2023

  60. [68]

    Spqr: A sparse-quantized representation for near-lossless llm weight compression, 2023

    Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. Spqr: A sparse-quantized representation for near-lossless llm weight compression, 2023

    2023

  61. [69]

    Awq: Activation-aware weight quantization for llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv preprint arXiv:2306.00978, 2023

    2023 arXiv

  62. [70]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023

    2023 arXiv

  63. [71]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, p...

    2023

  64. [72]

    Llm-pruner: On the structural pruning of large language models, 2023

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models, 2023. version 3. 14

    2023

  65. [73]

    Learning n: m fine-grained structured sparse neural networks from scratch

    Aojun Zhou, Yukun Ma, Junnan Zhu, Jianbo Liu, Zhijie Zhang, Kun Yuan, Wenxiu Sun, and Hongsheng Li. Learning n: m fine-grained structured sparse neural networks from scratch. arXiv preprint arXiv:2102.04010, 2021

    2021 arXiv

  66. [74]

    Slimmable neural networks

    Jiahui Yu, Linjie Yang, Ning Xu, Jianchao Yang, and Thomas Huang. Slimmable neural networks. arXiv preprint arXiv:1812.08928, 2018

    2018 arXiv

  67. [75]

    Universally slimmable networks and improved training techniques

    Jiahui Yu and Thomas S Huang. Universally slimmable networks and improved training techniques. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1803–1811, 2019

    2019

  68. [76]

    Autoslim: Towards one-shot architecture search for channel numbers

    Jiahui Yu and Thomas Huang. Autoslim: Towards one-shot architecture search for channel numbers. arXiv preprint arXiv:1903.11728, 2019

    1903 arXiv

  69. [77]

    Adabits: Neural network quantization with adaptive bit-widths

    Qing Jin, Linjie Yang, and Zhenyu Liao. Adabits: Neural network quantization with adaptive bit-widths. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2146–2156, 2020

    2020

  70. [78]

    Switchable precision neural networks

    Luis Guerra, Bohan Zhuang, Ian Reid, and Tom Drummond. Switchable precision neural networks. arXiv preprint arXiv:2002.02815, 2020

    2002 arXiv

  71. [79]

    Any-precision deep neural networks

    Haichao Yu, Haoxiang Li, Humphrey Shi, Thomas S Huang, and Gang Hua. Any-precision deep neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10763–10771, 2021

    2021

  72. [80]

    Any-precision llm: Low-cost deployment of multiple, different-sized llms

    Yeonhong Park, Jake Hyun, SangLyul Cho, Bonggeun Sim, and Jae W Lee. Any-precision llm: Low-cost deployment of multiple, different-sized llms. arXiv preprint arXiv:2402.10517, 2024

    2024 arXiv

  73. [81]

    Flextron: Many-in-one flexible large language model

    Ruisi Cai, Saurav Muralidharan, Greg Heinrich, Hongxu Yin, Zhangyang Wang, Jan Kautz, and Pavlo Molchanov. Flextron: Many-in-one flexible large language model. arXiv preprint arXiv:2406.10260, 2024. 15 NeurIPS Paper Checklist

    2024 arXiv

  74. [82]

    Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: We have accurately summarized our paper’s contributions and scope in the abstract and introduction. Guidelines: • The ...

  75. [83]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discussed this in Sec. 5.5 of our paper. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that the pap...

  76. [84]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory Assumptions and Proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] 16 Justification: We didn’t provide theoretical results. Guidelines: • The answer NA means that the paper do...

  77. [85]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental Result Reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  78. [86]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? 17 Answer: [Yes] Justification: We have open-sourced our...

  79. [87]

    5.1 of our paper and also provided sufficient references

    Experimental Setting/Details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: We offered this information i...

  80. [88]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment Statistical Significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [No] Justification: We follow our baselines and do not report the error...

  81. [89]

    5.1 of our paper

    Experiments Compute Resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We provided this inform...

  82. [90]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code Of Ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: Yes, it conforms with NeurIPS Code of Ethnics. Guidelines: • The answer NA mean...

  83. [91]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: This work targets the acceleration of large language models to facilitate its widespread use and thus does...

  84. [92]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: We built on top...

  85. [93]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Yes, we have add...

  86. [94]

    Guidelines: • The answer NA means that the paper does not release new assets

    New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: We did not introduce new assets. Guidelines: • The answer NA means that the paper does not release new assets. • Rese...

  87. [95]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and Research with Human Subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  88. [96]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.