Pith. sign in

REVIEW 5 major objections 5 minor 29 references

Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs

T0 review · 5 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AdaDSF converts off-the-shelf pre-trained LLMs into depth-sparse models by allocating compute according to cosine similarity between layer inputs and outputs, cutting inference FLOPs while keeping accuracy close to dense models.

desk verdict Plausible depth-sparse method with a real overclaim — the 90% retention results contradict the abstract's 'consistently' lower degradation. read the letter →

arxiv 2607.21291 v1 pith:545C6GXI submitted 2026-07-23 cs.CL cs.LG

Yidu Wu , Xiang Wang , Kejie Zhao , Zhangchi Wang , Qinghai Guo , Xiaoying Tang This is my paper · ORCID
classification cs.CLcs.LG
keywords adaptivedepthsparsitycosinesimilaritytokenroutinglayer-wiseretentionefficientLLMinferenceknowledgedistillationMixture-of-Depthspre-trainedLLMs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the compute spent on a Transformer layer should track how much that layer actually transforms its input representations. It proposes AdaDSF, which measures each layer's input-to-output cosine similarity on a calibration set, converts those similarities into per-layer token retention ratios, and uses a lightweight router to send only the most informative tokens through each full layer while the rest skip via residuals. A feature-preserving alignment loss then trains the sparse model to keep hidden states and output distributions close to the dense teacher. The reported result is that on GPT-NeoX-130M, Qwen2.5-0.5B, and Qwen2.5-1.5B, this reduces inference FLOPs to roughly 0.68–0.89x of dense while suffering less accuracy degradation than MoD, D-LLM, and DLO at matched sparsity. If correct, it offers a practical path to faster LLM inference from off-the-shelf checkpoints without redesigning the architecture.

What carries the argument

The central object is the layer-wise cosine similarity s_i = CosSim(x_in^(i), x_out^(i)). It feeds a temperature-normalized softmax, deviation scaling, and sigmoid mapping that yields per-layer retention ratios r_i satisfying sum r_i = tL; these ratios set the number of tokens K_i = floor(r_i * s) each sparse layer's router keeps. The router is a lightweight MLP that generates a differentiable Top-K mask. The alignment objective combines output-distribution KL divergence (Eq. 10) with per-layer hidden-state softmax L2 losses (Eq. 9). The machinery's job is to convert a cheap, parameter-free similarity statistic into a data-driven compute budget.

What would settle it

Compute per-layer cosine similarities on the same calibration set, then compare AdaDSF's allocation against an oracle that allocates tokens to maximize held-out accuracy, for example by grid-search or leave-one-layer-out importance. If the similarity-derived ratios do not correlate with oracle-optimal ratios across layers and models, the central allocation claim is unsupported. A simpler check: at fixed FLOPs, run AdaDSF with reversed ratios (more tokens to high-similarity layers) and see whether accuracy changes.

Watch

Extended reading notes

Core claim

AdaDSF's central claim is that representation-transformation statistics—specifically the cosine similarity between a layer's input and output hidden states—provide a usable signal for allocating compute across depth. Layers whose input and output are more different get higher token retention ratios, under a fixed global budget; a trainable MLP router picks the Top-K tokens per layer. Together with hidden-state and output-distribution alignment against the dense model, this converts ordinary pre-trained LLMs into depth-sparse models with minimal architectural change, yielding reported PPL 18.9 on Wikitext103 at 80% retention (vs 17.9 dense) and average commonsense accuracy within about one to

Load-bearing premise

The load-bearing premise is that a layer's input-output cosine similarity measures how much compute that layer contributes, so giving more retained tokens to low-similarity layers improves accuracy; the paper does not independently validate this correlation, and if it fails, the allocation is just a fixed heuristic whose main benefit would come from the alignment loss.

Editorial extensions

If this is right

  • If similarity-driven allocation is right, any off-the-shelf dense LLM can be made depth-sparse with a light training pass, no full retraining or architecture change.
  • At 70–90% token retention, inference FLOPs drop to roughly 0.68–0.89x dense while language modeling PPL stays within about two points of dense on the tested models.
  • Under matched sparsity, the method claims smaller accuracy degradation than MoD, D-LLM, and DLO across six commonsense benchmarks, suggesting the allocation plus alignment objective is doing more work than fixed schedules.
  • The approach generalizes across model families (GPT-NeoX and Qwen2.5) and across retention ratios, a necessary condition for practical deployment.
  • Because retention ratios come from a calibration set rather than online adaptation, the method is cheap to apply and does not require task-specific fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cosine-similarity proxy is only validated indirectly; a reader should check whether the same ratios emerge from an oracle that measures each layer's true contribution, such as ablating layers or measuring downstream loss change. If not, observed gains may be driven chiefly by the alignment objective rather than by the allocation itself.
  • The method's dependence on a calibration set means its allocation quality could degrade under distribution shift; an online or adaptive estimation of layer similarity is a natural testable extension.
  • Because experiments stop at 1.5B parameters, the scalability claim is untested at 7B+; whether similarity statistics remain stable at that scale is an open question.
  • The hidden-state alignment loss (softmax L2) is one particular choice; comparing it against cosine or KL-based intermediate alignment could shift the reported efficiency-accuracy frontier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes AdaDSF, a post-hoc depth-sparsification framework for pre-trained LLMs. It uses the cosine similarity between a layer's input and output hidden states (Eq. 4) to derive per-layer token retention ratios through temperature-normalized weighting, deviation scaling, sigmoid mapping, and global budget correction (Eqs. 5–8). A lightweight MLP router selects the top-K tokens for each layer; unselected tokens skip the Transformer block via the residual path. A feature-preserving alignment objective (Eq. 11), combining output-distribution KL and intermediate hidden-state alignment, is used to train the router and layers. Experiments on GPT-NeoX-130M and Qwen2.5-0.5B/1.5B report language-modeling PPL and six commonsense-reasoning accuracies against MoD, D-LLM, and DLO, together with normalized FLOPs. The central claim is that AdaDSF substantially reduces inference FLOPs while preserving dense-level performance and consistently outperforms strong depth-sparse baselines under comparable sparsity. The manuscript does not include code, error bars, or a validation protocol for hyperparameter selection.

Significance. If the claims are reproducible, the contribution is practically attractive: AdaDSF requires no retraining from scratch and no architectural change to attention, and it converts off-the-shelf checkpoints into depth-sparse models with a simple similarity-derived allocation. The method is easy to state, and the alignment objective is a sensible means of reducing representation shift. The empirical scope is reasonable for a short paper: two model families, three model sizes, and seven tasks. However, several load-bearing pieces of evidence are currently missing or under-specified: the headline comparison to DLO is contradicted at the 90% retention setting, the temperature is selected on the test set with no seeds, and the core 'similarity implies importance' premise is not directly validated. These issues prevent the paper from establishing its central claims as written, but they are addressable with additional analysis and reporting.

major comments (5)
  1. [§4.2, Tables 1b and 2] The abstract and §4.2 claim that AdaDSF 'consistently yields smaller accuracy degradation' than DLO under comparable sparsity. This is contradicted at the 90% retention setting. In Table 1b (Qwen2.5-0.5B), DLO has Diff −2.2 vs. AdaDSF −2.3; in Table 2 (GPT-NeoX, Wikitext-103), DLO has PPL 18.3 vs. AdaDSF 18.5. I note that DLO's normalized FLOPs at 90% is 1.110, i.e., it is not a sparse configuration, so the 90% row cannot serve as a 'comparable sparsity' comparison as it stands. The paper should either recompute matched-FLOP comparisons or explicitly restrict the consistency claim to retention ratios 70–80% / FLOPs<1. The current wording is overbroad, and the 0.1–0.2 point differences may be noise given the absence of error bars.
  2. [§4.3, Table 3] The reported optimal PPL of 18.91 is obtained by sweeping τ ∈ {0.25, 0.1, 0.05} and selecting the best value. No validation split is described; if the selection is made on the Wikitext-103 test set, this is test-set leakage. Moreover, all numbers appear to come from a single run with no seeds or error bars. Given that several headline differences are 0.1–0.2 points (e.g., DLO vs. AdaDSF at 90%), variance estimates are essential. Please add multi-seed results and a clear train/validation/test protocol, including how τ, β, and the target retention t are chosen.
  3. [§4.1 and all experiments] The efficiency metric 'normalized FLOPs' is defined only as 'sparse over dense' in §4.1, with no counting methodology: which operations are counted, how router overhead is handled, and how values such as DLO's 1.110 at 90% retention are computed. Without this, the claim of 'substantially reduces inference FLOPs' is not independently verifiable. Training details (optimizer, learning rate, epochs, batch size, router architecture, calibration-set size) are also absent, preventing reproduction.
  4. [§3.3–3.5, Tables 3–4] The core methodological premise is that input–output cosine similarity (Eq. 4) is a valid importance proxy, so that low-similarity layers should retain more tokens. Table 3 only varies τ, and Table 4 varies the loss; neither isolates the allocation mechanism. No comparison with uniform retention, inverse allocation, or random allocation at matched training budget is reported. As a result, the observed gains could be driven chiefly by the alignment objective in Eq. (11), not by similarity-driven allocation. Please add a direct ablation (e.g., uniform vs. similarity-derived ratios with identical alignment and budget) and, ideally, a correlation analysis between layer similarity and measured contribution to task performance.
  5. [§4.2, Table 2] The D-LLM baseline has PPL ≈1955–2078 on Wikitext-103, three orders of magnitude worse than the dense model, and the text itself says it is 'not suitable' while still using it as a strong baseline. This suggests an implementation or protocol mismatch. If D-LLM cannot be configured successfully for this setting, it should be removed from the comparison; otherwise its inclusion inflates AdaDSF's relative improvements and weakens the 'strong baselines' claim.
minor comments (5)
  1. [Eq. (11)] The symbol L is used both for the total loss and for the number of layers. Rename the total loss to L_total to avoid ambiguity.
  2. [Table 3] The row 'GPT-Neox-AdaDSF (w/o τ)' is ambiguous: does it mean no temperature scaling (i.e., uniform weights), no calibration set, or something else? Please define what is removed.
  3. [Fig. 1] The overview figure is dense and the flow from similarity statistics to retention ratios to the router is hard to follow. Consider separating the three components or enlarging the diagram.
  4. [§2, References] Some related-work references (e.g., [9], [15], [21], [27], [28]) are not clearly connected to depth sparsity or LLM efficiency. Please state their specific relevance or remove them.
  5. [§4.5] The statement that the method is 'inherently scalable' is unsupported by experiments limited to models ≤1.5B. Either add a larger-scale experiment or temper the claim. Also consider adding an availability statement for code.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the only self-citation ([9]) is non-load-bearing. The 'consistently' claim is contradicted by the paper's own 90%-retention tables, but that is a correctness issue, not circularity.

full rationale

AdaDSF's derivation chain is self-contained. The similarity statistics s_i (Eq. 4) are computed once from a frozen dense model over a calibration set; the layer-wise retention ratios r_i (Eqs. 5-8) are a deterministic function of s_i and the global budget t, and do not depend on the reported test metrics (PPL or reasoning accuracy). The router (Sec. 3.4) is trained to select Top-K tokens, and the alignment objective (Eqs. 9-11) trains the sparse model to match dense hidden states and output distributions. That makes 'close to dense' an optimized objective in representation space, not a post-hoc prediction: the reported accuracy/PPL is measured on held-out benchmarks after training and is never fed back into Eqs. (4)-(8) or into the loss. The cosine-similarity premise is an empirical heuristic, not a theorem; the ablation in Table 3 tests it directly (PPL 19.69 without vs 18.91 with). No load-bearing premise is justified by a self-citation: ref. [9] (which shares author Kejie Zhao) is invoked only as a stylistic analogy for instance-aware alignment, and the other self-group citations are related-work background. The paper does contain an internal-consistency problem independent of circularity: at 90% retention, Table 1b shows DLO Diff -2.2 vs AdaDSF -2.3 and Table 2 shows DLO PPL 18.3 vs AdaDSF 18.5, contradicting the abstract's and Sec. 4.2's claim of 'consistently' smaller degradation; this is a reporting/validity concern, not a circular-input concern.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

AdaDSF introduces no new physical or architectural entities beyond a standard MLP router. The load-bearing assumptions are the cosine-similarity importance proxy, representativeness of the calibration subset, and the sufficiency of the softmax-hidden alignment. Free parameters are tau (test-tuned), beta (hand-set), and the user-specified retention budget t.

free parameters (3)
  • tau (temperature) = 0.05 (GPT-NeoX best); 0.25/0.1/0.05 in ablation
    Controls the variance of similarity-derived weights in Eq. 5. Swept and the best value reported (Table 3) with no validation split described; appears selected on the test PPL.
  • beta (deviation scale) = 10
    Hand-set in Eq. 6 to scale deviation from the mean weight; no tuning or sensitivity analysis reported.
  • t (target average retention) = 0.9, 0.8, 0.7 in experiments
    User-specified global budget; determines final ratios via Eq. 8. Not fitted, but a key control variable and part of the 'comparable sparsity' framing.
assumptions (4)
  • domain assumption Cosine similarity between a layer's input and output hidden states measures how much the layer transforms representations, and lower similarity implies higher importance/compute need.
    Used to derive retention ratios in Section 3.3 (Eqs. 4-8). Never validated against an independent importance measure; the ablation only compares temperature values on one test set.
  • domain assumption The calibration subset used to compute s_i is representative of the inference distribution.
    Section 3.3 says 'Using a calibration subset' but does not specify its size, source, or overlap with evaluation data.
  • domain assumption Tokens can skip attention/FFN computation via residual connections while a trained router can identify which tokens to keep.
    Standard depth-sparse assumption adopted from MoD; used in Section 3.4 to justify the bypass mechanism.
  • domain assumption Softmax-normalized hidden states are a suitable alignment target for distillation.
    Eq. 9 uses Softmax(h) distance; no justification is given beyond an appeal to prior work [28].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs." pith.science (2026). https://pith.science/paper/545C6GXI

@misc{pith2026260721291,
  author       = {Pith},
  title        = {Pith review of: Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/545C6GXI}},
  note         = {Machine review of arXiv:2607.21291}
}
read the original abstract

Large language models (LLMs) achieve strong generation and reasoning performance, but the Transformer architecture incurs high inference cost. Existing acceleration methods often rely on task-specific fine-tuning or training from scratch, increasing adaptation cost and limiting cross-task usability. We present an Adaptive Depth Sparse Framework (AdaDSF) that converts off-the-shelf pre-trained LLMs into depth-sparse models without full retraining. Our key insight is that layers contribute unequally to representation transformation, characterized by the cosine similarity between layer input and output hidden states. Based on this, AdaDSF assigns layer-wise token retention ratios from similarity statistics, uses a lightweight router to select informative tokens at each layer, and introduces a feature-preserving alignment objective to match intermediate and final representations between sparse and dense models. On GPT-NeoX and Qwen2.5 over language modeling and commonsense reasoning, AdaDSF substantially reduces inference FLOPs while preserving performance close to dense counterparts. Under comparable sparsity, AdaDSF consistently yields smaller accuracy degradation than strong baselines including MoD, D-LLM, and DLO.

Figures

Figures reproduced from arXiv: 2607.21291 by the authors.

Figure 1
Figure 1. Overview of Adaptive Depth Sparse Framework: Similarity-Driven Re [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Layer-wise retention ratios produced by the Similarity-Driven allocation [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 11 linked inside Pith

  1. [1]

    In: Proceedings of the AAAI conference on artificial intelligence

    Bisk, Y., Zellers, R., Gao, J., Choi, Y., et al.: Piqa: Reasoning about physical commonsense in natural language. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 7432–7439 (2020)

    2020

  2. [2]

    arXiv preprint arXiv:2204.06745 (2022)

    Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al.: Gpt-neox-20b: An open-source autore- gressive language model. arXiv preprint arXiv:2204.06745 (2022)

    arXiv 2022

  3. [3]

    In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition

    Cai, Y., Yao, Z., Dong, Z., Gholami, A., Mahoney, M.W., Keutzer, K.: Zeroq: A novel zero shot quantization framework. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. pp. 13169–13178 (2020)

    2020

  4. [4]

    Chen, J., Qadri, R., Wen, Y., Jain, N., Kirchenbauer, J., Zhou, T., Goldstein, T.: Genqa: Generating millions of instructions from a handful of prompts (2024), https://arxiv.org/abs/2406.10323 AdaDSF: Similarity-Driven Resource Allocation for Pre-Trained LLMs 11

    arXiv 2024

  5. [5]

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., Tafjord, O.: Think you have solved question answering? try arc, the ai2 reasoning challenge (2018),https://arxiv.org/abs/1803.05457

    arXiv 2018

  6. [6]

    In: In- ternational Conference on Learning Representations (2021),https://openreview

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: In- ternational Conference on Learning Representations (2021),https://openreview. net/forum?id=YicbFdNTTy

    2021

  7. [7]

    Journal of Machine Learning Research 23(120), 1–39 (2022)

    Fedus,W.,Zoph,B.,Shazeer,N.:Switchtransformers:Scalingtotrillionparameter models with simple and efficient sparsity. Journal of Machine Learning Research 23(120), 1–39 (2022)

    2022

  8. [8]

    arXiv preprint arXiv:2210.17323 (2022)

    Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D.: Gptq: Accurate post- training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323 (2022)

    arXiv 2022

Show all 29 references
  1. [10]

    Advances in Neural Information Processing Systems37, 1725–1749 (2024)

    Jiang,Y.,Wang,H.,Xie,L.,Zhao,H.,Qian,H.,Lui,J.,etal.:D-llm:Atokenadap- tive computing resource allocation strategy for large language models. Advances in Neural Information Processing Systems37, 1725–1749 (2024)

    2024

  2. [11]

    In: Cohn, T., He, Y., Liu, Y

    Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., Liu, Q.: TinyBERT: Distilling BERT for natural language understanding. In: Cohn, T., He, Y., Liu, Y. (eds.) Findings of the Association for Computational Linguis- tics: EMNLP 2020. pp. 4163–4174. Association ...

    2020 doi

  3. [12]

    In: International conference on machine learning

    Katharopoulos, A., Vyas, A., Pappas, N., Fleuret, F.: Transformers are rnns: Fast autoregressive transformers with linear attention. In: International conference on machine learning. pp. 5156–5165. PMLR (2020)

    2020

  4. [13]

    Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N.,Chen,Z.:{GS}hard:Scalinggiantmodelswithconditionalcomputationandau- tomaticsharding.In:InternationalConferenceonLearningRepresentations(2021), https://openreview.net/forum?id=qrwe7XHTmYb

    2021

  5. [14]

    Li, J., Du, L., Zhao, H., wen Zhang, B., Wang, L., Gao, B., Liu, G., Lin, Y.: Infinity instruct: Scaling instruction selection and synthesis to enhance language models (2025),https://arxiv.org/abs/2506.11116

    2025 arXiv

  6. [15]

    In: ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Lu, Y., Hu, K., Zhang, L.: S3g: Stock state space graph for enhanced stock trend prediction. In: ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 4081–4085 (2026).https://doi.org/ 10.1109/ICASSP55912.2026.11463578

    2026

  7. [16]

    arXiv preprint arXiv:1809.02789 (2018)

    Mihaylov, T., Clark, P., Khot, T., Sabharwal, A.: Can a suit of armor con- duct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789 (2018)

    2018 arXiv

  8. [17]

    In: International confer- ence on machine learning

    Nagel, M., Amjad, R.A., Van Baalen, M., Louizos, C., Blankevoort, T.: Up or down? adaptive rounding for post-training quantization. In: International confer- ence on machine learning. pp. 7197–7206. PMLR (2020)

    2020

  9. [18]

    arXiv preprint arXiv:2404.02258 (2024) 12 Y

    Raposo, D., Ritter, S., Richards, B., Lillicrap, T., Humphreys, P.C., Santoro, A.: Mixture-of-depths: Dynamically allocating compute in transformer-based language models. arXiv preprint arXiv:2404.02258 (2024) 12 Y. Wu, X. Wang, K. Zhao, Z. Wang, Q. Guo, and X. Tang

    2024 arXiv

  10. [19]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Sakaguchi, K., Le Bras, R., Bhagavatula, C., Choi, Y.: Winogrande: An adversarial winograd schema challenge at scale. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 8732–8740 (2020)

    2020

  11. [20]

    Frontiers in Marine Science10, 1174347 (2023)

    Si, G., Xiao, Y., Wei, B., Bullock, L.B., Wang, Y., Wang, X.: Token-selective vision transformer for fine-grained image recognition of marine organisms. Frontiers in Marine Science10, 1174347 (2023)

    2023

  12. [21]

    Su, Y., Zhu, H., Tian, Y., Zhao, C., Peng, Z., Liu, Y., Fan, L., Li, B., Zhang, L.: Agentic-sql taxonomy: A survey of autonomous and interactive text-to-sql with llms (2026)

    2026

  13. [22]

    In: First Workshop on Scalable Optimization for Efficient and Adaptive Foundation Models (2025), https://openreview.net/forum?id=E9Jw3IHuDH

    Tan, Z., Dong, D., Zhao, X., Cai, J., Peng, J., Cheng, Y., Chen, T.: DLO: Dy- namic layer operation for efficient vertical scaling of LLMs. In: First Workshop on Scalable Optimization for Efficient and Adaptive Foundation Models (2025), https://openreview.net/forum?id=E9Jw3IHuDH

    2025

  14. [23]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Tang, Q., Zhang, B., Liu, J., Liu, F., Liu, Y.: Dynamic token pruning in plain vision transformers for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 777–786 (2023)

    2023

  15. [24]

    arXiv preprint arXiv:2407.10671 (2024)

    Team, Q.: Qwen2 technical report. arXiv preprint arXiv:2407.10671 (2024)

    2024 arXiv

  16. [25]

    Teknium: Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants (2023),https://huggingface.co/datasets/teknium/OpenHermes-2.5

    2023

  17. [26]

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: Llama: Open and efficient foundation language models (2023), https://arxiv.org/abs/2302.13971

    2023 arXiv

  18. [27]

    Preprints (March 2026).https://doi.org/10.20944/preprints202603.2262.v1,https:// doi.org/10.20944/preprints202603.2262.v1

    Wang, J., Fan, L., Li, B., Zhang, L.: A dynamic factor gating architec- ture with market regime awareness for stock return forecasting. Preprints (March 2026).https://doi.org/10.20944/preprints202603.2262.v1,https:// doi.org/10.20944/preprints202603.2262.v1

    2026

  19. [28]

    Wang, J., Fan, L., Li, B., Zhang, L.: Forecasting with guidance: Representation- level supervision for time series forecasting (2026),https://arxiv.org/abs/2603. 24262

    2026

  20. [29]

    arXiv preprint arXiv:2402.13116 (2024)

    Xu, X., Li, M., Tao, C., Shen, T., Cheng, R., Li, J., Xu, C., Tao, D., Zhou, T.: A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116 (2024)

    2024 arXiv

  21. [30]

    Zellers,R.,Holtzman,A.,Bisk,Y.,Farhadi,A.,Choi,Y.:Hellaswag:Canamachine really finish your sentence? (2019),https://arxiv.org/abs/1905.07830

    2019 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.