Pith. sign in

REVIEW 5 major objections 5 minor 41 references

Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hypernetwork-generated low-rank adapters can inject fact corpora into frozen language models with predictable power-law gains and steeper out-of-distribution scaling than fine-tuning.

desk verdict First systematic study of hypernetwork scaling for knowledge injection, with a valuable new benchmark, but the headline OOD advantage over fine-tuning rests on a few noise-sensitive power-law fits. read the letter →

arxiv 2607.19604 v1 pith:5DTDP5BA submitted 2026-07-21 cs.CL cs.LG

Nischay Dhankhar , Dos Baha , Abulhair Saparov This is my paper
classification cs.CLcs.LG
keywords hypernetworksscalinglawsknowledgeinjectionout-of-distributiongeneralizationparameter-efficientfine-tuninglow-rankadaptersmulti-hopreasoningfrozenlanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a hypernetwork—a small transformer trained from scratch to write low-rank adapters for a frozen language model—can internalize a large fact corpus at training time, and that this ability obeys smooth power laws in hypernetwork width, hypernetwork depth, target model size, and number of facts per example. If the paper is right, knowledge injection becomes predictable: practitioners can budget compute between the injector network and the target model, and bigger target models produce disproportionately better generalization to unseen domains, rephrased questions, and multiple-choice formats. The paper further claims that on all three out-of-distribution evaluations this generalization improves faster with target size than LoRA fine-tuning or full fine-tuning, so the hypernetwork's edge widens exactly where large-scale deployment matters. The evidence comes from a deterministic multi-hop question-answering benchmark built from a public knowledge graph, with tens of millions of generated examples and held-out domains designed to test generalization.

What carries the argument

The load-bearing mechanism is a randomly initialized transformer-based hypernetwork that maps a batch of verbalized facts to a pooled representation, from which linear projection heads produce low-rank weight updates (rank 4, scale 8) for the later half of a frozen target model's layers. This decouples injection capacity from general language ability, so hypernetwork depth and width and target size can be varied independently. The scaling analysis fits power laws L = a x^b to final-epoch loss on four metrics—in-distribution validation, OOD non-rephrased, OOD rephrased, and OOD multiple-choice—which is what turns architecture choices into quantitative predictions.

What would settle it

Rerun the target-scaling comparison with held-out domains chosen to match the pretraining accuracy of in-distribution domains, or with counterfactual facts the frozen model cannot know; if the hypernetwork's OOD exponents no longer exceed LoRA and full fine-tuning, the reported OOD advantage is an artifact of domain selection.

Watch

Extended reading notes

Core claim

The central claim is that train-time knowledge injection via a hypernetwork shows clean power-law scaling: validation and out-of-distribution loss decrease as L = a x^b in hypernetwork width (exponent about -0.10), depth (about -0.09), number of injected facts (about -0.08), and target model size (about -0.23). Comparing target-size scaling across adaptation methods, fine-tuning is marginally steeper in-distribution (LoRA -0.250, full fine-tuning -0.249 vs hypernetwork -0.226), but the hypernetwork is steeper on every OOD split: non-rephrased -0.184 vs -0.151/-0.183, rephrased -0.107 vs -0.083/-0.069, and multiple-choice -0.171 vs -0.119/-0.101. The paper interprets this as evidence that hyp

Load-bearing premise

The out-of-distribution advantage depends on the assumption that the held-out-domain losses measure generalization of the injected facts, even though the frozen target model already answers those domains about 4.8 percentage points above the in-distribution mean before any adaptation.

Editorial extensions

If this is right

  • Knowledge-injection performance can be forecast before training: the fitted power laws let a practitioner choose hypernetwork size and target model to meet a target loss or accuracy.
  • With fixed compute, increasing target model size is the highest-return lever; its exponent is roughly 2.5 times steeper than increasing hypernetwork width or depth.
  • If the OOD exponent gap holds, hypernetwork-based injection becomes progressively more attractive than fine-tuning as target models grow, instead of being a fixed-offset tradeoff.
  • Packing more facts per example (up to the hypernetwork context limit) reliably improves performance, so the fact-batch size is a safe scaling knob.
  • Deploying large hypernetworks requires addressing their parameter overhead: the largest injector studied is roughly 2.5B parameters against a 1.5B target, motivating compression or weight-sharing across layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the rephrased-OOD exponent is the flattest on every scaling axis, scale alone may not make injected knowledge robust to paraphrase; testing paraphrase augmentation or diverse question templates is a natural extension the paper leaves open.
  • The clean decoupling of injector and target suggests compositionality properties not explored here—one hypernetwork could potentially serve several targets, or one target could receive many independently trained fact adapters—which would be directly testable with the released benchmark.
  • If these power laws transfer from grammar-generated QA to noisy, unstructured corpora, hypernetworks could become a practical route for private or continuously updated knowledge in local models; that transfer is an empirical question the paper does not address.
  • The deterministic construction and explicit OOD protocol give future work a controlled testbed for comparing knowledge injection, editing, and retrieval methods on the same generalization axes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes scaling laws for hypernetwork-based knowledge injection into frozen LLMs. A hypernetwork generates LoRA adapters conditioned on an injected fact set, and the authors fit power laws L = a x^b to final-epoch loss as functions of hypernetwork width, depth, target model size, and number of injected facts, using a new deterministic QA dataset (MegaWikiQA) built from Wikidata5M. They compare hypernetwork scaling with LoRA fine-tuning and full fine-tuning and claim that, while fine-tuning scales slightly better in-distribution, hypernetworks exhibit steeper out-of-distribution (OOD) scaling across three OOD splits. The paper also releases code and data.

Significance. If the claims hold, this would be the first systematic scaling-law study for hypernetwork-based knowledge injection and would provide a practical design guideline for train-time adaptation. The paper has notable strengths: MegaWikiQA is large, deterministic, and publicly released; the compute accounting in Appendix A is explicit; and the authors candidly acknowledge in Appendix B that the chosen OOD domains have higher base-model accuracy. The central OOD-advantage claim, however, rests on fitted exponents from very few points with no uncertainty quantification, under a learning-rate protocol that can itself impose power-law structure, and on OOD domains confounded by pretraining familiarity. These issues are load-bearing and require additional analysis before the headline conclusion can be accepted.

major comments (5)
  1. [Table 1; Sections 5.5–5.7] The central claim that hypernetworks have steeper OOD scaling than LoRA and full fine-tuning rests on exponent differences estimated from only 4–5 points per fit, with single runs and no confidence intervals, seed variance, or leave-one-out analysis. The gaps are small (e.g., OOD rephrased −0.107 vs. −0.083 vs. −0.069; OOD MCQ −0.171 vs. −0.119 vs. −0.101) and could easily be within fitting error. The 0.5B point is explicitly identified as an outlier in Section 5.3, yet it is included and can dominate the target-model slope. Please provide confidence intervals, leave-one-out fits, multiple-seed variance, and fits with/without the 0.5B point. Without this, the exponent ordering in Table 1 is not statistically established.
  2. [Appendix D.6] The anchor-and-interpolate learning-rate protocol fits a power law η*(x)=a x^b + c to learning rates at three anchor points and then uses that fitted curve to set learning rates for all intermediate configurations. The final losses of those configurations are then fit to a power law. This creates a channel through which the assumed power-law relation is imposed on the data rather than discovered. Please show that the reported exponents are robust to the learning-rate protocol — e.g., by using a fixed LR, a per-configuration grid, or a non-power-law interpolation — or otherwise demonstrate that the fitted exponents are not an artifact of this tuning choice.
  3. [§3.3; Appendix B; Figure 8] The OOD split is confounded: the three held-out domains were chosen because they are 'most distinct,' but Appendix B reports they have approximately 4.8% higher base-model accuracy than the in-distribution mean. The OOD non-rephrased losses in Figures 2–7 are correspondingly close to ID losses. The rephrased and MCQ splits partially mitigate surface-level confounds, but they are built from the same high-familiarity domains and still rely on the fragile exponent fits above. Please report per-domain OOD results, control for base accuracy (e.g., include held-out domains with matched base accuracy), and show that the OOD advantage persists on questions that require the injected facts rather than reflecting the target model's pretraining familiarity.
  4. [Appendix A, Tables 2 and 4; Sections 5.1, 5.4] Width and fact-count scaling are not iso-compute: Appendix A states that varying d_model from 64 to 1024 introduces a ~7.4× compute range and varying N_facts from 2 to 52 introduces a ~13.7× range. The paper's framing that these axes characterize architectural or data-count scaling therefore conflates capacity with compute. This is especially relevant for the comparison of exponents across axes in Table 1 (e.g., width vs. depth), where only depth is approximately iso-compute. Please either provide iso-compute comparisons or explicitly reinterpret these exponents as joint architecture-compute scaling exponents.
  5. [Sections 5.5–5.8; Appendix D.6] The comparison between hypernetwork, LoRA, and full fine-tuning uses different hyperparameter and training budgets: LoRA uses r=16/α=32 while the hypernetwork uses r=4/α=8; learning rates are selected by different protocols (anchor-and-interpolate for the hypernetwork); epochs and batch sizes are selected per run based on convergence. Exponent differences in Table 1 may therefore reflect tuning choices rather than the adaptation mechanism. Also, full fine-tuning lacks the 14B point (Section 5.6), so the claimed 'full FT OOD non-rephrased' exponent is estimated on four points only. Please match compute and hyperparameter budgets across methods, or provide sensitivity analyses showing that the exponent ordering is robust to these choices.
minor comments (5)
  1. [Section 3.1 / Table 6] The text says the generation procedure produced approximately 10 million examples per hop count, but the final training split is 1.25M examples. Please clarify the filtering/balancing steps and report the number of examples per hop count in the final split.
  2. [Figures 2–3] There are minor inconsistencies between the fitted exponents reported in the figure captions and the text (e.g., depth validation fit is shown as y=0.68L^{−0.090} in Figure 3 while the text reports −0.088; width OOD rephrased is −0.036 in the figure and −0.036 in the text, but the coefficients could be stated more consistently). Please harmonize all reported exponents.
  3. [Section 3.3] The MCQ distractors are 'sampled uniformly at random from Ω.' Please specify whether duplicate distractors or distractors equal to the correct answer are filtered, and report the number of choices that are actually valid.
  4. [Section 4.2 / Appendix D.3] The hypernetwork output includes a fixed scalar c=0.01 in ΔW=αc BA/r. The interaction between this scalar, the LoRA rank, and the target-layer fraction is not studied; a sentence explaining why c=0.01 is appropriate across all target model sizes would help.
  5. [Appendix F] The loss-trajectory figures plot validation and OOD non-rephrased loss but not OOD rephrased or MCQ trajectories. Since the paper's strongest claim concerns rephrased and MCQ OOD scaling, showing those trajectories (or stating they are omitted for space) would strengthen confidence that the reported exponents are not driven by the final epoch alone.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the scaling laws are empirical fits with independent baselines, not self-referential derivations.

full rationale

The paper's central claims are empirical scaling-law fits, not derivations that assume their conclusions. The exponents in Table 1 are obtained by least-squares regression on final-epoch losses (Section 5) measured on held-out splits that are disjoint from training at the triplet level (Section 3.3). No equation defines a scaling exponent in terms of the fitted quantity, and the head-to-head comparison against LoRA and full finetuning uses independently trained baselines, so the OOD-scaling claim does not reduce to a self-citation or to a parameter fitted to that claim. The only self-referential design element is the anchor-and-interpolate LR schedule (Appendix D.6), where a fitted LR-vs-scale curve sets hyperparameters for intermediate runs whose losses are subsequently fit; this is a tuning procedure rather than a logical identity, and it does not make any reported exponent tautological. Appendix B candidly notes that the OOD domains have roughly 4.8% higher base-model accuracy, which is a validity confound for the non-rephrased split, but not circularity; the rephrased and MCQ splits are intended to mitigate this. Likewise, the acknowledged 0.5B outlier (Section 5.3) and the absence of confidence intervals affect the robustness of the exponent ordering, but they are correctness risks, not circularity. There is no load-bearing self-citation chain or imported uniqueness theorem. Overall, the derivation chain is self-contained and empirically grounded, with no significant circularity.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical or theoretical entities. The load-bearing free parameters are the fitted power-law exponents themselves and the per-configuration learning rates chosen from an anchor fit. The key domain assumptions are the unambiguity of the generated QA data, the comparability of Qwen2.5 sizes, and the validity of the OOD splits as tests of injected-fact generalization.

free parameters (9)
  • width scaling exponent b_width (ID validation) = -0.096
    Fitted by least squares to 5 final-epoch loss points (d_model 64-1024); no CI/R^2 reported; this exponent is the posted scaling law.
  • depth scaling exponent b_depth (ID validation) = -0.088
    Fitted to 5 points (1-16 layers); projection heads dominate parameter count, so depth scaling is near iso-compute.
  • target-model scaling exponent b_target (ID validation) = -0.226
    Fitted to Qwen2.5 0.5B-14B; the 0.5B point is noted as an outlier above the fit.
  • fact-count scaling exponent b_fact (ID validation) = -0.080
    N is facts per example, 2-52; the axis also changes context length and FLOPs by ~13.7x, so the exponent is not a pure knowledge-corpus scaling.
  • LoRA FT target scaling exponent (ID validation) = -0.250
    Fitted with r=16, alpha=32; the comparison with the hypernetwork uses unmatched trainable parameter budgets.
  • Full FT target scaling exponent (ID validation) = -0.249
    Fitted for 0.5B-7B; 14B omitted due to compute constraints.
  • Per-config learning rate from anchor-and-interpolate fit = e.g. width 5e-4 to 6e-5; target 6e-4 to 4e-5
    Intermediate LR values are obtained from a fitted curve eta*=a*x^b+c over three anchor points (Appendix D.6), which can imprint power-law structure on loss-vs-scale curves.
  • LoRA rank/scale and adaptation magnitude = r=4, alpha=8, c=0.01 for hypernetwork; r=16, alpha=32 for LoRA baseline
    Chosen by hand, not tuned per scale; the mismatch between methods directly affects the OOD comparison.
  • Target layer fraction = last floor(L/2) layers
    Design choice; when target L changes, the number of generated LoRA matrices changes, so target scaling conflates model size with adapter count.
assumptions (6)
  • ad hoc to paper Loss follows L = a*x^b over the studied ranges
    Section 5 states power-law fits are estimated via least squares in log-log space; no alternative functional forms or goodness-of-fit measures are reported. The central scaling-law claim is the fitted exponent.
  • domain assumption Wikidata5M triplets and grammar-based verbalization yield unambiguous QA labels
    Section 3.1 restricts to one-to-one or many-to-one relations to make answers deterministic; no human validation of question naturalness or answer correctness beyond domain-label checks is reported.
  • domain assumption Qwen2.5 models of different sizes are comparable along a single target-size axis
    Section 5 uses Qwen2.5 0.5B-14B and applies adapters to the last floor(L/2) layers; architecture consistency is assumed, and adapter count varies with L.
  • domain assumption OOD domain evaluation measures generalization of injected facts rather than target-model prior knowledge
    Section 3.3 and Figure 8 show the selected OOD domains have 4.8% higher base accuracy than the ID mean; the paper acknowledges this but still interprets OOD losses as hypernetwork generalization.
  • domain assumption Random initialization of the hypernetwork isolates architectural capacity
    Section 4.2 deliberately avoids pretrained hypernetwork weights; the observed scaling then mixes architectural capacity with optimization difficulty.
  • ad hoc to paper Anchor-and-interpolate learning-rate selection is a fair tuning protocol
    Appendix D.6 fits eta*=a*x^b+c on three anchors and applies it to all intermediate configurations; this is not a fixed-protocol scaling analysis and can manufacture power-law-looking loss curves.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Models." pith.science (2026). https://pith.science/paper/5DTDP5BA

@misc{pith2026260719604,
  author       = {Pith},
  title        = {Pith review of: Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5DTDP5BA}},
  note         = {Machine review of arXiv:2607.19604}
}
read the original abstract

Injecting factual knowledge into large language models (LLMs) reliably and at scale remains an open challenge. Hypernetworks provide a promising solution to large-scale knowledge injection. Although hypernetworks are typically applied for test-time adaptation, we explore their use in train-time knowledge injection, where, given a large corpus of facts, we train a hypernetwork to generate a fixed LoRA adapter that, when inserted into the target model, enable the model to answer questions about those facts. In this work, we investigate whether hypernetworks can be used to perform train-time knowledge injection and how this ability varies with scale. The scaling behavior of hypernetworks remains largely unstudied. Our design decouples the hypernetwork's injection capacity from the target model's general capability, enabling, for the first time, a rigorous study of scaling laws for hypernetwork architectures. We characterize how loss, reasoning accuracy, and out-of-distribution (OOD) generalization vary with hypernetwork depth, width, and target network size. We construct a large-scale dataset, called MegaWikiQA, containing tens of millions of multi-hop question-answer examples across 39 domains constructed from examples in Wikidata5M. Our results reveal: (i) hypernetwork-based injection exhibits broadly predictive power law scaling along all architecture axes; and (ii) hypernetworks are capable of reliable OOD generalization at increasing scales, suggesting that hypernetwork provides a promising alternative to other train-time adaptation methods such as LoRA finetuning and full fine-tuning, exhibiting steeper scaling exponents in all OOD evaluations. Together, these results establish hypernetworks as a principled and scalable substrate for train-time adaptation, and provide the first empirically grounded scaling laws to guide hypernetworks for factual reasoning in large language models.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 5 canonical work pages

  1. [1]

    Pyrkin and Sergei Popov and Artem Babenko , title =

    Anton Sinitsin and Vsevolod Plokhotnyuk and Dmitry V. Pyrkin and Sergei Popov and Artem Babenko , title =. 8th International Conference on Learning Representations,. 2020 , url =

    2020

  2. [2]

    Editing Factual Knowledge in Language Models , booktitle =

    Nicola De Cao and Wilker Aziz and Ivan Titov , editor =. Editing Factual Knowledge in Language Models , booktitle =. 2021 , url =. doi:10.18653/V1/2021.EMNLP-MAIN.522 , timestamp =

    doi:10.18653/v1/2021.emnlp-main.522 2021

  3. [3]

    Locating and Editing Factual Associations in

    Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov , editor =. Locating and Editing Factual Associations in. Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022 , year =

    2022

  4. [5]

    Andonian and Yonatan Belinkov and David Bau , title =

    Kevin Meng and Arnab Sen Sharma and Alex J. Andonian and Yonatan Belinkov and David Bau , title =. The Eleventh International Conference on Learning Representations,. 2023 , url =

    2023

  5. [6]

    Why Does New Knowledge Create Messy Ripple Effects in LLMs? , booktitle =

    Jiaxin Qin and Zixuan Zhang and Chi Han and Pengfei Yu and Manling Li and Heng Ji , editor =. Why Does New Knowledge Create Messy Ripple Effects in LLMs? , booktitle =. 2024 , url =. doi:10.18653/V1/2024.EMNLP-MAIN.700 , timestamp =

    work page doi:10.18653/v1/2024.emnlp-main.700 2024

  6. [7]

    Model Editing at Scale leads to Gradual and Catastrophic Forgetting , booktitle =

    Akshat Gupta and Anurag Rao and Gopala Anumanchipalli , editor =. Model Editing at Scale leads to Gradual and Catastrophic Forgetting , booktitle =. 2024 , url =. doi:10.18653/V1/2024.FINDINGS-ACL.902 , timestamp =

    doi:10.18653/v1/2024.findings-acl.902 2024

  7. [8]

    Dai and Quoc V

    David Ha and Andrew M. Dai and Quoc V. Le , title =. 5th International Conference on Learning Representations,. 2017 , url =

    2017

  8. [9]

    Manning , title =

    Eric Mitchell and Charles Lin and Antoine Bosselut and Chelsea Finn and Christopher D. Manning , title =. The Tenth International Conference on Learning Representations,. 2022 , url =

    2022

Show all 41 references
  1. [10]

    CoRR , volume =

    Zeyu Leo Liu and Greg Durrett and Eunsol Choi , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2506.08920 , eprinttype =. 2506.08920 , timestamp =

    2025 doi arXiv

  2. [11]

    Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen

    Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen. LoRA: Low-Rank Adaptation of Large Language Models , booktitle =. 2022 , url =

    2022

  3. [12]

    Xiaozhi Wang and Tianyu Gao and Zhaocheng Zhu and Zhengyan Zhang and Zhiyuan Liu and Juanzi Li and Jian Tang , title =. Trans. Assoc. Comput. Linguistics , volume =. 2021 , url =. doi:10.1162/TACL\_A\_00360 , timestamp =

    2021 doi

  4. [13]

    Brown and Benjamin Chess and Rewon Child and Scott Gray and Alec Radford and Jeffrey Wu and Dario Amodei , title =

    Jared Kaplan and Sam McCandlish and Tom Henighan and Tom B. Brown and Benjamin Chess and Rewon Child and Scott Gray and Alec Radford and Jeffrey Wu and Dario Amodei , title =. CoRR , volume =. 2020 , url =. 2001.08361 , biburl =

    2020 arXiv

  5. [14]

    Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue , booktitle =

    Jia. Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue , booktitle =. 2024 , url =. doi:10.18653/V1/2024.EMNLP-MAIN.934 , timestamp =

    2024 doi

  6. [15]

    Hyper-X:

    Ahmet. Hyper-X:. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,. 2022 , url =. doi:10.18653/V1/2022.EMNLP-MAIN.541 , timestamp =

    2022 doi

  7. [16]

    Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection , booktitle =

    Louis B. Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection , booktitle =. 2025 , url =

    2025

  8. [17]

    Transformer Feed-Forward Layers Are Key-Value Memories , booktitle =

    Mor Geva and Roei Schuster and Jonathan Berant and Omer Levy , editor =. Transformer Feed-Forward Layers Are Key-Value Memories , booktitle =. 2021 , url =. doi:10.18653/V1/2021.EMNLP-MAIN.446 , timestamp =

    2021 doi

  9. [18]

    AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models , booktitle =

    Junfeng Fang and Houcheng Jiang and Kun Wang and Yunshan Ma and Jie Shi and Xiang Wang and Xiangnan He and Tat. AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models , booktitle =. 2025 , url =

    2025

  10. [19]

    Peng Wang and Zexi Li and Ningyu Zhang and Ziwen Xu and Yunzhi Yao and Yong Jiang and Pengjun Xie and Fei Huang and Huajun Chen , editor =. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vanc...

    2024

  11. [20]

    Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwal and Ariel Herbert. Language Models are Few-Shot Learners , booktitle =. 202...

    2020

  12. [21]

    Long Ouyang and Jeffrey Wu and Xu Jiang and Diogo Almeida and Carroll L. Wainwright and Pamela Mishkin and Chong Zhang and Sandhini Agarwal and Katarina Slama and Alex Ray and John Schulman and Jacob Hilton and Fraser Kelton and Luke Miller and Maddie Simens and Amanda Askell ...

    2022

  13. [22]

    Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models , booktitle =

    Jeffrey Gu and Serena Yeung. Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models , booktitle =. 2025 , url =

    2025

  14. [23]

    Jianlin Su and Murtadha H. M. Ahmed and Yu Lu and Shengfeng Pan and Wen Bo and Yunfeng Liu , title =. Neurocomputing , volume =. 2024 , url =. doi:10.1016/J.NEUCOM.2023.127063 , timestamp =

    2024

  15. [24]

    CoRR , volume =

    An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and...

    2024 doi arXiv

  16. [25]

    7th International Conference on Learning Representations,

    Ilya Loshchilov and Frank Hutter , title =. 7th International Conference on Learning Representations,. 2019 , url =

    2019

  17. [26]

    The Twelfth International Conference on Learning Representations,

    Suhas Kotha and Jacob Mitchell Springer and Aditi Raghunathan , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  18. [27]

    Revisiting Catastrophic Forgetting in Large Language Model Tuning , booktitle =

    Hongyu Li and Liang Ding and Meng Fang and Dacheng Tao , editor =. Revisiting Catastrophic Forgetting in Large Language Model Tuning , booktitle =. 2024 , url =. doi:10.18653/V1/2024.FINDINGS-EMNLP.249 , timestamp =

    2024 doi

  19. [28]

    SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models , booktitle =

    Hossein Hajipour and Ning Yu and Cristian. SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models , booktitle =. 2024 , url =. doi:10.18653/V1/2024.FINDINGS-NAACL.90 , timestamp =

    2024 doi

  20. [29]

    CoRR , volume =

    OpenAI , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2303.08774 , eprinttype =. 2303.08774 , timestamp =

    2023 doi arXiv

  21. [30]

    CoRR , volume =

    Qizhou Chen and Dakan Wang and Taolin Zhang and Zaoming Yan and Chengsong You and Chengyu Wang and Xiaofeng He , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2505.12345 , eprinttype =. 2505.12345 , timestamp =

    2025 doi

  22. [31]

    Zero-Shot Relation Extraction via Reading Comprehension , booktitle =

    Omer Levy and Minjoon Seo and Eunsol Choi and Luke Zettlemoyer , editor =. Zero-Shot Relation Extraction via Reading Comprehension , booktitle =. 2017 , url =. doi:10.18653/V1/K17-1034 , timestamp =

    2017 doi

  23. [32]

    WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs , booktitle =

    Lukas Thede and Karsten Roth and Matthias Bethge and Zeynep Akata and Thomas Hartvigsen , editor =. WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs , booktitle =. 2025 , url =

    2025

  24. [33]

    FinGPT: Open-Source Financial Large Language Models , journal =

    Hongyang Yang and Xiao. FinGPT: Open-Source Financial Large Language Models , journal =. 2023 , url =. doi:10.48550/ARXIV.2306.06031 , eprinttype =. 2306.06031 , timestamp =

    2023 doi

  25. [34]

    2026 , url =

    Chen Ling and Xujiang Zhao and Jiaying Lu and Chengyuan Deng and Can Zheng and Junxiang Wang and Tanmoy Chowdhury and Yun Li and Hejie Cui and Xuchao Zhang and Tianjiao Zhao and Amit Panalkar and Dhagash Mehta and Stefano Pasquali and Wei Cheng and Haoyu Wang and Yanchi Liu an...

    2026 doi

  26. [35]

    Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? , booktitle =

    Zorik Gekhman and Gal Yona and Roee Aharoni and Matan Eyal and Amir Feder and Roi Reichart and Jonathan Herzig , editor =. Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? , booktitle =. 2024 , url =. doi:10.18653/V1/2024.EMNLP-MAIN.444 , timestamp =

    2024 doi

  27. [36]

    CoRR , volume =

    Sara Kothari and Ayush Gupta , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2501.13687 , eprinttype =. 2501.13687 , timestamp =

    2025 doi arXiv

  28. [37]

    CoRR , volume =

    Jiateng Liu and Zhenhailong Wang and Xiaojiang Huang and Yingjie Li and Xing Fan and Xiang Li and Chenlei Guo and Ruhi Sarikaya and Heng Ji , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2510.11588 , eprinttype =. 2510.11588 , timestamp =

    2025 doi

  29. [38]

    Chi and Quoc V

    Jason Wei and Xuezhi Wang and Dale Schuurmans and Maarten Bosma and Brian Ichter and Fei Xia and Ed H. Chi and Quoc V. Le and Denny Zhou , editor =. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , booktitle =. 2022 , url =

    2022

  30. [39]

    The Thirty-Fourth

    Weijie Liu and Peng Zhou and Zhe Zhao and Zhiruo Wang and Qi Ju and Haotang Deng and Ping Wang , title =. The Thirty-Fourth. 2020 , url =. doi:10.1609/AAAI.V34I03.5681 , timestamp =

    2020 doi

  31. [40]

    K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters , booktitle =

    Ruize Wang and Duyu Tang and Nan Duan and Zhongyu Wei and Xuanjing Huang and Jianshu Ji and Guihong Cao and Daxin Jiang and Ming Zhou , editor =. K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters , booktitle =. 2021 , url =. doi:10.18653/V1/2021.FINDINGS-ACL....

    2021 doi

  32. [41]

    Proceedings of the 57th Conference of the Association for Computational Linguistics,

    Zhengyan Zhang and Xu Han and Zhiyuan Liu and Xin Jiang and Maosong Sun and Qun Liu , editor =. Proceedings of the 57th Conference of the Association for Computational Linguistics,. 2019 , url =. doi:10.18653/V1/P19-1139 , timestamp =

    2019 doi

  33. [42]

    Fine-Tuning or Retrieval? Comparing Knowledge Injection in

    Oded Ovadia and Menachem Brief and Moshik Mishaeli and Oren Elisha , editor =. Fine-Tuning or Retrieval? Comparing Knowledge Injection in. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,. 2024 , url =

    2024

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.