Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Selective state-space recurrence inside low-rank adapters—as a per-token gate and as a segment-level retriever—consistently improves multi-hop reasoning over static LoRA.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:43 UTC pith:CFI6TN23

load-bearing objection Solid empirical paper on stateful Mamba-gated LoRA plus model-attached retrieval; the missing per-seed variance in the headline table is the one real flaw, and it's fixable. the 2 major comments →

arxiv 2607.19326 v1 pith:CFI6TN23 submitted 2026-07-21 cs.CL

Selective State-Space Adaptation and Retrieval for Language Model Reasoning

Atahan Dokme , Larry Heck This is my paper
classification cs.CL
keywords low-rank adaptationstate-space modelstoken-level modulationretrieval adaptermulti-hop QAMambaparameter-efficient fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Low-rank adaptation is usually a static, input-independent update, but this paper argues it should be stateful and input-adaptive. It introduces two adapters built on the same selective state-space recurrence: MaLoRA, which makes the LoRA scaling factor a stateful per-token gate carrying memory of the token trajectory, and MaRA, which scores input segments from the frozen backbone's own hidden states and selects the top-k supporting paragraphs before generation. On three frozen 7-9B backbones (Qwen, Llama, Gemma) and two multi-hop QA benchmarks (MuSiQue, 2WikiMultihopQA), the combined system improves F1 over LoRA on every cell of the 3x2 grid by +6.8 F1 on average and up to +9.3 F1 on the hardest cell; token-level gains also survive longer-context RULER QA-2. If true, frozen models already encode evidence in their intermediate hidden states, and two cheap state-space mechanisms can exploit that signal without training a separate retriever.

Core claim

On the paper's own terms, the discovery is that a single primitive—the selective state-space (Mamba) recurrence—can be placed at two adaptation granularities and deliver complementary gains. MaLoRA replaces TopLoRA's stateless per-token diagonal with a recurrent scalar gate lambda that suppresses or amplifies the LoRA update according to the token's trajectory; the trained gate is role-sensitive but paragraph-relevance-blind, suppressing structure and query tokens while leaving supporting and distractor content nearly identical. MaRA supplies the missing relevance signal: it encodes each candidate segment together with the query through the first 16 frozen layers, pools to one embedding per

What carries the argument

The load-bearing object is the selective state-space recurrence (a Mamba block), used twice: once as the per-token modulator of the LoRA update in MaLoRA, where the recurrence runs across tokens and emits a scalar gate lambda; and once as the segment-level mixer in MaRA, where the recurrence runs across attention-pooled segment embeddings and emits per-segment relevance scores. Its work is to give the adapter memory of the trajectory leading up to each token and to propagate reasoning state across context segments, enabling evidence selection without a separate retriever. The only other trained components of MaRA are a small attention pool plus two linear scoring heads (about 3M parameters).

Load-bearing premise

MaRA's evidence selection rests on the premise that the first 16 layers of the frozen backbone, after encoding each candidate segment with the query, already contain enough information for a small trained mixer to pick supporting paragraphs; the paper's own recall numbers show this is backbone-dependent (Gemma trails at 0.93 on MuSiQue), so on a new backbone or task the retrieval contribution could shrink to near zero even if the token-level modulator still helps.

What would settle it

Concrete check: run the canonical training protocol on a held-out backbone (e.g., a 3B-class model) and measure MaRA recall@12 on MuSiQue; if recall fails to reach 0.95 and the combined system's F1 gain over LoRA falls below the smallest reported +4.0, the claim that the family improves on every cell would not transfer.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A frozen 7-9B backbone's first 16 layers already contain query-conditioned evidence: MaRA's recall reaches 0.97 at k=12 on MuSiQue and about 0.99 at k=4 on 2WikiMultihopQA, so no external retriever is needed for these tasks.
  • Stateness matters: MaLoRA's recurrent gate beats the stateless TopLoRA on every cell, and a control probe shows the Mamba state accumulates information across positions rather than acting decoratively.
  • The two mechanisms compose: modulation and retrieval each contribute, and their gains mostly add (only about 18% overlap on the hardest cell), so the combined system is more robust than either alone.
  • Retrieval gains depend on how well the backbone already discriminates evidence: on easier 2Wiki retrieval alone contributes more, while on harder MuSiQue both are needed; Gemma's weaker router shrinks the retrieval lift.
  • The method holds k fixed across backbones by a recall criterion rather than end-task F1, so the same setting transfers across all three models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the first 16 layers already carry the evidence, then a much simpler probe—such as a linear classifier on pooled layer-16 states—might recover a large fraction of MaRA's retrieval gain; the Mamba mixer's added value could be isolated by comparing against such a probe.
  • MaRA's causal segment-level recurrence makes evidence selection order-sensitive: the score of segment i depends on segments 1..i-1, so reordering candidate paragraphs could change which segments are retrieved. This is a testable prediction the paper does not make.
  • The paper's own limitation—that a solution-step variant for mathematical reasoning did not yield a clean lift—suggests the benefit may depend on paragraphs being a semantically coarse, self-contained unit; applying MaRA to token-level or sentence-level evidence units is an open question.
  • If MaRA's per-segment states were injected into the generator (as the paper lists as future work), the explicit top-k cut could become unnecessary, since the generator could attend to graded relevance rather than a binary selection; this would trade recall for robustness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes two complementary adapters for frozen LLMs: MaLoRA, a Mamba-based per-token scalar gate that modulates the LoRA update with recurrence across tokens, and MaRA, a segment-level retrieval adapter that scores candidate paragraphs from the first 16 layers of the frozen backbone with a segment-level Mamba mixer and selects top-k before generation. Experiments are conducted on MuSiQue and 2WikiMultihopQA with Qwen-2.5-7B, Llama-3.1-8B, and Gemma-2-9B, plus a RULER QA-2 probe. The central claim is that the combined system improves over LoRA on every cell of the 3×2 grid, by +6.8 F1 on average and up to +9.3 F1, with the two mechanisms contributing complementary gains. The paper also reports architecture ablations for MaRA, comparison to BM25/embedding retrievers, efficiency, and limitations.

Significance. If the central claim holds, the contribution is significant for parameter-efficient adaptation of LLMs and for retrieval-augmented reasoning: it introduces a stateful PEFT mechanism that goes beyond stateless gating, and a model-attached retriever that avoids a separate encoder by using frozen-backbone hidden states. The experimental protocol is careful: three seeds, a shared canonical training pool, validation-based k selection, no best-seed selection, and full disclosure of training costs and limitations. However, the headline 'every cell' claim is not yet fully supported statistically because per-seed standard deviations are missing for the combined-system rows in Table 4. The work also makes the evidence-selection premise explicit and tests it with ablations; this transparency is a strength.

major comments (2)
  1. [Table 4 / Appendix D] The headline claim is "improves on every cell of the 3×2 grid" (abstract, §5). All Table 4 rows are stated to be means over three seeds (Appendix D), but per-seed standard deviations are only given for the token modulators (Table 7), not for the combined MaLoRA+MaRA rows. The smallest gain is +4.0 F1 on MuSiQue Gemma (59.8→63.8); with MaLoRA-only σ=2.3 F1 on the same cell and three seeds, this difference is about 1.7σ, so the "every cell" claim is not statistically supported as reported. The same missing variance affects the average +6.8 F1 and the claim that modulation and retrieval gains "accumulate rather than overlap." Please report per-seed EM/F1 for every row in Table 4 and provide significance tests for the LoRA comparison in each cell.
  2. [Section 5 / Appendix K] The evidence-selection premise is backbone-dependent. k=12 on MuSiQue is chosen because Qwen/Llama reach recall 0.97, and the same k is imposed on Gemma, whose recall is 0.93 (Table 11). The text acknowledges this, but the abstract's unconditional "every cell" claim obscures that the smallest gain (+4.0 F1 on MuSiQue Gemma) is exactly the cell where the retrieval component contributes least. To make the main result robust, either apply the recall criterion per backbone and report the resulting k, or present the main grid under both fixed-k and per-backbone k; at minimum, qualify the abstract.
minor comments (6)
  1. [Section 5] The phrase "approximately 18% overlap" in the MuSiQue Qwen decomposition is not defined. Please specify the formula (e.g., 1 − S/(M+R), where S is the combined gain and M, R are the individual gains).
  2. [Table 3 / Table 10] Recall@4 numbers are reported without per-seed standard deviations for MaRA architecture comparisons and retrieval baselines. The gaps are large, but adding variance would make these tables consistent with Table 7 and would strengthen the comparative claims.
  3. [Section 3.4] The stateless counterfactual z_stateless_t = Mamba(P x_t) zeroes the recurrent state but keeps the same A, B, C parameters. Clarify that this measures the information carried by the state under the trained dynamics, not a comparison to a separately learned stateless head.
  4. [Table 1] The RULER QA-2 column reports accuracy only at 4k context. State whether 4k exceeds the training sequence length for these checkpoints and how this supports the "length stress" interpretation.
  5. [Abstract] Consider adding a sentence that the "every cell" result is for the three specific backbones and two benchmarks under a fixed k chosen by a validation recall criterion, to avoid overgeneralization in the abstract.
  6. [Appendix D] The tanh activation failure on MuSiQue Gemma (F1 26.3) is marked with a dagger but not discussed. A sentence explaining this failure would help readers interpret the activation ablation.

Circularity Check

0 steps flagged

No circular derivation: the paper's claims are empirical comparisons against external baselines, with no self-citation chain or fit-to-target construction.

full rationale

I walked the claimed derivation chain and find no significant circularity. The paper makes empirical claims about two adapters (MaLoRA and MaRA) evaluated against external baselines (LoRA, TopLoRA, DoRA, AdaLoRA, BM25, BGE, Qwen3-Embedding, oracle evidence). The one hyperparameter that could look like it is fitted to the target, k, is explicitly set by a validation recall criterion rather than end-task F1: 'The number of retrieved segments k is fixed once per dataset by a recall criterion on the validation supporting-paragraph labels, not by end-task F1, and is then held fixed across all backbones and methods' (Section 5). MaRA is trained with binary supporting-segment labels and evaluated with Recall@k, which is a legitimate supervised retrieval target; its downstream F1 is then measured on held-out validation answer labels, so the retrieval component is not 'predicting' its own training signal. MaLoRA's modulator is defined by an independent equation (Eq. 3) and compared against LoRA, TopLoRA, and oracle-evidence rows, so the token-level gain is not forced by construction. The paper contains no author self-citations, no imported uniqueness theorem, and no ansatz justified solely by prior work of the same authors; references are external and the baselines are independent. The state-usage probe (Appendix G) and modulation-behavior analyses are post-hoc diagnostics, not load-bearing derivations. Concerns about missing per-seed standard deviations for the combined Table 4 rows, or about the backbone dependence of MaRA's recall, are statistical-robustness and generalization risks rather than circularity; the paper itself discloses the Gemma recall gap (Appendix K) and the staged-training limitation. Therefore no circular step meets the evidentiary bar required by the instructions.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new physical or theoretical entities are postulated; MaLoRA and MaRA are new model architectures, not new entities with independent falsifiable handles. The central claims rest on empirical comparisons plus several hand-set hyperparameters and domain assumptions about frozen-backbone hidden states.

free parameters (5)
  • top-k selection budget k = 4 (2WikiMultihopQA), 12 (MuSiQue)
    Chosen once per dataset by a validation recall criterion (>0.95 on Qwen/Llama) and held fixed across backbones; explicitly not tuned on end-task F1, but it is a hand-set hyperparameter that directly controls how much context MaRA exposes.
  • iterative selection pass count N = 2 (2WikiMultihopQA), 4 (MuSiQue)
    Matched by hand to dataset reasoning depth; no ablation on N is reported.
  • Mamba state dimensions = ds=16 (MaLoRA), ds=64 (MaRA)
    Hand-chosen; Appendix C cites ds=16 for the MaLoRA Mamba step, Appendix B sets MaRA Mamba state dim to 64.
  • LoRA rank r = 16
    Fixed across all methods and backbones; standard but arbitrary.
  • retrieval loss weights = positive BCE weight 8.0, pairwise margin weight 0.5 (margin 1.0), global loss weight gamma=0.5
    Hand-chosen in Appendix B; not swept in the paper.
axioms (4)
  • domain assumption The frozen backbone's first 16 layers encode enough query-conditioned evidence in each segment's hidden states for a small trained mixer to rank supporting paragraphs.
    Load-bearing for MaRA (Section 4 stage (i): 'The evidence information already resides in the backbone’s hidden states'). Supported empirically on two datasets but not guaranteed; Gemma's recall at k=12 (0.93) is below Qwen/Llama (0.97), showing backbone dependence (Appendix K).
  • domain assumption Gold supporting-paragraph labels in MuSiQue and 2WikiMultihopQA are a valid and sufficient supervision signal for the retrieval adapter.
    MaRA is trained entirely on these binary labels (Eq. 8); the paper notes distantly/self-supervised variants are out of scope (Limitations).
  • standard math Mamba selective state-space update rules (Eq. 4/6) behave as described in Gu & Dao with the chosen hyperparameters and frozen backbones.
    The paper relies on the correctness and stability of Mamba blocks; no formal verification is provided.
  • domain assumption Low-rank adaptation of frozen LLMs is a valid base method against which improvements are measured.
    All comparisons assume LoRA is the appropriate reference point; this is the paper's chosen baseline framework.

pith-pipeline@v1.3.0-alltime-deepseek · 19924 in / 14069 out tokens · 130192 ms · 2026-08-01T12:43:25.677160+00:00 · methodology

0 comments
read the original abstract

Low-rank adaptation introduces a static learned update applied identically to every input. The update provides task-level adaptation but does not explicitly represent token-level or instance-level state variation. A family of adapters is proposed that introduces selective state-space recurrence at two complementary granularities. At the token level, \textbf{MaLoRA} (Mamba-modulated low-rank adaptation) makes the adapter's scaling factor a dynamic input-dependent function with recurrent state across tokens, in contrast to the stateless modulators of prior work. At the context level, \textbf{MaRA} (Mamba Retrieval Adapter) tracks cross-segment state and selects the segments most relevant to the query, before the modulated language model generates its answer. Across three frozen backbones (Qwen-2.5-7B, Llama-3.1-8B, Gemma-2-9B) and two reasoning benchmarks (MuSiQue, 2WikiMultihopQA), the family improves reasoning accuracy on every cell of the $3{\times}2$ grid, by $+6.8$ F1 ($+10.5\%$ relative) on average and up to $+9.3$ F1 ($+18.2\%$ relative) on the hardest cell over the LoRA baseline, and the token-level gains carry to RULER QA-2 under length stress.

Figures

Figures reproduced from arXiv: 2607.19326 by Atahan Dokme, Larry Heck.

Figure 1
Figure 1. Figure 1: From LoRA to MaLoRA. (a) LoRA: a static low-rank update BAxt adds to the frozen output W xt. (b) TopLoRA (Li et al., 2025): a per-token diagonal modulator Λ(xt) = diag exp(RMSNorm(P xt)) is inserted between the LoRA factors, yielding ht = (W + B Λ(xt) A) xt. The modulator is stateless: Λ(xt) depends only on the current token. (c) MaLoRA: the per-token head is replaced by a selective state-space module (Ma… view at source ↗
Figure 2
Figure 2. Figure 2: Per-token modulation λ(xt) for MaLoRA on a MuSiQue example (Llama-3.1-8B, seed-43). Cell colour averages λ(xt) over 160 modulated projections; darker = stronger suppression. Question tokens are sup￾pressed most; supporting and distractor content retain higher λ. Per-region breakdown in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: MaRA architecture and data flow. Each (query, segment) pair (Q, Ci) is encoded by the first K=16 layers of the frozen backbone, yielding hidden states Hi . A learned attention pool reduces each Hi to a single embedding pi . A two-layer Mamba runs over the segment-level sequence (p1, . . . , pN ), producing recurrent states (s1, . . . , sN ). A local scoring head maps each recurrent state si to a relevance … view at source ↗
Figure 4
Figure 4. Figure 4: Two-level selective state-space adaptation around a frozen LM. Token-level adapter (MaLoRA, left) modulates the LoRA scaling factor per token; context-level adapter (MaRA, right) selects query-relevant input segments. Same frozen backbone, different granularities, different targets. seeds (42, 43, 44). All headline numbers in [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Where the modulator suppresses. Mean λ(xt) by projection type (left) and transformer layer (right), aggregated over the full MuSiQue validation set. Three cells: MaLoRA on Qwen-2.5-7B, Llama-3.1-8B, and Gemma-2-9B. non-supporting paragraph content), and context￾other. Two findings are robust. First, λ on supporting and distractor paragraphs is statistically iden￾tical in every cell (|∆| ≤ 0.002): the modul… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Latent-IM: Latent Interaction Management for Speech LLMs

    cs.CL 2026-07 conditional novelty 6.0

    A streaming residual-stream controller plus move-specific activation steering recovers selection and realization of five conversational moves in frozen speech LLMs, matching fine-tuning on human-move accuracy.

Reference graph

Works this paper leans on

48 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [1]

    Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , journal=

  2. [2]

    2024 , eprint=

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces , author=. 2024 , eprint=

    2024

  3. [3]

    Efficiently Modeling Long Sequences with Structured State Spaces , journal =

    Albert Gu and Karan Goel and Christopher R. Efficiently Modeling Long Sequences with Structured State Spaces , journal =. 2021 , url =. 2111.00396 , timestamp =

    Pith/arXiv arXiv 2021

  4. [4]

    arXiv preprint arXiv:2303.10512 , year=

    Adalora: Adaptive budget allocation for parameter-efficient fine-tuning , author=. arXiv preprint arXiv:2303.10512 , year=

    Pith/arXiv arXiv

  5. [5]

    Forty-first International Conference on Machine Learning , year=

    Dora: Weight-decomposed low-rank adaptation , author=. Forty-first International Conference on Machine Learning , year=

  6. [6]

    Zhao, Jiawei and Zhang, Zhenyu and Chen, Beidi and Wang, Zhangyang and Anandkumar, Anima and Tian, Yuandong , journal=

  7. [7]

    2024 , eprint=

    READ: Recurrent Adaptation of Large Transformers , author=. 2024 , eprint=

    2024

  8. [8]

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D , journal=

  9. [9]

    M u S i Q ue: Multihop Questions via Single-hop Question Composition

    Trivedi, Harsh and Balasubramanian, Niranjan and Khot, Tushar and Sabharwal, Ashish. M u S i Q ue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics. 2022. doi:10.1162/tacl_a_00475

    doi:10.1162/tacl_a_00475 2022

  10. [10]

    Dua, Dheeru and Wang, Yizhong and Dasigi, Pradeep and Stanovsky, Gabriel and Singh, Sameer and Gardner, Matt , journal=

  11. [11]

    arXiv preprint , year=

    Qwen2.5 Technical Report , author=. arXiv preprint , year=

  12. [12]

    Neural Computation , volume=

    Long Short-Term Memory , author=. Neural Computation , volume=

  13. [13]

    Learning Phrase Representations using

    Cho, Kyunghyun and van Merrienboer, Bart and Gulcehre, Caglar and Bahdanau, Dzmitry and Bougares, Fethi and Schwenk, Holger and Bengio, Yoshua , journal=. Learning Phrase Representations using

  14. [14]

    Synthesis of Parallel Algorithms , year=

    Prefix Sums and Their Applications , author=. Synthesis of Parallel Algorithms , year=

  15. [15]

    International Conference on Learning Representations (ICLR) , year=

    Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer , author=. International Conference on Learning Representations (ICLR) , year=

  16. [16]

    Journal of Machine Learning Research , volume=

    Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity , author=. Journal of Machine Learning Research , volume=

  17. [17]

    Parameter-Efficient Transfer Learning for

    Houlsby, Neil and Giurgiu, Andrei and Jastrz. Parameter-Efficient Transfer Learning for. International Conference on Machine Learning (ICML) , year=

  18. [18]

    Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=

    The Power of Scale for Parameter-Efficient Prompt Tuning , author=. Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=

  19. [19]

    Annual Meeting of the Association for Computational Linguistics (ACL) , year=

    Adaptive Attention Span in Transformers , author=. Annual Meeting of the Association for Computational Linguistics (ACL) , year=

  20. [20]

    Valipour, Mojtaba and Rezagholizadeh, Mehdi and Kobyzev, Ivan and Ghodsi, Ali , journal=

  21. [21]

    Renduchintala, Adithya and Konuk, Tugrul and Kuchaiev, Oleksii , journal=. Tied-

  22. [22]

    Kopiczko, Dawid Jan and Blankevoort, Tijmen and Nagel, Markus , journal=

  23. [23]

    Transformers are

    Dao, Tri and Gu, Albert , booktitle =. Transformers are. 2024 , editor =

    2024

  24. [24]

    arXiv preprint arXiv:2208.04933 , year=

    Simplified state space layers for sequence modeling , author=. arXiv preprint arXiv:2208.04933 , year=

    Pith/arXiv arXiv

  25. [25]

    Advances in Neural Information Processing Systems , volume=

    Hydralora: An asymmetric lora architecture for efficient fine-tuning , author=. Advances in Neural Information Processing Systems , volume=

  26. [26]

    M o LA : M o E L o RA with Layer-wise Expert Allocation

    Gao, Chongyang and Chen, Kezhen and Rao, Jinmeng and Liu, Ruibo and Sun, Baochen and Zhang, Yawen and Peng, Daiyi and Guo, Xiaoyuan and Subrahmanian, Vs. M o LA : M o E L o RA with Layer-wise Expert Allocation. Findings of the Association for Computational Linguistics: NAACL 2025. 2025. doi:10.18653/v1/2025.findings-naacl.284

    work page doi:10.18653/v1/2025.findings-naacl.284 2025

  27. [27]

    L o RAM o E : Alleviating World Knowledge Forgetting in Large Language Models via M o E -Style Plugin

    Dou, Shihan and Zhou, Enyu and Liu, Yan and Gao, Songyang and Shen, Wei and Xiong, Limao and Zhou, Yuhao and Wang, Xiao and Xi, Zhiheng and Fan, Xiaoran and Pu, Shiliang and Zhu, Jiang and Zheng, Rui and Gui, Tao and Zhang, Qi and Huang, Xuanjing. L o RAM o E : Alleviating World Knowledge Forgetting in Large Language Models via M o E -Style Plugin. Procee...

    doi:10.18653/v1/2024.acl-long 2024

  28. [28]

    L ong B ench: A Bilingual, Multitask Benchmark for Long Context Understanding

    Bai, Yushi and Lv, Xin and Zhang, Jiajie and Lyu, Hongchang and Tang, Jiankai and Huang, Zhidian and Du, Zhengxiao and Liu, Xiao and Zeng, Aohan and Hou, Lei and Dong, Yuxiao and Tang, Jie and Li, Juanzi. L ong B ench: A Bilingual, Multitask Benchmark for Long Context Understanding. Proceedings of the 62nd Annual Meeting of the Association for Computation...

    doi:10.18653/v1/2024.acl-long.172 2024

  29. [29]

    Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps

    Ho, Xanh and Duong Nguyen, Anh-Khoa and Sugawara, Saku and Aizawa, Akiko. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. Proceedings of the 28th International Conference on Computational Linguistics. 2020. doi:10.18653/v1/2020.coling-main.580

    doi:10.18653/v1/2020.coling-main.580 2020

  30. [30]

    Grattafiori, Aaron and others , journal=. The

  31. [31]

    arXiv preprint arXiv:2408.00118 , year=

    Pith/arXiv arXiv

  32. [32]

    2024 , eprint=

    RULER: What's the Real Context Size of Your Long-Context Language Models? , author=. 2024 , eprint=

    2024

  33. [33]

    Hayou, Soufiane and Ghosh, Nikhil and Yu, Bin , journal=

  34. [34]

    Anonymous , booktitle=. Gated

  35. [35]

    arXiv preprint arXiv:2505.06120 , year=

    Llms get lost in multi-turn conversation , author=. arXiv preprint arXiv:2505.06120 , year=

    Pith/arXiv arXiv

  36. [36]

    arXiv preprint arXiv:2505.15424 , year=

    Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models , author=. arXiv preprint arXiv:2505.15424 , year=

    arXiv

  37. [37]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  38. [38]

    LLM -Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models

    Hu, Zhiqiang and Wang, Lei and Lan, Yihuai and Xu, Wanyu and Lim, Ee-Peng and Bing, Lidong and Xu, Xing and Poria, Soujanya and Lee, Roy. LLM -Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.319

    doi:10.18653/v1/2023.emnlp-main.319 2023

  39. [39]

    2025 , eprint=

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models , author=. 2025 , eprint=

    2025

  40. [40]

    Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval , pages=

    Colbert: Efficient and effective passage search via contextualized late interaction over bert , author=. Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval , pages=

  41. [41]

    arXiv preprint arXiv:2112.09118 , year=

    Unsupervised dense information retrieval with contrastive learning , author=. arXiv preprint arXiv:2112.09118 , year=

    Pith/arXiv arXiv

  42. [42]

    arXiv preprint arXiv:2212.03533 , year=

    Text Embeddings by Weakly-Supervised Contrastive Pre-training , author=. arXiv preprint arXiv:2212.03533 , year=

    Pith/arXiv arXiv

  43. [43]

    The Probabilistic Relevance Framework: BM25 and Beyond , author=. Found. Trends Inf. Retr. , year=

  44. [44]

    Dense Passage Retrieval for Open-Domain Question Answering

    Karpukhin, Vladimir and Oguz, Barlas and Min, Sewon and Lewis, Patrick and Wu, Ledell and Edunov, Sergey and Chen, Danqi and Yih, Wen-tau. Dense Passage Retrieval for Open-Domain Question Answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.550

    doi:10.18653/v1/2020.emnlp-main.550 2020

  45. [45]

    2020 , eprint=

    Passage Re-ranking with BERT , author=. 2020 , eprint=

    2020

  46. [46]

    Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval , pages=

    C-pack: Packed resources for general chinese embeddings , author=. Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval , pages=

  47. [47]

    Advances in Neural Information Processing Systems , volume=

    Beyond higher rank: Token-wise input-output projections for efficient low-rank adaptation , author=. Advances in Neural Information Processing Systems , volume=

  48. [48]

    SSML o RA : Enhancing Low-Rank Adaptation with State Space Model

    Yu, Jiayang and Zhang, Yihang and Wang, Bin and Lin, Peiqin and Liu, YongKang and Feng, Shi. SSML o RA : Enhancing Low-Rank Adaptation with State Space Model. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025. doi:10.18653/v1...

    work page doi:10.18653/v1/2025.naacl-long.230 2025