Pith. sign in

REVIEW 3 major objections 6 minor 39 references

Temporal Information Retrieval via Time-Specifier Model Merging

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Uniform averaging of seven separately fine-tuned retrievers, one per time specifier, yields a single model that beats fine-tuning on temporal queries and matches or exceeds it on general queries.

desk verdict A practical, believable empirical result for temporal dense retrieval via simple parameter averaging, but the claimed mechanism is not actually tested. read the letter →

arxiv 2507.06782 v1 pith:VA6PB6WD submitted 2025-07-09 cs.IR cs.AIcs.LG

SeungYoon Han , Taeho Hwang , Sukmin Cho , Soyeong Jeong , Hoyun Song , Huije Lee , Jong C. Park This is my paper
classification cs.IRcs.AIcs.LG
keywords temporalinformationretrievalmodelmergingdensecatastrophicforgettingtimespecifiersQAretrieverfine-tuningparameteraveraging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that catastrophic forgetting in temporal retrieval can be avoided without routing, ensembling, or extra modules: train one dense retriever per time specifier (e.g., "in," "after," "between") on the appropriate TimeQA subset, then simply average the seven checkpoints. This Time-Specifier Model Merging (TSM) produces a unified retriever whose temporal Recall and nDCG are the highest among all compared methods, while non-temporal performance on NQ and MS MARCO is preserved or improved. A sympathetic reader would care because this is a parameter-free merging step that converts otherwise conflicting temporal and general retrieval skills into one model.

What carries the argument

The central object is the merged parameter vector $\theta_{\text{merged}} = \frac{1}{k}\sum_{i=1}^k \theta_i$, the uniform arithmetic mean of the seven independently fine-tuned retriever checkpoints (Eq. 6). The load-bearing mechanism is that this low-magnitude weight averaging preserves each specifier-specific retriever's skill, so the merged encoder can match temporal relevance signals across all seven specifier types while keeping the general-domain representations of the vanilla encoder largely intact.

What would settle it

Check whether the merged TSM model, on the TimeQA test set, attains Recall@20 at least as high as the single-specifier retriever that was trained only on that specifier's data; if, for any specifier (e.g., "between"), the merged model underperforms that one specialist, then the averaging is not preserving that specialist's skill and the reported gains must be attributed to some other effect.

Watch

Extended reading notes

Core claim

TSM establishes that averaging the parameters of time-specifier-specialized dense retrievers yields a single retriever that simultaneously handles temporally constrained and non-temporal queries. The merged model inherits each specialist's temporal expertise — it attains the highest Recall@20 and nDCG@20 on TimeQA and Nobel Prize among Contriever- and DPR-based baselines — while avoiding the non-temporal degradation caused by full fine-tuning. The method's core evidence is that the merged model outperforms ensembling and routing, achieves the smallest total parameter weight change, and that Recall@20 on both temporal and non-temporal datasets rises monotonically as more specifier-specific retrievers are merged.

Load-bearing premise

TSM assumes that simply averaging the fine-tuned parameters of seven independently trained retrievers preserves each specialist's expertise, meaning the fine-tuned checkpoints all lie in a single loss basin where uniform interpolation stays meaningful.

Editorial extensions

If this is right

  • Temporal retrieval can be improved by training on labeled specifier subsets and merging, without needing a separate query router or any inference-time ensemble of multiple models.
  • The monotone improvement in Figure 2 implies that more fine-grained specifier specialization, up to and beyond seven categories, would further raise both temporal and non-temporal Recall@20.
  • Because TSM is architecture-agnostic, the same recipe transfers to other dense or even cross-encoder backbones wherever time-specifier-labeled data exists.
  • TSM's minimal weight-change magnitude suggests it can serve as a general recipe for mitigating catastrophic forgetting when integrating domain-specialist checkpoints.
  • The merged model can be deployed with the same memory footprint and latency as a single dense retriever, making temporal awareness practical in retrieval-augmented generation pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is genuine expertise preservation in a shared loss basin, then TSM should also work when the specifier set is expanded to include interval, relative, and underspecified temporal expressions — the paper's own limitation section names this as future work.
  • A controlled falsification: if the merged model does not match or exceed each single-specifier retriever on that specifier's own TimeQA subset, then the average is not actually preserving specialist skill, and gains must come from some other averaging artifact.
  • The routing baseline's weakness is its dependence on a perfect query classifier; TSM effectively sidesteps classification, so a comparison against an oracle router (one that always sends temporal queries to the temporally fine-tuned model) would quantify how much of TSM's advantage is purely from avoiding misclassification.
  • Since TSM already beats ensembling, comparing it against mixture-of-experts routing with shared experts could clarify whether specialization is better stored in parameters or in activations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Time-Specifier Model Merging (TSM): partition TimeQA queries by seven explicit time specifiers, fine-tune a separate dense retriever on each partition, and then average the seven parameter sets into a single unified retriever (Eq. 6). Experiments on TimeQA and Nobel Prize (temporal) and NQ and MS MARCO (non-temporal) compare TSM with vanilla Contriever/DPR, full fine-tuning, fine-tuning with regularization, LoRA, routing, and ensembling. The paper reports that TSM improves temporal retrieval while largely preserving non-temporal performance, and it includes analyses of the merging curve, per-specifier recall, parameter weight-change magnitude, and qualitative case studies.

Significance. If the central claim holds, TSM is a simple and practical way to obtain temporal retrieval gains while mitigating catastrophic forgetting, with no extra inference cost compared with routing or ensembling. The evaluation has real strengths: it includes an out-of-domain temporal dataset (Nobel Prize), compares against a broad set of relevant baselines including routing and ensembling, and the code is promised publicly. The small magnitude of parameter updates in TSM is an interesting mechanistic observation. However, the mechanism that gives TSM its claimed advantage is not directly verified, and the headline claim in the abstract is stronger than Table 2 supports; both issues are fixable with additional experiments and careful rewriting.

major comments (3)
  1. [§3.2.3, §6.2, Eq. (6)] The paper claims that TSM 'inherits the strengths of each specialist model,' but it never directly verifies that each specifier-specific specialist's expertise survives uniform parameter averaging. Figure 3 (left) compares TSM only with Contriever, FT+Reg, and Ensembling for each specifier; it does not plot the corresponding single-specifier retriever, and no leave-one-specifier-out merge is reported. The monotone improvement in Figure 2 is also consistent with a simpler explanation: averaging seven models trained on disjoint data subsets approximates a single model fine-tuned on all temporal data with a smaller effective update, in which case the seven-way specifier decomposition is not load-bearing. I recommend adding (a) per-specifier comparison of TSM against the individual specialist, (b) a leave-one-specifier-out merging ablation, and (c) ideally a random-partition control in which seven models are trained on randomly split subsets of the same TimeQA data and averaged. If the random-partition control matches TSM, the contribution should be reframed as model-averaging regularization rather than specifier-specific expertise.
  2. [Abstract, §5, Table 2] The abstract's 'consistently outperforming other baseline methods' is not supported by Table 2. On MS MARCO with Contriever, TSM trails vanilla Contriever on Recall@20 (44.28 vs. 45.99) and nDCG@20 (22.92 vs. 23.20); on NQ with DPR, vanilla DPR is better on all four reported metrics (e.g., nDCG@20 52.67 vs. 43.85). The body of §5 acknowledges some of these exceptions, but the abstract and introduction do not. In addition, no error bars, repeated runs, or significance tests are reported anywhere; small reported advantages such as MS MARCO Contriever Recall@5 (25.26 vs. 25.24) cannot be distinguished from noise. Please temper the claim to 'improves average performance across the evaluated datasets' and add variance estimates or significance tests.
  3. [§3.2.1, Appendix A.2] The TimeQA augmentation procedure is undocumented. The paper increases the after, before, in early, and in late partitions using 'official TimeQA data processing scrips and annotation labels,' but it provides no description of how new queries were generated or selected, and no release of the augmented data is mentioned. This matters because all fine-tuned baselines are trained on the augmented set, so comparisons against published numbers on the original TimeQA may not be direct. There is also a numerical discrepancy: the main text says the original training set has 25,064 instances and the dev set 5,348, while Table 4 reports 24,981 and 5,238, respectively. Please document the augmentation, resolve the discrepancy, and make the augmented data available.
minor comments (6)
  1. [§3] The sentence 'We define the temporal and non-temporal retrieval problem and introduce out method' contains a typo: 'out' should be 'our'.
  2. [§3.2.1] 'scrips' should be 'scripts' in the description of the official TimeQA data processing scripts.
  3. [Abstract] The phrase 'merges them in to a unified model' should be 'merges them into a unified model.'
  4. [§4.2] 'We employContriever' is missing a space; it should read 'We employ Contriever.'
  5. [References] Several reference entries have typos that should be corrected: 'forgeting' in Goodfellow et al., 'Time-sensitve' in Wu et al., and 'V enice' in Triki et al.
  6. [Figure 2] In each panel of Figure 2, the left and right vertical axes have identical labels but different scales; consider unifying the axes or labeling them more explicitly to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TSM's reported gains are measured on held-out data, and the merging rule is not fitted to the target metrics.

full rationale

The paper's central derivation chain is empirical rather than definitional. TSM fine-tunes seven retriever checkpoints on disjoint TimeQA subsets grouped by time specifier (Section 3.2.1), then merges them by uniform parameter averaging (Eq. 6, Section 3.2.3). The claimed outcomes—improved Recall/nDCG on temporal datasets and preserved performance on non-temporal datasets—are evaluated on held-out TimeQA, Nobel Prize, NQ, and MS MARCO test sets, none of which are used to fit the merge weights or any parameter of the final model. The merging operation is a fixed arithmetic mean of checkpoints, not an optimization whose objective includes the reported evaluation metrics. The statement that TSM 'inherits the strengths of each specialist model' (Section 6.2) is an interpretive claim about why parameter averaging works, not an equation that defines the result in terms of the target numbers; no derivation in the paper equates the merged model's performance with the training data or with the citations. The model-merging citations (Alexandrov et al., 2024; Yang et al., 2024; Xiao et al., 2024) are external prior work, and the paper does not rely on a self-citation chain or on an author-imported uniqueness theorem to force its choice. The limitations section explicitly concedes that the method depends on labeled specifier data and that only simple parameter averaging was tested, which further indicates that the authors do not present the approach as a mathematically forced consequence of prior results. The skeptic's concern—that per-specifier skill retention inside the merged model is not directly verified—is a legitimate validity/ablation concern, but it is not a circularity: the aggregate improvement is still an independent empirical finding. Accordingly, no circular step can be quoted and exhibited from the text, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fit to the target results; the only hyperparameters (learning rate, temperature, epochs) come from the Contriever training code. The key unstated inputs are that parameter averaging preserves specialist skills, that the seven specifiers cover temporal query types, and that the augmented data is reliable.

assumptions (3)
  • domain assumption Fine-tuned retrievers that share a base initialization can be merged by uniform parameter averaging without losing task-specific skills.
    Invoked in Eq. (6) (Section 3.2.3); supported by citations to Alexandrov et al. (2024) and Yang et al. (2024), not by a formal guarantee or direct per-specifier verification.
  • domain assumption The seven TimeQA time-specifier categories cover the space of explicit temporal constraints relevant to the evaluation.
    Section 3.1 defines S = {before, between, ...}; the method trains one model per category. The Limitations section acknowledges that other specifiers or implicit temporal expressions may not be covered.
  • ad hoc to paper The augmented TimeQA training data for rare specifiers (after, before, in early, in late) is a faithful sample of temporal queries.
    Section 3.2.1 reports augmentation via official scripts and annotation labels but does not describe the procedure; the quality of the merged model depends on this data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Temporal Information Retrieval via Time-Specifier Model Merging." pith.science (2026). https://pith.science/paper/VA6PB6WD

@misc{pith2026250706782,
  author       = {Pith},
  title        = {Pith review of: Temporal Information Retrieval via Time-Specifier Model Merging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VA6PB6WD}},
  note         = {Machine review of arXiv:2507.06782}
}
read the original abstract

The rapid expansion of digital information and knowledge across structured and unstructured sources has heightened the importance of Information Retrieval (IR). While dense retrieval methods have substantially improved semantic matching for general queries, they consistently underperform on queries with explicit temporal constraints--often those containing numerical expressions and time specifiers such as ``in 2015.'' Existing approaches to Temporal Information Retrieval (TIR) improve temporal reasoning but often suffer from catastrophic forgetting, leading to reduced performance on non-temporal queries. To address this, we propose Time-Specifier Model Merging (TSM), a novel method that enhances temporal retrieval while preserving accuracy on non-temporal queries. TSM trains specialized retrievers for individual time specifiers and merges them in to a unified model, enabling precise handling of temporal constraints without compromising non-temporal retrieval. Extensive experiments on both temporal and non-temporal datasets demonstrate that TSM significantly improves performance on temporally constrained queries while maintaining strong results on non-temporal queries, consistently outperforming other baseline methods. Our code is available at https://github.com/seungyoonee/TSM .

Figures

Figures reproduced from arXiv: 2507.06782 by the authors.

Figure 1
Figure 1. Recall@20 performance of vanilla Contriever, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Recall@20 on temporal datasets (TimeQA, Nobel Prize; left) and non-temporal (NQ, MS MARCO; right) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Left: Recall@20 for each time specifier on the TimeQA test set, comparing vanilla Contriever, FT + Reg, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Recall@20 for each time specifier on the [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Heatmap of normalized query document similarity scores for the query “ [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 11 canonical work pages

  1. [1]

    Abdelrahman Abdallah, Bhawna Piryani, Jonas Wallat, Avishek Anand, and Adam Jatowt. 2025. https://arxiv.org/abs/2502.21024 Tempretriever: Fusion-based temporal dense passage retrieval for time-sensitive questions . Preprint, arXiv:2502.21024

    arXiv 2025

  2. [2]

    Vechev, and Kristina Toutanova

    Anton Alexandrov, Veselin Raychev, Mark Mueller, Ce Zhang, Martin T. Vechev, and Kristina Toutanova. 2024. https://aclanthology.org/2024.findings-emnlp.1000 Mitigating catastrophic forgetting in language transfer via model merging . In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024 , pages ...

    work page 2024

  3. [3]

    James F. Allen. 1983. https://doi.org/10.1145/182.358434 Maintaining knowledge about temporal intervals . Commun. ACM, 26(11):832–843

    doi:10.1145/182.358434 1983

  4. [4]

    Omar Alonso, Jannik Str \" o tgen, Ricardo Baeza - Yates, and Michael Gertz. 2011. https://ceur-ws.org/Vol-707/TWAW2011-paper1.pdf Temporal information retrieval: Challenges and opportunities . In WWW2011 Workshop on Linked Data on the Web, Hyderabad, India, March 29, 2011 , volume 813 of CEUR Workshop Proceedings , pages 1--8. CEUR-WS.org

    work page 2011

  5. [5]

    Wenhu Chen, Xinyi Wang, and William Yang Wang. 2021. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/1f0e3dad99908345f7439f8ffabdffc4-Abstract-round2.html A dataset for answering time-sensitive questions . In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021...

    work page 2021

  6. [6]

    Cole, Aditi Chaudhary, Bhuwan Dhingra, and Partha Talukdar

    Jeremy R. Cole, Aditi Chaudhary, Bhuwan Dhingra, and Partha Talukdar. 2023. https://doi.org/10.18653/v1/2023.eacl-main.222 Salient span masking for temporal understanding . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3052--3060, Dubrovnik, Croatia. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.eacl-main.222 2023

  7. [8]

    Jetcheva

    Anoushka Gade and Jorjeta G. Jetcheva. 2024. https://doi.org/10.48550/ARXIV.2401.13222 It's about time: Incorporating temporality in retrieval augmented language models . CoRR, abs/2401.13222

    arXiv doi:10.48550/arxiv.2401.13222 2024

  8. [9]

    Suganthan

    Mudasir Ahmad Ganaie, Minghui Hu, Mohammad Tanveer, and Ponnuthurai N. Suganthan. 2021. https://arxiv.org/abs/2104.02395 Ensemble deep learning: A review . CoRR, abs/2104.02395

    arXiv 2021

Show all 39 references
  1. [10]

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee’s mergekit: A toolkit for merging large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natura...

    2024

  2. [11]

    Goodfellow, Mehdi Mirza, Xia Da, Aaron C

    Ian J. Goodfellow, Mehdi Mirza, Xia Da, Aaron C. Courville, and Yoshua Bengio. 2014. http://arxiv.org/abs/1312.6211 An empirical investigation of catastrophic forgeting in gradient-based neural networks . In 2nd International Conference on Learning Representations, ICLR 2014, ...

    2014 arXiv

  3. [12]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . CoRR, abs/2106.09685

    2021 arXiv

  4. [13]

    Domor Mienye Ibomoiye and Yanxia Sun. 2022. https://doi.org/10.1109/ACCESS.2022.3207287 A survey of ensemble learning: Concepts, algorithms, applications, and prospects . IEEE Access , 10:99129--99149

    2022

  5. [14]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. https://openreview.net/forum?id=jKN1pXi7b0 Unsupervised dense information retrieval with contrastive learning . Trans. Mach. Learn. Res., 2022

    2022

  6. [15]

    Dong-Hwan Jang, Sangdoo Yun, and Dongyoon Han. 2024. https://arxiv.org/abs/2403.19522 Model stock: All we need is just a few fine-tuned models . Preprint, arXiv:2403.19522

    2024 arXiv

  7. [16]

    Nattiya Kanhabua and Avishek Anand. 2016. https://doi.org/10.1145/2911451.2914805 Temporal information retrieval . In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '16, page 1235–1238, New York, NY, USA. ...

    2016

  8. [17]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen - tau Yih. 2020. https://doi.org/10.18653/V1/2020.EMNLP-MAIN.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on...

    2020 doi

  9. [18]

    Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. https://openreview.net/forum?id=HklBjCEKvH Generalization through memorization: Nearest neighbor language models . In 8th International Conference on Learning Representations, ICLR 2020, Addis...

    2020

  10. [19]

    Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A

    James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska - Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2016. https://arxiv.org/abs/161...

    2016 arXiv

  11. [20]

    Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming - Wei Chang, Andrew M

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming - Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and...

    2019 doi

  12. [21]

    Yunshi Lan, Gaole He, Jinhao Jiang, Jing Jiang, Wayne Xin Zhao, and Ji - Rong Wen. 2021. https://arxiv.org/abs/2105.11644 A survey on complex knowledge base question answering: Methods, challenges and solutions . CoRR, abs/2105.11644

    2021 arXiv

  13. [22]

    Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, and Kyle Lo. 2023. https://doi.org/10.48550/ARXIV.2311.09765 Back to basics: A simple recipe for improving out-of-domain retrieval in dense encoders . CoRR, abs/2311.09765

    2023 doi arXiv

  14. [23]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e...

    2020

  15. [24]

    Mingda Li, Xinyu Li, Yifan Chen, Wenfeng Xuan, and Weinan Zhang. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.288 Unraveling and mitigating retriever inconsistencies in retrieval-augmented large language models . In Findings of the Association for Computational Linguist...

    2024 doi

  16. [25]

    Zhizhong Li and Derek Hoiem. 2016. https://doi.org/10.1007/978-3-319-46493-0\_37 Learning without forgetting . In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV , volume 9908 of Lecture Notes in Com...

    2016 doi

  17. [26]

    Ilya Loshchilov and Frank Hutter. 2017. https://arxiv.org/abs/1711.05101 Fixing weight decay regularization in adam . CoRR, abs/1711.05101

    2017 arXiv

  18. [27]

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. https://doi.org/10.48550/ARXIV.2308.08747 An empirical study of catastrophic forgetting in large language models during continual fine-tuning . CoRR, abs/2308.08747

    2023 doi arXiv

  19. [28]

    Ammar Mohammed and Rania Kora. 2023. https://doi.org/10.1016/J.JKSUCI.2023.01.014 A comprehensive review on ensemble deep learning: Opportunities and challenges . J. King Saud Univ. Comput. Inf. Sci., 35(2):757--774

    2023 doi

  20. [29]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. https://arxiv.org/abs/1611.09268 MS MARCO: A human generated machine reading comprehension dataset . CoRR, abs/1611.09268

    2016 arXiv

  21. [30]

    Rosin, Ido Guy, and Kira Radinsky

    Guy D. Rosin, Ido Guy, and Kira Radinsky. 2021. https://arxiv.org/abs/2110.06366 Time masking for temporal language models . CoRR, abs/2110.06366

    2021 arXiv

  22. [31]

    Nandan Thakur, Nils Reimers, Andreas R \" u ckl \' e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/65b9eea6e1cc6bb9f0cd2a47751a186f-Abstract-round2.html BEIR: A heterogeneous benchmark for zero-shot evaluati...

    2021

  23. [32]

    Blaschko, and Tinne Tuytelaars

    Amal Rannen Triki, Rahaf Aljundi, Matthew B. Blaschko, and Tinne Tuytelaars. 2017. https://doi.org/10.1109/ICCV.2017.148 Encoder based lifelong learning . In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017 , pages 1329--1337. IEE...

    2017 doi

  24. [33]

    Denny Vrande c i\' c and Markus Kr\" o tzsch. 2014. https://doi.org/10.1145/2629489 Wikidata: a free collaborative knowledgebase . Commun. ACM, 57(10):78–85

    2014 doi

  25. [34]

    Jiexin Wang, Adam Jatowt, Masatoshi Yoshikawa, and Yi Cai. 2023. Bitimebert: Extending pre-trained language representations with bi-temporal information. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 812--821

    2023

  26. [35]

    Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, and Bing Xiang. 2019. https://doi.org/10.18653/v1/D19-1599 Multi-passage BERT : A globally normalized BERT model for open-domain question answering . In Proceedings of the 2019 Conference on Empirical Methods in Natural La...

    2019 doi

  27. [36]

    Feifan Wu, Lingyuan Liu, Wentao He, Ziqi Liu, Zhiqiang Zhang, Haofen Wang, and Meng Wang. 2024. https://doi.org/10.1145/3627673.3679800 Time-sensitve retrieval-augmented generation for question answering . In Proceedings of the 33rd ACM International Conference on Information ...

    2024

  28. [37]

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Xingrun Xing. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.145 Lm-cocktail: Resilient tuning of language models via model merging . In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and v...

    2024 doi

  29. [38]

    Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. 2024. https://doi.org/10.48550/ARXIV.2408.07666 Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities . CoRR, abs/2408.07666

    2024 doi arXiv

  30. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  31. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.