Pith. sign in

REVIEW 3 major objections 3 minor 20 references

Catastrophic forgetting in low-rank continual adaptation is caused by imbalanced singular-value spectra, and forcing equal-strength components plus gradient-orthogonal update directions largely eliminates it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 05:57 UTC pith:LMYCEZ74

load-bearing objection The balanced sUV^T parameterization is genuinely new and the gains are real, but the causal claim is confounded by update magnitude; the method still deserves peer review. the 3 major comments →

arxiv 2602.00722 v2 pith:LMYCEZ74 submitted 2026-01-31 cs.LG

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

Hao Gu , Mao-Lin Luo , Zi-Hao Zhou , Han-Chen Zhang , Min-Ling Zhang , Tong Wei This is my paper
classification cs.LG
keywords continual learningcatastrophic forgettinglow-rank adaptationsingular value spectrumenergy-balanced updatesStiefel manifold optimizationgradient orthogonalityvision-language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish a causal account of catastrophic forgetting in continual learning with low-rank adapters: when a task update is a low-rank matrix, its singular values are highly imbalanced, so a handful of components carry most of the adaptation energy, and it is exactly those dominant directions that later tasks overwrite. If that is right, forgetting is a structural consequence of imbalanced updates rather than merely an optimization failure, and the fix is to regulate the internal shape of each update instead of only defending old knowledge. The proposed method, EBLoRA (Energy-Balanced Low-Rank Adaptation), rewrites every task update as a scalar magnitude times two orthonormal bases, making all components carry equal energy, and requires the update basis to be orthogonal to a stored set of previous-task gradient directions. Across two vision-language continual learning benchmarks this reduces backward forgetting from around −15 to −2 points and raises final accuracy by more than ten points, without sacrificing the model's ability to keep learning new tasks. The reason to care is that the recipe is a design principle — balance the spectrum, orthogonalize against past gradients — rather than replay buffers, stored data, or task-specific architectures.

Core claim

Central claim: spectral imbalance, not interference with earlier updates per se, drives forgetting in low-rank continual adaptation. LoRA updates show long-tailed singular value spectra whose variance grows during training, and merging experiments show that flattening singular values toward their mean, with singular vectors fixed, reduces cross-task interference. The paper concludes that a balanced update is both less disruptive to prior knowledge and less vulnerable to later overwriting. The design: factor each update as ΔW_t = s_t U_t V_t^T (scalar magnitude, orthonormal bases), so all r singular values are equal; enforce G^T_{t-1} U_t = 0 to avoid previously sensitive directions. A projec

What carries the argument

The load-bearing object is the factorization ΔW_t = s_t U_t V_t^T — a scalar magnitude times two orthonormal bases — which decouples how much adaptation happens from which directions it takes, and makes the update's singular values exactly equal by construction, i.e., energy-balanced. Optimization happens on the restricted Stiefel manifold M_t = {U : U^T U = I_r, G_{t-1}^T U = 0}, where the two constraints are orthonormality of the update basis and orthogonality against a stored subspace G_{t-1} of gradient directions gathered from earlier tasks. Two geometric facts carry the method: the tangent-space projection and the whitening retraction are each the unique Frobenius-norm-closest map onto

Load-bearing premise

The argument's load-bearing premise is that the experiments isolating balance actually isolate balance — replacing singular values with their mean also lowers the update's total energy whenever the spectrum is skewed, so the measured drop in interference could come from smaller updates rather than from equal-strength components; a second pillar is that the stored gradient subspace, whose size and sampling the paper leaves unspecified, faithfully captures every direction previ

What would settle it

Merge LoRA adapters with singular values smoothed to the mean, then rescale the update so its total energy (sum of squared singular values) exactly matches the unsmoothed version: if the rescaling erases the interference benefit, then reduced energy, not balance, was causing the improvement. A second check: run the full method with the gradient-orthogonality constraint disabled while keeping the balanced factorization; if backward transfer degrades sharply, the orthogonality constraint, not the spectral balance, is doing the work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If spectral imbalance is the cause, then balancing the singular values of an update should reduce forgetting for any low-rank continual method, regardless of whether it uses replay, regularization, or architecture changes.
  • The stability–plasticity trade-off is looser than commonly believed: backward transfer goes from −15.4 to −2.0 on UCIT and from −11.2 to −0.7 on MLLM-DCL while final accuracy rises by more than ten points on both benchmarks.
  • The gains hold across heterogeneous task types — image classification, captioning, visual question answering, and multiple-choice — with no access to past data, so the principle applies where storing or replaying old samples is infeasible.
  • Forward transfer also improves (34.6 vs 26.8 on UCIT), indicating that balanced components generalize better to tasks never seen during training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the spectral story is right, the balancing recipe should transfer beyond continual learning to other low-rank parameter-efficient regimes — model merging, task arithmetic, and sequential fine-tuning in domains other than vision-language — wherever imbalanced components would produce the same overwriting pattern.
  • A cheap probe would be to keep the sU V^T factorization but drop the gradient-orthogonality constraint: if most of the forgetting reduction survives, then balance alone carries the effect and the geometric machinery matters only for the residual gap.
  • The paper's equal-norm comparison contrasts two end-to-end training paradigms (EBO vs standard LoRA); a tighter test would randomize only the singular values of otherwise identical adapters while holding total energy fixed, to separate 'balance helps' from 'smaller perturbation helps.'

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper argues that catastrophic forgetting in low-rank continual adaptation is caused by imbalanced singular value spectra of LoRA updates, and proposes EBLoRA, which factorizes each task update as ΔW_t = s_t U_t V_t^T with orthonormal U_t, V_t and enforces G^T_{t-1} U_t = 0 to avoid previously sensitive gradient directions. The constrained problem is solved by projected first-order optimization on a restricted Stiefel manifold. The authors report strong gains on UCIT and MLLM-DCL, with ablations (EBO, GO, IL) attributed to energy balance, gradient orthogonality, and depth-aware initialization. Theoretical results (Props. 3.1–3.2) prove optimality of the proposed tangent projection and retraction.

Significance. If the causal claim is established, the paper offers a simple and actionable design principle: balanced low-rank updates plus gradient orthogonality are sufficient to reduce both backward and forward forgetting. The theoretical parts are sound: Props. 3.1–3.2 are correct, and the appendix proofs are valid. The paper also ships detailed per-step tables (Tables 4–6), from which the headline MFN/BWT/FWT values recompute, and provides a public code link. The main empirical claim, however, currently rests on experiments that conflate spectral balance with update magnitude; the causal interpretation therefore needs additional controls before the paper's central message can be accepted.

major comments (3)
  1. [§2.2 and §4.2 (Fig. 2b and Fig. 4 left)] The smoothing experiment replaces each singular value σ_i with (1−α)σ_i + ασ̄, where σ̄ = (1/r)Σ_j σ_j. This changes two quantities at once: the spectrum becomes flatter, and the Frobenius norm shrinks, since ‖ΔW_smooth‖_F^2 = rσ̄² ≤ Σ_i σ_i², with strict inequality whenever the spectrum is not flat. The increased NAI after smoothing could therefore be caused by smaller update energy rather than by balance. No control is reported that renormalizes the smoothed update to the original norm or that applies a uniform scaling to the unsmoothed update. Please add such a magnitude-matched control (e.g., scale the smoothed adapter back to the original Frobenius norm before merging, or compare against c·ΔW for c<1). This is load-bearing because the abstract and Section 2.2 attribute reduced interference specifically to spectral balance.
  2. [§4.2 and Table 3 (EBO ablation)] The ablation isolating 'energy-balanced optimization' (EBO) removes gradient orthogonality and depth-aware initialization, but it still differs from LoRA-FT in both the factorization (s_t U_t V_t^T) and the learned magnitude s_t. The paper does not report the final Frobenius norms of EBO updates versus LoRA-FT updates, so the improvement of EBO over LoRA-FT (70.2 MFN vs 61.4) could reflect a different update scale rather than spectral balance. The statement in §4.2 that Fig. 4 right uses 'equal-norm perturbations' is about the injected noise, not about matching the norms of the compared target updates. Please provide a norm-matched comparison — for example, rescale each LoRA-FT update per layer to the same Frobenius norm as the EBO update (or vice versa) and report the resulting metrics.
  3. [Algorithm 2 (lines 4–8) and Appendix C] The entire gradient-orthogonality mechanism depends on the snapshot G_t and the stored subspace G_{t-1}. Algorithm 2 says only 'Sample a small subset of mini-batches' to form G_t, without specifying the number of mini-batches, the number of gradient steps, or whether the snapshot is taken from the initial model or after some training. Appendix C fixes the energy threshold ε=0.95 but gives no sensitivity analysis. Since Eq. (4), the initialization in lines 7–8, and the retraction in Algorithm 1 all rely on this subspace estimate, the method is not fully reproducible and the robustness of the reported gains to this choice is unknown. Please specify the snapshot protocol and report results for a range of ε (and snapshot sizes).
minor comments (3)
  1. [§4.1] Typo: 'LLaV A' should be 'LLaVA'.
  2. [Appendix A.2] The proof of Prop. 3.2 assumes G^T G = I (used in the step G^T G G^T = G^T). This holds for the GPM construction, but the paper should state explicitly that G always has orthonormal columns, since the projector P^⊥_G = I − G G^T is only an orthogonal projector under that condition.
  3. [Fig. 5 caption] The reference 'Tab. 4a and 6' is slightly confusing; consider clarifying which rows of Table 4 correspond to LoRA-FT.

Circularity Check

1 steps flagged

No circular derivation in the core optimization theory; the motivating causal evidence is confounded, and the one same-group baseline is not load-bearing.

specific steps
  1. other [§4.2, Fig. 4 left; see also §2.2, Fig. 2b]
    "each singular value σ_i is replaced by (1−α)σ_i + ασ̄, where σ̄ is the mean singular value. We observe that increasing the smoothing ratio consistently improves performance across all target tasks, indicating that adapters with balanced singular value spectra exhibit lower interference when merged with existing knowledge."

    The smoothing operation is a convex contraction toward the mean: for a non-flat spectrum, ||ΔW_smooth||_F^2 = r σ̄^2 < Σ_i σ_i^2 = ||ΔW||_F^2. Thus the 'balanced' condition also has strictly smaller Frobenius norm by construction. Since no norm-matched control is reported, the experiment cannot separate spectral balance from update magnitude; the motivating causal premise is therefore not identified. This is an experimental-validity confound rather than an equation-to-equation circularity in the optimization derivation.

full rationale

Walking the claimed derivation chain: Eq. (3)-(7) define the EBLoRA parameterization and the restricted Stiefel optimization problem; Propositions 3.1-3.2 prove optimality of the tangent-space projection and retraction. These are self-contained mathematical results and do not presuppose the forgetting claim. The main empirical comparisons (Tables 1-3) are external benchmarks. The only same-group citation, KeepLoRA (Luo et al., 2026), is used as a baseline and its numbers are reused without re-running; it is shared lineage, not a load-bearing input to the derivation. The §2.2/§4.2 smoothing experiments are confounded: smoothing shrinks the update norm, so the causal attribution to balance is not isolated. That is a validity concern, not a circularity, because the outcome is not equivalent to the input by construction. Score 2 reflects the minor same-group baseline reuse and the un-isolated confound, not a circular derivation.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The novel structural choice is the equal-singular-value parameterization; everything else (GPM threshold, LiNeS scaling, gradient snapshot protocol) is borrowed. The causal claim additionally rests on the interpretive 'knowledge components' layer, which is not independently evidenced. No new entities are postulated. The most under-audited item is the GPM gradient snapshot: its quality determines both the initialization and the constraint that drives the reported gains, yet its construction is described only as 'a small subset of mini-batches.'

free parameters (3)
  • depth-aware scale initialization s_min, s_max = s_min = 0.002, s_max = 0.010
    Initial magnitude of s_t per layer (Eq. 17, App. B.1), chosen by hand following LiNeS (Wang et al., 2025). The final update magnitude depends on these values; no sensitivity analysis is reported.
  • GPM gradient-energy threshold ε = 0.95
    Controls how many past-task gradient directions are stored in G_{t-1} (App. C) and hence how aggressive the orthogonality constraint (Eq. 4) is. Borrowed unchanged from GPM (Saha et al., 2021).
  • rank r and learning rates = rank r not stated in main text; LR 2e-5 (MLLM-DCL), 2e-4 / 1e-4 (UCIT)
    Standard hyperparameters, but the balance-vs-capacity tradeoff of the flat-spectrum parameterization is tied to r, which is never swept.
axioms (4)
  • domain assumption Stored gradient basis satisfies G^T G = I
    Invoked in the proof of Prop. 3.2 ('where we used G^T G = I', App. A.2). Guaranteed only if the GPM orthonormal basis update (App. C) is performed exactly as specified.
  • domain assumption Rank-one LoRA components 'encode a distinct input–output interaction pattern'
    The knowledge-component premise (Eq. 1, §2.1) that gives semantic meaning to singular vectors; it underpins the causal narrative but is not independently tested.
  • domain assumption A small gradient snapshot G_t faithfully represents task t's sensitive subspace
    Algorithm 2, lines 4–8: both the initial U,V bases and the GPM subspace update come from G_t; the snapshot size/composition is unspecified and the method's stability depends on it.
  • domain assumption Deeper layers benefit from larger update scales (LiNeS finding)
    Justifies the depth-aware s_t initialization (§3.2, App. B.1). Adopted from cited prior work; not validated for this setting.

pith-pipeline@v1.3.0-alltime-deepseek · 21852 in / 30022 out tokens · 317739 ms · 2026-08-03T05:57:10.864114+00:00 · methodology

0 comments
read the original abstract

Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge. Most existing approaches treat continual learning as avoiding interference with past updates, rather than considering what properties make the current task-specific update naturally preserve previously acquired knowledge. From a knowledge-decomposition perspective, we observe that low-rank adaptations exhibit highly imbalanced singular value spectra: a few dominant components absorb most of the adaptation energy, thereby (i) more likely to disrupt previously acquired knowledge and (ii) making the update more vulnerable to interference from subsequent tasks. To enable explicit balance among components, we decouple the magnitude of the task update from its directional structure and formulate it as a constrained optimization problem on a restricted Stiefel manifold. We address this problem using a projected first-order method compatible with standard deep-learning optimizers used in vision-language models. Our method mitigates both backward and forward forgetting, consistently outperforming continual learning baselines. The implementation code is available at https://github.com/haodotgu/EBLoRA.

Figures

Figures reproduced from arXiv: 2602.00722 by Han-Chen Zhang, Hao Gu, Mao-Lin Luo, Min-Ling Zhang, Tong Wei, Zi-Hao Zhou.

Figure 1
Figure 1. Figure 1: Comparison of parameter-efficient methods on the UCIT benchmark in terms of MFN and FWT. MFN measures the model’s final performance after learning all tasks, whereas FWT reflects the ability to generalize learned knowledge to unseen tasks. Zero￾shot upper bound is the mean accuracy of the base model, while LoRA-first upper bound is the mean accuracy of LoRA on each task immediately after it is learned. EBL… view at source ↗
Figure 2
Figure 2. Figure 2: Imbalanced low-rank components amplify cross-task interference. (a) LoRA updates exhibit long-tailed singular value spectra where a few components dominate adaptation energy, while variance increases during training. (b) Multi-task LoRA merging reveals that direct merging degrades performance, while singular value smoothing can reduce interference and improve task coexistence. a common backbone to avoid co… view at source ↗
Figure 3
Figure 3. Figure 3: Comparison between LoRA-FT and EBLoRA (Ours) on the MLLM-DCL (left) and UCIT (right) benchmarks. The radar plots display the accuracy of each task immediately after it is learned, showing that our method preserves the model’s ability to continuously learn new tasks. 0 20 40 60 80 100 Smooth ratio of noise(%) 0.6 0.7 0.8 0.9 1.0 Normalized Accuracy Performance under part-smoothed noise ImageNet-R ArxivQA Vi… view at source ↗
Figure 4
Figure 4. Figure 4: Interference analysis of balanced singular value. Noise is generated through merging LoRA weights trained on UCIT tasks. Left: applying partially smoothed noise on LoRA reduces interference in seen tasks. Right: under equal-norm perturbations, EBO approach exhibits higher robustness compared to LoRA. factorization and energy-balancing mechanism while remov￾ing the gradient-orthogonality constraint and dept… view at source ↗
Figure 5
Figure 5. Figure 5: Heatmaps of LoRA (left) and EBO (right) in UCIT tasks. The upper triangle denotes the performance on unseen tasks, while the lower triangle denotes performance on seen tasks. Raw data for these visualizations can be found in Tab. 4a and 6. performance). This experiment is designed to isolate the effect of balanced energy on continual transfer by removing gradient-orthogonality and initialization components… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 10 linked inside Pith

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [9]

    Vlm-assisted continual learning for visual question answering in self-driving

    Lin, Y ., Qi, M., Liu, L., and Ma, H. Vlm-assisted continual learning for visual question answering in self-driving. In arXiv preprint arXiv:2502.00843,

    arXiv

  3. [10]

    Lindstr¨om, A. D. and Abraham, S. S. Clevr-math: A dataset for compositional language, visual and mathematical rea- soning. InarXiv preprint arXiv:2208.05358,

    Pith/arXiv arXiv

  4. [12]

    Continual learning for natural language generation in task- oriented dialog systems

    Mi, F., Chen, L., Zhao, M., Huang, M., and Faltings, B. Continual learning for natural language generation in task- oriented dialog systems. InFindings of the Association for Computational Linguistics: EMNLP 2020, Online,

    2020

  5. [13]

    Orthogonal sub- space learning for language model continual learning

    Wang, X., Chen, T., Ge, Q., Xia, H., Bao, R., Zheng, R., Zhang, Q., Gui, T., and Huang, X.-J. Orthogonal sub- space learning for language model continual learning. In Findings of the Association for Computational Linguis- tics: EMNLP 2023, 2023a. Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.-Y ., Ren, X., Su, G., Perot, V ., Dy, J., et al...

    Pith/arXiv arXiv 2023

  6. [14]

    Robust- merge: Parameter-efficient model merging for mllms with direction robustness.arXiv preprint arXiv:2502.17159,

    Zeng, F., Guo, H., Zhu, F., Shen, L., and Tang, H. Robust- merge: Parameter-efficient model merging for mllms with direction robustness.arXiv preprint arXiv:2502.17159,

    arXiv

  7. [15]

    Multi- prototype grouping for continual learning in visual ques- tion answering

    11 Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation Zhang, L., Mao, Z., Peng, Y ., Fu, Z., and Zhang, Y . Multi- prototype grouping for continual learning in visual ques- tion answering. InICASSP 2025 - 2025 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP),

    2025

  8. [16]

    Mllm- cl: Continual learning for multimodal large language models

    Zhao, H., Zhu, F., Wang, R., Meng, G., and Zhang, Z. Mllm- cl: Continual learning for multimodal large language models. InarXiv preprint arXiv:2506.05453,

    arXiv

  9. [17]

    It covers 5 specialized areas: Remote Sensing, Medical, Driving, Finance, and Science

    consists of multiple downstream VQA datasets: RSVQA (Lobry et al., 2020), PathVQA (He et al., 2020), DriveLM (Sima et al., 2024), FinVis (Wang et al., 2023b), AI2D (Kembhavi et al., 2016), SciVerse (Guo et al., 2025d), MapQA (Chang et al., 2022), and TQA (Kembhavi et al., 2017). It covers 5 specialized areas: Remote Sensing, Medical, Driving, Finance, and...

    2020

  10. [18]

    benchmark contains 8 multimodal datasets as seen tasks: ScienceQA (Lu et al., 2022), ImageNet (Deng et al., 2009), VQAv2 (Goyal et al., 2017), REC-COCO (Kazemzadeh et al., 2014; Mao et al., 2016), OCRVQA (Mishra et al., 2019), Flickr30k (Plummer et al., 2015), VizWiz-caption (Gurari et al.,

    2022

  11. [19]

    Each dataset is treated as a task

    and IconQA (Lu et al., 2021). Each dataset is treated as a task. B.3. Evaluation Metrics We present the definitions and mathematical expressions of the four aggregate metrics employed in our evaluation: Mean Final Accuracy (MFN), Mean Average Accuracy (MAA), Backward Transfer (BWT), and Forward Transfer (FWT). MFN measures the average accuracy across all ...

    2021

  12. [1975]

    Mapqa: A dataset for question answering on choropleth maps

    Chang, S., Palzer, D., Li, J., Fosler-Lussier, E., and Xiao, N. Mapqa: A dataset for question answering on choropleth maps. InarXiv preprint arXiv:2211.08545,

    Pith/arXiv arXiv

  13. [1999]

    Oasis: Online sample selection for continual visual instruction tuning.arXiv preprint arXiv:2506.02011,

    Lee, M., Seo, M., Qu, T., Tuytelaars, T., and Choi, J. Oasis: Online sample selection for continual visual instruction tuning.arXiv preprint arXiv:2506.02011,

    arXiv

  14. [2017]

    Hide-llava: Hier- archical decoupling for continual instruction tuning of multimodal large language model

    Guo, H., Zeng, F., Xiang, Z., et al. Hide-llava: Hier- archical decoupling for continual instruction tuning of multimodal large language model. InarXiv preprint arXiv:2503.12941, 2025a. Guo, H., Zeng, F., Zhu, F., Wang, J., Wang, X., Zhou, J., Zhao, H., Liu, W., Ma, S., Wang, D.-H., Zhang, X.- Y ., and Liu, C.-L. Continual learning for generative ai: From...

    Pith/arXiv arXiv 2025

  15. [2018]

    Pathvqa: 30000+ questions for medical visual question answering

    He, X., Zhang, Y ., Mou, L., Xing, E., and Xie, P. Pathvqa: 30000+ questions for medical visual question answering. InarXiv preprint arXiv:2003.10286,

    Pith/arXiv arXiv 2003

  16. [2021]

    In our method, GPM is used to accumulate gradient information from past tasks

    is a well-established method that maintains a subspace of task-sensitive gradient directions for mitigating interference. In our method, GPM is used to accumulate gradient information from past tasks. For clarity and completeness, we detail the procedure in this section. Initialization.Since no prior tasks exist att= 1, the subspace is initialized asG 0 =...

    2021

  17. [2022]

    Iap: Improving continual learning of vision-language models via instance-aware prompting.arXiv preprint arXiv:2503.20612,

    Fu, H., Zhao, H., Dong, J., Zhang, C., and Qian, H. Iap: Improving continual learning of vision-language models via instance-aware prompting.arXiv preprint arXiv:2503.20612,

    Pith/arXiv arXiv

  18. [2023]

    F., Cheng, K.-T., and Chen, M.-H

    Liu, S.-Y ., Wang, C.-Y ., Yin, H., Molchanov, P., Wang, Y .-C. F., Cheng, K.-T., and Chen, M.-H. Dora: Weight-decomposed low-rank adaptation.arXiv preprint arXiv:2402.09353,

    Pith/arXiv arXiv

  19. [2024]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261,

    Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261,

    Pith/arXiv arXiv

  20. [2025]

    Multimodal arxiv: A dataset for improving scientific comprehension of large vision- language models

    Li, L., Wang, Y ., Xu, R., et al. Multimodal arxiv: A dataset for improving scientific comprehension of large vision- language models. InarXiv preprint arXiv:2403.00231,

    Pith/arXiv arXiv