Pith. sign in

REVIEW 4 major objections 5 minor 38 references

Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an entire audio-visual recording can be distilled into a fixed-budget LoRA adapter, letting a frozen omni-modal model answer questions with zero multimodal tokens in context while still beating full-context and…

desk verdict A serious, unusually candid empirical paper with a real architectural mismatch between the advertised rank-selection mechanism and what the scoring network can actually do, plus headline numbers that overshoot the paper's own tables. read the letter →

arxiv 2608.09227 v1 pith:LMEA6KKK submitted 2026-08-10 cs.AI

Puneet Mathur , Manan Suri , Dinesh Manocha This is my paper · ORCID
classification cs.AI
keywords parametricmemorycompressionlow-rankadaptationaudio-visualcoherencetokenomnimodallanguagemodelsGRPOhypernetworkvideoquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Omni2LoRA claims that the token bottleneck of omni-modal language models can be bypassed entirely: a synchronized audio-visual recording is compressed into a recording-specific Low-Rank Adaptation (LoRA) adapter, and all downstream questions are answered with zero audio or video tokens in the context window. The paper's load-bearing result is that a reinforcement-learned rank-allocation policy, optimized with a coherence-aware reward that penalizes loss of cross-modal anchors, lets a 30% rank budget beat both full-context inference and strong token-compression baselines on four audio-visual question-answering benchmarks. If correct, this would make long-video understanding practical: per-query time to first token drops by up to 12 times, amortizing to under 0.5 seconds after a handful of queries. The paper also reports that accuracy stays stable at 75% compression and even improves as frame count grows to 1,024, where in-context methods collapse.

What carries the argument

The mechanism is a two-stage pipeline. In Stage 1, a Perceiver-style hypernetwork consumes intermediate representations $C_t = E(v_t, i)$ of each temporal chunk of the frozen encoder and emits a bank of rank-one LoRA slots $\{A_{\ell,m,t,r}, B_{\ell,m,t,r}\}$. In Stage 2, a lightweight scoring network $g_\psi$ assigns a logit $z_{\ell,m,t,r} = g_\psi(\bar{C}_t, \ell, m, r)$ to every slot from the mean-pooled chunk summary and positional index; sequential without-replacement sampling defines an autoregressive allocation policy, optimized with clipped GRPO. The coherence mechanism is the advantage shaping: a unimodal counterfactual penalty multiplies the degradation $\Delta_i$ by $1 + \eta \Omega(q)$, where $\Omega(q) = \mathrm{ReLU}(R_{AV} - \max(R_V, R_A))$ is the Audio-Visual Dependence Score, so allocations that drop cross-modal anchors on strictly joint queries are penalized.

What would settle it

Run the two-stage pipeline on UGC-AVQA at a 30% budget with the coherence penalty disabled (setting $\eta = 0$); if accuracy does not drop relative to the full Omni2LoRA reward, then the counterfactual advantage shaping is not what preserves cross-modal anchors.

Watch

Extended reading notes

Core claim

The central claim is that parametric memory compression can outperform token compression for joint audio-visual reasoning, provided the adapter's rank budget is allocated by a coherence-aware policy rather than by magnitude or uniform selection. Across three omnimodal backbones (Qwen2.5-Omni-3B, InteractiveOmni-4B, Qwen2.5-Omni-7B), the 30%-budget adapter beats the full-context baseline and OmniZip, OMAC, and O-MARC on the average accuracy across DailyOmni, UGC-AVQA, OmniVideoBench, and WorldSense, with the largest margin on UGC-AVQA, a benchmark whose items are retained only if removing the audio track degrades a strong reference model. The paper further claims that accuracy remains stable at 75% compression and increases monotonically with frame count up to 1,024 frames, because the adapter is built once and reused, so the context window never fills. Efficiency is reported with the one-time encoding cost included in the single-query TTFT; the sub-second amortized latency is what the paper argues makes the method practical for repeated querying.

Load-bearing premise

The rank-allocation policy decides which LoRA slots to keep from only the mean-pooled chunk summary and the slot's layer, module, and rank position; it never sees the actual LoRA coefficient values, so any cross-modal information that survives only in fine-grained activation patterns invisible to mean pooling could not be protected by the learned allocation, and the claimed coherence preservation at a 30% budget would fail.

Editorial extensions

If this is right

  • At a 30% rank budget, Omni2LoRA is reported to beat both uncompressed full-context inference and token-pruning baselines (OmniZip, OMAC, O-MARC) on DailyOmni, UGC-AVQA, OmniVideoBench, and WorldSense across three backbones.
  • On UGC-AVQA, accuracy stays at 60.7% under 75% compression, while OmniZip drops to 47.1, OMAC to 49.1, and O-MARC to 56.3.
  • With 1,024 sampled frames, Omni2LoRA reaches an average score of 46.2 on OmniVideoBench, where direct full-context inference collapses to 22.0 because the context window is exhausted.
  • Zero multimodal tokens at answer time reduce single-query TTFT on VidCapBench to 0.49 seconds on the 7B backbone, versus 3.45 seconds for O-MARC and 6.03 seconds for full context, amortizing below 0.5 seconds after about five queries.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed-budget adapter acts like a per-recording memory that is reusable across queries; a natural extension the paper leaves open is composing or swapping adapters for multi-recording conversations, which would require cross-adapter routing rather than a single cached adapter.
  • Because $g_\psi$ conditions only on mean-pooled summaries, the policy cannot detect fine-grained activation patterns; if cross-modal anchors live mainly in high-variance or sparse activations, the reported stability at 30% budget may depend on Stage 1 having already encoded anchors redundantly across many candidate slots.
  • The coherence reward requires cached unimodal reference rollouts from the frozen answer model; in streaming or on-device settings where those rollouts are costly, one could approximate $\Omega(q)$ with a cheap joint-versus-single-modality probe, but that replaces a grounded counterfactual with an estimate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Omni2LoRA, a two-stage framework that compresses audio-visual recordings into parametric memory for frozen omnimodal language models. Stage 1 trains a Perceiver hypernetwork to map chunked encoder hidden states to a full-rank bank of candidate LoRA slots. Stage 2 trains a lightweight scoring network via GRPO to select a fixed budget of slots, using a counterfactual advantage that penalizes loss on queries with high audio-visual dependence. The paper reports accuracy gains over token-compression baselines on four AVQA benchmarks, robustness at 75% compression, and large TTFT reductions. The appendix provides detailed pseudocode, reward definitions, and explicit limitations.

Significance. If the empirical results are reliable, the parametric-memory direction is significant: it removes in-context multimodal tokens at answer time and amortizes a one-time encoding cost. The manuscript is unusually transparent: algorithms and reward specifications are given in full, and Appendix E correctly disclaims any theoretical contribution. However, the headline improvement number and the efficiency numbers are not supported by the paper's own tables, and the scoring network's input space does not implement the content-dependent rank-direction selection that is the paper's central mechanism. These issues are load-bearing and must be resolved before the claims can be accepted.

major comments (4)
  1. [Methodology, 'Scoring Network'; Appendix B.6] The scoring network g_psi is conditioned only on the mean-pooled chunk summary C_bar_t and the positional embedding of (l,m,r); it never reads the continuous LoRA factors A and B. Consequently, within a given chunk, all slots share the same content representation, and the logit for a given (l,m,r) depends only on the index embedding. The policy cannot distinguish a rank direction that encodes a sparse acoustic anchor from one that encodes a visually dominant background direction inside the same chunk. This contradicts the stated mechanism in the Abstract and Stage 2 ('allocates its fixed sub-linear rank budget to synergistic cross-modal anchors rather than isolated visual features') and the ablation interpretation in the Ablation Analysis ('learns to allocate ranks specifically where audio and video intersect'). At best, the described architecture performs chunk/layer/module selection from pooled statistics. Please condition g_psi on per-slot features (e.g., the A/B factors, per-slot norms, or per-slot activation summaries), or revise the central contribution and the ablation claims to chunk-level selection and provide supporting analysis.
  2. [Abstract and Introduction, 'Main Result'; Table 1] The abstract and introduction claim an '8-12%' improvement over the strongest baseline, but Table 1 does not support this. The average accuracy gaps over O-MARC are 47.3 vs. 45.8 (3.3% relative), 47.6 vs. 45.8 (3.9% relative), and 53.2 vs. 51.1 (4.1% relative) on the three backbones. Even on UGC-AVQA overall accuracy, the relative gaps are 6.6%, 6.7%, and 5.3%, respectively. The stated '8-12%' does not appear anywhere in the reported tables. Please correct the headline numbers to the measured values or explicitly identify the baseline/calculation that yields 8-12%.
  3. [Inference Efficiency and Amortization; Figure 4] The caption of Figure 4(a) states that single-question TTFT includes the time to internalize the video, and the text reports Omni2LoRA 7B single-question TTFT of 0.49s and an amortized TTFT of 0.82s after five queries. If T_setup + t_query = 0.49, then the amortized value (T_setup + 5*t_query)/5 equals 0.098 + 0.8*t_query, which is at most 0.49 for any non-negative t_query. The reported 0.82s is arithmetically impossible under the stated definition. The 3B numbers (0.43s single vs. 0.72s after five queries) are inconsistent in the same way. Please reconcile the definition of TTFT or the reported numbers; the efficiency claim is a headline contribution and cannot rest on contradictory figures.
  4. [Coherence-Aware Advantage Shaping, Eq. (6)] The Audio-Visual Dependence Score Omega(q) is computed from the frozen model's own rewards R_AV, R_V, and R_A. If the frozen backbone cannot exploit joint audio-visual evidence even with full tokens (i.e., R_AV is low), then Omega(q) is near zero and the coherence penalty in Eq. (8) is inactive. Thus the training signal is only as strong as the backbone's intrinsic joint-reasoning ability. The paper evaluates on backbones that do solve the full-token setting, so this is not fatal, but it is an important assumption that should be stated explicitly and, ideally, validated by reporting the distribution of Omega(q) over the training corpus or by comparing against an oracle-based dependence score.
minor comments (5)
  1. [Table 1] The Wilcoxon significance marks (†) are reported without any detail on the number of runs, seeds, or paired samples; please add this information to the caption or experimental setup.
  2. [Figure 3] Figure 3 reports accuracy curves without error bars or confidence intervals, despite Table 2 reporting bootstrap CIs; please add variability information or note that the curves are single-run.
  3. [Terminology] The paper interchanges 'compression ratio' and 'retained ratio' (Appendix B.7 defines compression ratio = 1 - retained ratio), but the abstract's 'compression ratios as tight as 75%' is never tied to the retained-ratio values used in Tables 1 and 2; please make the correspondence explicit.
  4. [Abstract] The abstract says 'four audio-visual question answering benchmarks' while five benchmarks are evaluated; VidCapBench is used for efficiency, but the wording should clarify which four are accuracy benchmarks.
  5. [Eq. (3)] The notation 's(xA^T_{l,m,t,r})B_{l,m,t,r}' is ambiguous about the scaling factor's placement; a parenthetical definition or a small example would improve readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central 30%-budget results are measured against external benchmarks with gold-anchored, rule-based rewards; the self-citational Frames2LoRA dependency is positioning, and the Ω(q) advantage shaping, though self-referential to the frozen model, does not reduce the predictions to its inputs.

full rationale

This paper's load-bearing result — that a 30%-rank-budget adapter beats full-context inference and token-compression baselines — is not circular. The Stage 2 reward is rule-based accuracy against gold labels (App. B.9: 'All rewards are rule-based; no auxiliary judge model is invoked'), the comparisons are against external baselines (OmniZip, OMAC, O-MARC) on held-out benchmark splits, and no parameter fitted to the target metric is renamed as a prediction; R_total is tuned on a held-out validation split. The coherence shaping (Eqs. 6-9) self-referentially derives Ω(q) from the frozen model's own ablated contexts, and if F answers from visual cues alone the penalty is inactive — the authors disclose this ('corpora where audio is redundant with vision would leave the coherence penalty inactive', App. F.1). That is a training-signal coverage property, not a reduction of the prediction to its inputs: Δ_i is computed against the gold-anchored RAV, and the final accuracy is measured on UGC-AVQA, whose joint-evidence filter comes from Wu et al.'s reference model, not from this paper's F, so the evaluation is externally anchored. Self-citation is present (Frames2LoRA (Suri, Baskar, Manocha 2026), two of three authors overlapping) but it appears only as positioning in Related Work and as background in App. C; it is not load-bearing, since the hypernetwork's capacity claim is internally validated by the Full-rank Adapter ≈ Direct AV-in-context near-parity in Table 2. No uniqueness theorem is imported, and no known result is renamed. The strongest non-circularity concerns, weighed here: (1) the scoring network consumes only mean(C_t) and the positional index (ℓ,m,r), and 'It never reads the continuous factors A, B' (App. B.6), so per-slot selection within a chunk cannot be content-dependent, undercutting the narrative that the policy 'learns to allocate ranks specifically where audio and video intersect' (Ablation Analysis); this is a mechanism/implementation mismatch, not an Eq.-equals-Eq. reduction. (2) The abstract's '8-12%' gain exceeds the roughly 3-4-point margins over the strongest baseline (O-MARC) in Table 1, a numeric inconsistency rather than a circularity. (3) App. A's 'Two-stage dependence' limitation ('information absent from the candidate bank is unrecoverable') is honestly disclosed.

Assumptions & free parameters 7 free parameters · 7 assumptions · 1 invented entities

The central claim rests on several domain assumptions and a set of tuned hyperparameters. No invented physical entities are introduced; the recording-specific adapter is a learned data structure. The method's efficiency and accuracy claims depend on hyperparameter values that are not reported, and on the validity of using the frozen model's own ablated responses as a coherence measure.

free parameters (7)
  • eta (coherence penalty strength) = tuned, not reported
    Controls the strength of the cross-modal penalty in Eq. (8); tuned on a held-out validation split.
  • lambda (advantage shaping weight) = tuned, not reported
    Scales the penalty multiplier in Eq. (9).
  • beta (KL regularization coefficient) = tuned, not reported
    KL coefficient in Eq. (10).
  • R_total (rank budget) = 30% retained ratio in main tables
    Sets the number of selected slots; all main results use a 30% budget.
  • R_max (candidate rank cap) = not reported
    Number of rank directions generated per (layer, module, chunk).
  • tau (numerical stability constant) = not reported
    Used in denominators of Eq. (7) and advantage normalization.
  • G (GRPO group size) = not reported
    Number of allocation rollouts per query.
assumptions (7)
  • domain assumption Frozen OLM intermediate states C_t are a sufficient representation to reconstruct task-relevant cross-modal memory as LoRA factors.
    Stage 1 assumes Eq. (1) contains all needed audio-visual information and that H_phi can distill it; if the encoder states discard cross-modal anchors, Stage 2 cannot recover them (the paper's own limitation notes this two-stage dependence).
  • domain assumption Rule-based rewards R_acc and R_fmt are faithful proxies for answer quality.
    Section B.9 defines rewards as option-letter matches and template checks; coarse rewards may under-credit correct free-form answers, as the authors acknowledge in Limitations.
  • domain assumption The frozen model's ablated contexts provide a valid measure of cross-modal necessity for each query.
    Eq. (6) defines Omega(q) using R_AV, R_V, R_A from the same model F; if audio masking breaks temporal alignment or if F answers from visual priors, Omega underestimates true dependence.
  • domain assumption The scoring network's pooled-summary input C_bar_t plus positional index is sufficient to identify coherence-critical rank slots.
    g_psi never sees the A,B values, so it must infer slot utility from mean pooled hidden states; visual norm dominance may hide acoustic cues (Sec. 'Scoring Network', Stage 2). This is the weakest assumption.
  • domain assumption The Perceiver hypernetwork can generate valid rank-one LoRA factors in a single forward pass.
    Eqs. (2)-(3) assume H_phi outputs compose as a valid adapter; prior works (Doc2LoRA, Frames2LoRA) support feasibility but not for synchronized audio-visual streams.
  • domain assumption Training on downstream benchmark training splits aligns the policy with the evaluation query distribution without overfitting.
    Stage 2 draws queries from downstream training splits; if these overlap in style with test queries, gains may not transfer to novel query distributions.
  • standard math Composing per-chunk rank-one updates into a single forward LoRA update is mathematically valid.
    Eq. (3) sums rank-one outer products across chunks and ranks, which is linear addition in parameter space and follows from LoRA's additive definition.
invented entities (1)
  • Recording-specific LoRA adapter (parametric memory)
    purpose: Stores the audio-visual context of a recording in model weights so the frozen backbone can answer queries with zero multimodal tokens in context.
    No external falsifiable prediction beyond the paper's own benchmark results; the adapter's behavior is only measured through the specific benchmarks used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models." pith.science (2026). https://pith.science/paper/LMEA6KKK

@misc{pith2026260809227,
  author       = {Pith},
  title        = {Pith review of: Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LMEA6KKK}},
  note         = {Machine review of arXiv:2608.09227}
}
read the original abstract

Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive. While recent token compression methods attempt to alleviate this burden, compressing modalities in isolation often destroys the temporal cross-modal anchors necessary for coherent reasoning. We introduce Omni2LoRA, a two-stage framework for efficient parametric memory compression via coherence-preserving context distillation that bypasses the token bottleneck entirely. First, a Perceiver hypernetwork processes intermediate representations from a frozen OLM to encode the multimodal context into a full-rank Low-Rank Adaptation (LoRA) adapter in a single forward pass. To prevent the resulting parameter footprint from scaling linearly with recording length, we optimize a discrete rank allocation policy via Group Relative Policy Optimization (GRPO) that uses a modality-ablated counterfactual reward to explicitly penalize the loss of audio-visual coherence, forcing the model to allocate its fixed sub-linear rank budget to synergistic cross-modal anchors rather than isolated visual features. Across three omnimodal backbones, Omni2LoRA operating at a 30% rank budget outperforms direct full-context inference and strong token-compression baselines (OmniZip, OMAC, O-MARC) on four audio-visual question answering benchmarks, improving average accuracy by 8-12% over the strongest baseline and remaining stable under compression ratios as tight as 75%, where token-pruning methods degrade sharply. By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero, cutting per-query Time to First Token (TTFT) by up to 12x relative to full-context inference and amortizing to under 0.5s after a handful of queries.

Figures

Figures reproduced from arXiv: 2608.09227 by the authors.

Figure 1
Figure 1. Omni2LoRA internalizes audio–visual recordings into reusable LoRA memory, enabling zero-context multimodal inference with improved efficiency and cross-modal coherence. transitions, and visual actions often provide complementary evidence for understanding complex events. Despite these advances, efficient omnimodal inference re￾mains a significant challenge. Omni models encode video frames and their high-frequency au… view at source ↗
Figure 2
Figure 2. Overview of Omni2LoRA. A synchronized audio–visual recording is first encoded by a frozen omni LM, whose intermediate representations are consumed by a Perceiver hypernetwork to generate a full-rank bank of candidate LoRA slots (Stage 1). A trainable scoring policy then sequentially selects a fixed-budget subset of slots to construct a compact adapter (Stage 2), which is optimized with GRPO using a coherence-aware a… view at source ↗
Figure 4
Figure 4. Inference efficiency of Omni2LoRA compared with base models (Qwen-2.5 3B and 7B variants) across compression baselines (OmniZip, OMAC, OMARC) on VidCapBench. the average Time to First Token (TTFT). On the 7B back￾bone, it averages 0.49s per question, outperforming both the full-context baseline (6.03s) and the strongest token￾compression baseline, OMARC (3.45s).This performance gap stems from how different framework… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 8 canonical work pages

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    arXiv 1981

  4. [4]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  5. [5]

    Dan Gusfield , title =. 1997

    1997

  6. [6]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  7. [7]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  8. [8]

    International conference on machine learning , pages=

    Robust speech recognition via large-scale weak supervision , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

Show all 38 references
  1. [9]

    arXiv preprint arXiv:2605.26584 , year=

    O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding , author=. arXiv preprint arXiv:2605.26584 , year=

    arXiv

  2. [10]

    arXiv preprint arXiv:2502.04326 , year=

    Worldsense: Evaluating real-world omnimodal understanding for multimodal llms , author=. arXiv preprint arXiv:2502.04326 , year=

    arXiv

  3. [11]

    arXiv preprint arXiv:2510.10689 , year=

    Omnivideobench: Towards audio-visual understanding evaluation for omni mllms , author=. arXiv preprint arXiv:2510.10689 , year=

  4. [12]

    arXiv preprint arXiv:2505.17862 , year=

    Daily-omni: Towards audio-visual reasoning with temporal alignment across modalities , author=. arXiv preprint arXiv:2505.17862 , year=

  5. [13]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Vidcapbench: A comprehensive benchmark of video captioning for controllable text-to-video generation , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  6. [14]

    arXiv preprint arXiv:2511.14582 , year=

    OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models , author=. arXiv preprint arXiv:2511.14582 , year=

    arXiv

  7. [15]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Valor: Vision-audio-language omni-perception pretraining model and dataset , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2024 , publisher=

    2024

  8. [16]

    arXiv preprint arXiv:2210.09461 , year=

    Token merging: Your vit but faster , author=. arXiv preprint arXiv:2210.09461 , year=

    arXiv

  9. [17]

    European Conference on Computer Vision , pages=

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  10. [18]

    arXiv preprint arXiv:2406.07476 , year=

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms , author=. arXiv preprint arXiv:2406.07476 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:2509.17765 , year=

    Qwen3-omni technical report , author=. arXiv preprint arXiv:2509.17765 , year=

    arXiv

  12. [20]

    Advances in Neural Information Processing Systems , volume=

    Javisgpt: A unified multi-modal llm for sounding-video comprehension and generation , author=. Advances in Neural Information Processing Systems , volume=

  13. [21]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    arXiv

  14. [22]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  15. [23]

    arXiv preprint arXiv:2307.03172 , year=

    Lost in the middle: How language models use long contexts , author=. arXiv preprint arXiv:2307.03172 , year=

    arXiv

  16. [24]

    Proceedings of the European conference on computer vision (ECCV) , pages=

    Audio-visual event localization in unconstrained videos , author=. Proceedings of the European conference on computer vision (ECCV) , pages=

  17. [25]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Learning to answer questions in dynamic audio-visual scenarios , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  18. [26]

    , author=

    Lora: Low-rank adaptation of large language models. , author=. Iclr , volume=

  19. [27]

    International conference on learning representations , year=

    Hypernetworks , author=. International conference on learning representations , year=

  20. [28]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    What makes training multi-modal classification networks hard? , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  21. [29]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  22. [30]

    International conference on machine learning , pages=

    Perceiver: General perception with iterative attention , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  23. [31]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  24. [32]

    arXiv preprint arXiv:2602.15902 , year=

    Doc-to-lora: Learning to instantly internalize contexts , author=. arXiv preprint arXiv:2602.15902 , year=

  25. [33]

    arXiv preprint arXiv:2605.28889 , year=

    Context Distillation as Latent Memory Management , author=. arXiv preprint arXiv:2605.28889 , year=

    arXiv

  26. [34]

    arXiv preprint arXiv:2605.30260 , year=

    How LoRA Remembers? A Parametric Memory Law for LLM Finetuning , author=. arXiv preprint arXiv:2605.30260 , year=

    arXiv

  27. [35]

    arXiv preprint arXiv:2603.01097 , year=

    Understanding lora as knowledge memory: An empirical analysis , author=. arXiv preprint arXiv:2603.01097 , year=

    arXiv

  28. [36]

    arXiv preprint arXiv:2606.04351 , year=

    Frames2LoRA: Parametric Video Internalization for Vision-Language Models , author=. arXiv preprint arXiv:2606.04351 , year=

    arXiv

  29. [37]

    Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pa...

  30. [38]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Hyperlora: Parameter-efficient adaptive generation for portrait synthesis , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.