{"id":"b4ff9b33-480a-4c79-a732-82b0cc7bdab4","arxiv_id":"2507.00068","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":10,"one_line_summary":"A multimodal retrieval pipeline that projects video and audio into text and claims near-optimal context selection, with reported gains of up to 22.6% on Video-MME that rest on circular theory and unreleased data.","lead":"MANTA is a pipeline that turns video and audio into text descriptions, then uses retrieval to pick the most useful text for a language model to answer questions. The paper claims very large accuracy gains on long-video QA, but the support is weak: key proofs assume their own conclusion and no code or data is released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2 is not applicable to MANTA: the density score D(s_i) in Eq. (5) is query-independent, so assumption (iii) |D(s_i)-I(s_i;Q)|<γ cannot hold across queries; the 1-(ε+δ+γ) guarantee therefore does not cover the claimed system.","rationale":"The reader's weakest assumption identifies exactly the load-bearing premise: the hand-designed D(s_i) approximates the query mutual information I(s_i; Q), and no evidence is given. My stress-test sharpens this from 'unproven' to 'internally inconsistent': Eq. (5) has no query-dependent term at all, so assumption (iii) cannot hold for more than one query in any informative retrieval problem. This is the single most decisive flaw because Theorem 2 is the paper's theoretical centerpiece and the stated foundation for the reported gains. The proof has additional unsupported steps, notably the greedy knapsack claim, but the query-independence of D is sufficient to invalidate the guarantee. The empirical claims are also not independently verifiable, but under the instruction to focus on the argument, the theoretical failure is the cleanest ground for rejection. I agree with the reader's verdict; no adjustment is needed beyond reaffirming REJECT.","tokens_in":12713,"tokens_out":3784,"duration_ms":41393,"concrete_test":"Analytical test: fix a video and instantiate Theorem 2 with two queries Q1 and Q2 whose gold relevant segments are disjoint, e.g., 'What color is the player's jersey?' and 'What is the final score?' Since Eq. (5) contains no query argument, D(s_i) is identical in both cases. If assumption (iii) holds, then |I(s_i; Q1) - I(s_i; Q2)| < 2γ for every segment. Estimating I(s_i; Q) for these two queries on any Video-MME sample would show differences far exceeding 2γ, disproving the premise. If the authors intended D to depend on Q, they must restate Eq. (5), re-derive the bound, and show which component of D encodes the query.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is Theorem 2 (Section 3.5): selecting segments by the density score D achieves a 1-(ε+δ+γ) approximation to the optimal query-relevant selection. The target objective, Eq. (1), maximizes I_α,β(S; Q), and the proof reduces it to maximizing the sum of I(s_i; Q) over selected segments. Assumption (iii) requires |D(s_i) - I(s_i; Q)| < γ. But Eq. (5) defines D(s_i) = -log p(s_i|s_<i) + α·H(s_i) + β·I(c_i; t_i) - γ·R(s_i). None of these four terms depends on Q. For a fixed video, D(s_i) is identical for every query, whereas I(s_i; Q) is query-dependent. Applying assumption (iii) to two arbitrary queries Q1 and Q2 yields |I(s_i; Q1) - I(s_i; Q2)| < 2γ for every segment, meaning all queries would have near-identical relevance scores across all segments. This is false in any nontrivial retrieval setting and is contradicted by MANTA's own two-stage retrieval (Eq. 12), which depends on the query embedding. The theorem therefore cannot justify the optimization of Eq. (1). Additionally, the proof's transition from assumption (ii) to 'a greedy algorithm achieves 1-δ' is not derived: the standard knapsack greedy guarantee is 1/2, and a correlation bound of δ does not by itself produce 1-δ. The stated approximation ratio is thus unsupported independent of the proxy issue. The shared notation γ for the redundancy weight in Eq. (5) and the error tolerance in Theorem 2(ii) further obscures the gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MANTA, a pipeline for long-form video question answering that transcribes video and audio into textual segments, scores segments with a query-independent information-density heuristic, aligns visual and textual representations with contrastive learning, and retrieves query-relevant segments under a token budget for a downstream LLM. The authors claim a provable near-optimality guarantee for this selection procedure (Theorem 2) and report large accuracy gains on Video-MME plus two newly introduced benchmarks, LVU-QA and MultiModal-TempRel. The manuscript also includes a convergence theorem for contrastive alignment, ablation studies, and qualitative case studies.","tokens_in":13143,"tokens_out":3524,"duration_ms":39676,"significance":"If the theoretical guarantee and the empirical gains were both valid, the paper would be significant: it would give a principled, provably near-optimal mechanism for constructing LLM context from long-form multimodal input, and the reported improvements over strong baselines would be practically important. The contrastive alignment result, however, is a standard InfoNCE-style statement, and the central optimality theorem is not established: its main assumption effectively assumes the query-independent heuristic equals the query-dependent mutual information, and the knapsack approximation step is asserted rather than derived. The empirical evaluation is also difficult to assess because the two new benchmarks are not released, no error bars or significance tests are given, and several baselines are not cited. The paper is therefore currently a plausible engineering recipe with an unsupported optimality claim and incomplete validation.","major_comments":[{"comment":"The optimality theorem is not applicable to the system as described because the density score D(s_i) defined in Eq. (5) is query-independent, while the quantity it is assumed to approximate, I(s_i; Q), is query-dependent. Each term in Eq. (5) -- -log p(s_i|s_<i), H(s_i), I(c_i; t_i), and R(s_i) -- is a property of the video segment alone. For a fixed video, D(s_i) is the same for every query, whereas I(s_i; Q) varies with Q. Applying assumption (iii) to two queries Q1 and Q2 gives |I(s_i; Q1) - I(s_i; Q2)| < 2γ for every segment, which would mean all queries have nearly identical relevance scores across all segments. This contradicts any nontrivial retrieval setting and is also contradicted by the paper's own query-dependent two-stage retrieval in Eq. (12), which projects the query and reranks candidates by query similarity. Theorem 2 therefore cannot justify the optimization of Eq. (1).","section":"Section 3.5, Theorem 2, assumption (iii)"},{"comment":"The approximation-ratio derivation has unsupported steps that are load-bearing. Eq. (9) bounds the gap between total mutual information and the sum of per-segment mutual informations by C(|S|,2)·ε, yet the theorem's conclusion uses only ε in the ratio 1-(ε+δ+γ); no argument is given for how the quadratic term is absorbed. More seriously, the proof states that under assumption (ii) 'a greedy algorithm selecting items based on value density achieves an approximation ratio of 1-δ,' but this is asserted without proof. The standard greedy guarantee for the knapsack problem is 1/2, and the uncorrelatedness bound δ does not by itself yield a 1-δ approximation. Consequently, the claimed ratio is unsupported independently of the proxy issue in assumption (iii).","section":"Section 3.5, proof of Theorem 2"},{"comment":"The empirical validation is insufficient for the strength of the claims. LVU-QA and MultiModal-TempRel are introduced in Section 4.1 as 'newly collected' benchmarks, but no release information, dataset construction details, annotation protocols, or basic statistics beyond video/question counts are provided, and no evaluation code or data is made available. No table in the paper reports error bars, multiple seeds, or statistical significance tests, despite the main results showing differences of more than 20 percentage points. Given that the central claims of the paper rest on these numbers, the evaluation as presented cannot be independently verified or reproduced.","section":"Section 4.1 and Tables 1-3"}],"minor_comments":[{"comment":"The symbol γ is used both as the redundancy-penalty weight in Eq. (5) and as the error tolerance in Theorem 2, which is confusing because the two quantities play entirely different roles; please use distinct notations.","section":"Eq. (5) and Theorem 2"},{"comment":"The displayed equation in the proof of Theorem 2 appears garbled, with absolute-value bars rendered inconsistently; if the bound is retained, it should be written cleanly and the combinatorial factor should be addressed.","section":"Eq. (9)"},{"comment":"Several baselines in Table 1 -- MLLM-Projection, MCA-VILLA, Vision-Flan, VideoGPT-4, and MultiVision-7B -- are not identified by citations, so readers cannot determine what systems were used or whether the comparisons are apples-to-apples.","section":"Table 1 and References"},{"comment":"The abstract states a '27.3%' gain on videos exceeding 30 minutes, but the closest evidence in the paper is the 27.3% improvement on Long-Range Dependencies in Table 2, which is not the same as a duration-stratified result; please clarify which claim is supported.","section":"Section 5 and Abstract"},{"comment":"The phrase 'information density estimation' suggests estimation of a probability density, but Eq. (5) is a heuristic weighted sum of novelty, entropy, cross-modal coherence, and redundancy terms; the terminology should be adjusted to avoid overclaiming.","section":"Section 3.5 and Appendix B"}],"recommendation":"reject","confidential_remarks":"The central optimality theorem is not salvageable as stated because its key assumption makes the theorem vacuous for query-dependent retrieval, and the knapsack approximation step is incorrect. The empirical claims rely on two unreleased benchmarks and no error bars. I would not invite resubmission unless the authors substantially reformulate the theoretical claim, provide a valid derivation or clearly weaken the guarantee, and release the evaluation artifacts."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: the engineering recipe here is coherent, but the paper's central theoretical claim is broken and the empirical numbers cannot be checked. If you read only one thing, read Theorem 2 and the stress-test note; the note is right.\n\nThe pipeline — multi-scale captioning, ASR transcription, contrastive alignment, density-based selection, two-stage retrieval — is a reasonable extension of existing RAG-based video QA systems. The components are all known, but the packaged combination is not present in the cited literature, so there is some novelty in the system design. The qualitative example in Section 4.3 is also a nice illustration of how cross-modal fusion could help.\n\nThat is where the credit ends. Theorem 2 is not a valid derivation. Assumption (iii) requires the hand-designed density score D(s_i) to approximate the query mutual information I(s_i;Q), but D(s_i) in Eq. (5) is query-independent — it is computed from the segment alone, its entropy, cross-modal coherence, and redundancy. For a fixed video, D(s_i) is the same for every query, whereas I(s_i;Q) changes with the question. The stress-test's argument is correct: applying (iii) to two distinct queries forces all query-relevance scores to be within 2γ of each other, which contradicts the system's own query-dependent retrieval in Eq. (12). The knapsack step is also unsupported; the standard greedy guarantee for knapsack is 1/2, and the paper does not derive the claimed 1-δ. The shared notation γ for a redundancy weight and an error tolerance only adds to the problem.\n\nThe experimental section is another soft spot. There are no error bars, no code, no released benchmarks, and the two new benchmarks (LVU-QA, MultiModal-TempRel) are only described by name. The headline numbers are huge — near-saturating accuracy on Video-MME is a red flag without independent artifacts. The abstract's 27.3% gain on videos over 30 minutes does not match Table 1's long-video column (max +25.6%) or Table 2's long-range dependencies (+27.3%); a minor mismatch, but it suggests carelessness. The authors do list limitations in the conclusion, which is honest, but the main issues are not addressed.\n\nWho is this for? A reader building a practical video QA system might steal a few ideas from the pipeline, but the theory should be ignored. It does not deserve publication in its current form, though it is substantive enough to send to a referee who can demand fixes: remove the bogus optimality claim, provide code and benchmarks, and reduce the claims to what is actually demonstrated.","headline":"A plausible RAG-for-video pipeline wrapped in an invalid optimality theorem, with unverifiable benchmark claims.","tokens_in":13672,"tokens_out":2300,"would_cite":false,"duration_ms":25522,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MANTA claims that long-form video reasoning reduces to selecting query-relevant text under a token budget with a provable near-optimality ratio.","keywords":["long-form video question answering","cross-modal alignment","information-theoretic content selection","token-budget optimization","multi-scale temporal modeling","retrieval-augmented generation","multimodal large language models"],"falsifier":"Take a labeled set of video-QA pairs, compute $D(s_i)$ for every segment, estimate $I(s_i;Q)$ with a calibrated mutual-information estimator or a strong proxy such as an LLM's pointwise relevance judgments, and compare the two; if the mean absolute error exceeds $\\gamma$ on a meaningful fraction of segments, the claimed $1-(\\epsilon+\\delta+\\gamma)$ guarantee cannot hold for that setting.","tokens_in":12435,"feed_emoji":"🎬","tokens_out":7067,"duration_ms":69552,"temperature":0.7,"pith_summary":"MANTA tries to show that long-form video question answering can be handled by translating both video frames and audio into text segments, scoring each segment by how much information it carries about the query, and selecting a budgeted set of segments to feed an LLM. The paper's central theoretical claim is that this selection is provably near-optimal: under stated assumptions, the density-score greedy choice achieves at least $1 - (\\epsilon + \\delta + \\gamma)$ of the information that the best possible budgeted selection would achieve. Empirically, the paper reports that attaching MANTA to strong video-QA models raises accuracy by up to 22.6 percentage points on Video-MME, with the largest gains on videos over 30 minutes and on temporal and cross-modal reasoning. A careful reader should care because the method offers a transferable design principle: choose context by information density under a token cap, a recipe that would apply to any long multimodal document.","feed_headline":"Near-optimal text scoring lifts long-video QA by up to 22.6%","feed_subtitle":"MANTA scores transcribed video segments by query relevance, then proves the budgeted selection is near optimal.","key_machinery":"The load-bearing object is the per-segment information density score $$D($s_i^{{(l)}}$) = -\\log p($s_i^{{(l)}}$|s_{<i}^{(l)}) + \\$\\alpha$ H($s_i^{{(l)}}$) + \\$\\beta$ I($c_i^{{(l)}}$; $t_i^{{(l)}}$) - \\gamma R($s_i^{{(l)}}$),$$ whose terms respectively penalize redundancy, reward internal richness, reward visual-audio coherence, and penalize overlap with already-selected content. The proof carries the argument by casting budgeted segment selection as a knapsack over these scores: with near-independence of segment information, value density $D/|s|$ becomes the right greedy key, and the approximation ratio follows from the standard greedy knapsack bound plus the assumed closeness of $D$ to $I(s;Q)$. A second mechanism is the hierarchical three-scale segmentation (micro 1-3 seconds, meso 10-30 seconds, macro 1-5 minutes) that feeds the scorer, and a contrastive InfoNCE-style alignment loss that makes visual and auditory representations comparable before they are fused.","core_discovery":"On its own terms, the paper's discovery is a reduction: the problem of building an LLM prompt from a long video, under a token budget, can be treated as a knapsack problem in which each candidate text segment has value equal to its mutual information with the question and weight equal to its token length. MANTA's Theorem 2 asserts that a greedy selection by the density score $D(s_i)$ from Eq. (5) is within a factor $1 - (\\epsilon + \\delta + \\gamma)$ of the optimal query-relevant segment set, provided segment contributions are nearly independent, length and information are nearly uncorrelated, and $D$ approximates the query mutual information within $\\gamma$. The paper pairs this with a contrastive alignment objective that pulls visual captions and audio transcripts into one semantic space, and reports that the full pipeline lifts Video-MME accuracy from 78.9% to 98.2% on its strongest baseline and improves temporal-ordering, causal, and cross-modal reasoning by roughly 23 to 27 percentage points.","pith_inferences":["Left implicit is a testable stronger claim: because the proof only needs a token budget and a query-score proxy, MANTA should transfer to long documents with embedded figures, podcasts, or surveillance feeds by simply swapping the visual and audio captioners; a reader could probe this by running the same scorer on an audio-only or document-only dataset.","The optimality guarantee is stated for a fixed query, so in settings where many queries share one prompt or no query is available at indexing time, the approximation argument does not directly apply and the paper's success there would be entirely empirical.","An editor would want a direct check of assumption (iii): estimating $I(s_i;Q)$ on a labeled subset and comparing it to $D(s_i)$ would either confirm the proxy or bound how far the reported approximation ratio can be trusted."],"forward_implications":["If Theorem 2 holds, any question-answering system with a token budget can use MANTA's density score as a drop-in selection rule and inherit a concrete optimality guarantee rather than a heuristic ranking.","The reported Video-MME numbers imply that the method is most valuable exactly where long-form systems fail: videos longer than 30 minutes gain 27.3% in accuracy, and rare-event detection improves 26.2%.","The ablations imply that multi-scale temporal modeling and hierarchical fusion carry most of the benefit, so coarse single-scale captioning would not achieve the same gains.","Because the framework is modality-agnostic once content is textualized, the same selection and alignment recipe applies to other paired streams, such as images plus OCR, or sensor logs plus natural language."],"supporting_citations":[{"why":"supplies the Video-MME benchmark and its QA pairs that all accuracy numbers are measured on","marker":"(Fu et al., 2024)"},{"why":"provides the Whisper-Large-v2 model used to transcribe audio into the textual segments MANTA aligns","marker":"(Radford et al., 2023)"},{"why":"supplies FAISS, the approximate nearest-neighbor engine that implements the retrieval stage","marker":"(Douze et al., 2024)"},{"why":"defines the joint retrieval-generation training paradigm MANTA compares against","marker":"(Lin and Byrne, 2022)"},{"why":"frames RAG for video QA as proactive visual-information gathering, the approach MANTA's retrieval extends","marker":"(Wang et al., 2023)"}],"fun_headline_variants":["MANTA rethinks video QA: info-theoretic text scoring gains 22.6%","Proven greedy text picker boosts long-video QA by 22.6%","Cross-modal alignment lifts long-video answers up to 22.6%","Near-optimal prompt selection unlocks 22.6% better video QA","MANTA: proving near-optimal selection for long-video questions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantee depends on assumption (iii) of Theorem 2: the hand-designed density score $D(s_i)$ must be within $\\gamma$ of the true query mutual information $I(s_i; Q)$, and the paper does not measure that gap directly, so if $D$ is a poor proxy the near-optimality claim is not established.","fun_headline_variants_meta":{"raw":{"variants":["MANTA rethinks video QA: info-theoretic text scoring gains 22.6%","Proven greedy text picker boosts long-video QA by 22.6%","Cross-modal alignment lifts long-video answers up to 22.6%","Near-optimal prompt selection unlocks 22.6% better video QA","MANTA: proving near-optimal selection for long-video questions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000524,"raw_usage":{"total_tokens":2553,"prompt_tokens":986,"completion_tokens":1567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":1463}},"tokens_in":602,"tokens_out":1567,"duration_ms":11225,"temperature":1.0,"reasoning_tokens":1463,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:56:00.829664+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a labeled set of video-QA pairs, compute $D(s_i)$ for every segment, estimate $I(s_i;Q)$ with a calibrated mutual-information estimator or a strong proxy such as an LLM's pointwise relevance judgments, and compare the two; if the mean absolute error exceeds $\\gamma$ on a meaningful fraction of segments, the claimed $1-(\\epsilon+\\delta+\\gamma)$ guarantee cannot hold for that setting.","supporting_citations":[],"review_version":1}