{"id":"184e61ae-fcfa-4ede-84fc-89d03e5daf23","arxiv_id":"2510.15470","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MSAM introduces two drone-video/text datasets and a CLIP-based multi-semantic pooling model that reports 0.6–3.8 point R@1 gains over earlier video-text retrieval methods.","lead":"This paper defines a new search task—matching drone videos to natural-language descriptions—and builds two overhead-video datasets with text captions, plus a CLIP-based model that uses text to focus on relevant frames. A generalist might care because drone imagery is growing fast and text-based retrieval could make aerial video searchable without manual labels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dataset leakage from random splits of self-built captioned videos is the key unvalidated premise; MSAM's small gains may vanish under scene-level splitting.","rationale":"The reader's weakest assumption is the same as mine: the evaluation could be an artifact of random splits of self-built datasets. I agree. The key load-bearing condition is that train/test partitions are independent at scene level; the paper provides no evidence for this. A scene-level re-split is a decisive check. Other concerns (ambiguous equations in §III-C, no error bars) are secondary because they could be resolved by code release, but the dataset integrity is a prerequisite for trusting any number. The paper's own Table IV shows increments of 0.2–1.3 R@1 between configurations, within plausible seed-to-seed noise on 400–800 test items, reinforcing that the reported advantage may be noise. I recommend keeping the conditional verdict: the paper should only be accepted once the authors release the datasets, provide scene-level splits, and show the reported gains survive.","tokens_in":20022,"tokens_out":5505,"duration_ms":45832,"concrete_test":"Construct scene-level splits: for USRD, use the source ERA event/scene identifier (or video ID) as the grouping key; for UMCRD, group by source YouTube video ID or by drone flight/geolocation session. Ensure no group spans train and test, and also remove any test frame with perceptual hash within a small distance of any training frame. Retrain MSAM and the top-3 baselines (CLIP4Clip, X-Pool, TS2-Net) on these leakage-free splits using the reported hyperparameters. If MSAM's R@1 margin over the best baseline drops below ~1 point or reverses on either dataset, the random-split evaluation is the cause and the state-of-the-art claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that MSAM outperforms 13 baselines on the two new DVTR datasets rests on the integrity of those datasets and their splits. Section IV-A describes USRD (2,864 samples from ERA) and UMCRD (4,235 samples, partly from YouTube and partly manually filmed) with captions 'obtained through both manual annotation and automatic generation using the GPT large model' and random splits (70/20/10 and 60/20/20). If videos or frames originating from the same underlying scene/event appear in both training and test partitions—for example, multiple ERA clips of the same incident, repeated drone passes over one site, or YouTube uploads of the same footage—the model can exploit low-level scene similarity instead of language-grounding, inflating every R@K. The reported margins are small (USRD t2v R@1: 29.9 vs 28.6 for CLIP4Clip; UMCRD: 49.5 vs 48.0), so even modest leakage could account for the advantage. The paper gives no scene-level split, no near-duplicate analysis, no caption QA or inter-annotator agreement, and no error bars/multiple seeds. Without ruling out scene leakage, the headline 'outperforms all existing methods' is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the drone video-text retrieval (DVTR) task, constructs two self-built datasets (USRD from ERA, UMCRD from YouTube plus manual filming), and proposes MSAM, consisting of a cross-modal interactive feature fusion pooling mechanism (CIFFP), probabilistic multi-semantic embeddings, a distribution-driven semantic loss, and a diversity term. The authors report improved retrieval over 13 baselines on both datasets, e.g., USRD text-to-video R@1 29.9 vs 28.6 for CLIP4Clip, and UMCRD text-to-video R@1 49.5 vs 48.0.","tokens_in":20336,"tokens_out":5720,"duration_ms":55983,"significance":"If the two datasets and their splits are valid benchmarks, the paper would provide useful resources and a reasonable baseline for a new task, and the extensive comparison against recent video-text retrieval methods is a strength. The method itself is largely a combination of established ideas: text-guided pooling (X-Pool) and probabilistic embeddings (PCME/UATVR), with an additional diversity regularizer. The main scientific value lies in the proposed DVTR evaluation setup and in whether the reported gains are reproducible on independent scene-disjoint splits. The manuscript ships no code or data yet and the equations for CIFFP are not implementable as printed, so at present the empirical claim is not independently verifiable.","major_comments":[{"comment":"The central CIFFP equations are not well-formed as printed. Eq. (3) contains a bare sum over D with no visible numerator; Eq. (4) multiplies a B×F×T attention score by a norm ||N_v||_F instead of the frame embeddings N_v; Eq. (5) produces a B×T×1 output from a D-dimensional sum; Eq. (9) writes the final similarity as a sum over D of a norm rather than as an explicit inner product. Since CIFFP is a key contribution and every subsequent objective uses S_vt, the method cannot be implemented or checked from the text. Rewrite all pooling equations with explicit tensors, summation axes, and output shapes.","section":"Section III-C, Eqs. (3)-(9)"},{"comment":"The core empirical claim depends on unvalidated self-built benchmarks. The two datasets use random splits, captions are partly generated by GPT/Gemini, and there is no scene/event-level split, near-duplicate analysis, caption QA, or inter-annotator agreement. Because USRD clips come from ERA and UMCRD partly from YouTube, clips from the same scene or flight can easily appear in both training and test sets. The reported margins are small (Table II USRD t2v R@1 29.9 vs 28.6; Table III UMCRD 49.5 vs 48.0), and no standard deviations or multiple-seed results are reported. Without scene-disjoint splits and variance estimates, the claim that MSAM outperforms all baselines is not established.","section":"Section IV-A and Tables II-III"},{"comment":"Hyperparameters λ, k, and the frame count appear to be selected by inspecting test-set R@K, with no separate validation protocol or seed averaging. Fig. 5 reports λ=0.01 as the best value but λ=0.1 is used in the final model; Table V selects k=7 from test R@1 differences of about one point. This selection procedure can create optimistic results and makes the reported gains weak evidence for the method. Report hyperparameter selection on a held-out validation split (or averaged over multiple splits/seeds) and give mean ± std for the final configuration.","section":"Section IV-E, IV-F, IV-G, Fig. 5, Tables V-VII"}],"minor_comments":[{"comment":"T_sigma and V_sigma are D-dimensional, but Eq. (20) is written as a scalar expression; specify whether the loss is summed or averaged over dimensions. Also clarify whether F_mu and F_sigma are shared between modalities; Eq. (17) appears to feed both p_i_t and p_i_v into a single function for each output.","section":"Section III-D, Eqs. (17)-(20)"},{"comment":"The text first says captions were obtained using 'the GPT large model' and then says the Gemini API was used, citing ShareGPT4V. The inconsistency should be corrected and the prompt design / manual annotation instructions described in enough detail for replication.","section":"Section IV-A"},{"comment":"The ablation discussion says adding L_dst slightly decreases R@1 and R@5 on USRD, but Table IV shows increases from 29.7/70.9 (++L_ddsl) to 29.9/71.2 (+++L_dst). On UMCRD, R@1 decreases from 49.8 to 49.5. Please reconcile the text with the table and explain the non-monotonic behavior.","section":"Section IV-D, Table IV"},{"comment":"Fig. 5 lacks axis labels and a legend, and the caption 'Visual analysis of the differences parameter λ' is unclear. Typos include 'Comparism' in Fig. 1, 'presentes' in Section I, 'a orange-roofed' in Fig. 1, and the heading 'T otal Objective Term.'","section":"Figures and typos"},{"comment":"Give the exact CLIP checkpoint, training epochs, number of runs, and the evaluation protocol (e.g., whether metrics are over the full test set). The statement that code and datasets 'will be made publicly available' is not yet a release; please provide a reproducibility statement with the actual links or an ethics/data release plan.","section":"General reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The dataset-quality concern is the main risk. Given that the paper's headline result is measured only on the authors' own randomly split and partially LLM-captioned data, I would want the editor to ensure that the released data is checked for scene-level duplicates. The 'first to systematically propose DVTR' claim should also be verified against the authors' own prior UAV image-text retrieval work (ref. [14]) and existing UAV video-language datasets. If a scene-disjoint split invalidates the reported gains, the central claim would need to be substantially revised."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper introduces drone video-text retrieval (DVTR), builds two datasets, and reports MSAM ahead of 13 baselines by small margins (roughly 1–4 R@1 points). The datasets and task framing are the genuine new parts; the model is mostly CLIP plus existing ideas — probabilistic embeddings from PCME/UATVR, X-Pool-style cross-modal attention, and orthogonality regularization. That is a reasonable assembly, and the paper is honest about the components.\n\nWhat it does well: the baseline comparison is broad, the ablations are systematic, and the visualizations are useful. They also report parameter counts and time complexity. That is more than many retrieval papers bother with.\n\nThe soft spots are real and load-bearing. Both datasets are self-built; captions come partly from LLMs (GPT in the abstract, Gemini in Section IV-A — already inconsistent). Splits are random. ERA is an event-recognition dataset, so clips of the same incident or repeated drone passes over one site could easily straddle train and test. With margins this small, even modest leakage could explain the advantage. There is no near-duplicate analysis, no scene-level split, no caption QA, and no error bars — one seed per result. The pooling equations (3)-(9) are also malformed as printed: dimensions do not line up, and the softmax-over-Frobenius notation is confusing. A competent reader cannot reconstruct the method from the text alone. Hyperparameters λ and k are chosen on the same data, and the frame-number analysis is non-monotonic, so the optimal-setting claim is weak.\n\nNone of this means the method is wrong. The consistent positive margins across many baselines and the ablations suggest the components do something. But the central claim that MSAM outperforms all existing methods is only as good as the datasets, and the datasets are the weakest link. A scene-level split, a caption quality audit, and code/data release would settle most of my concerns.\n\nThis paper deserves serious refereeing — the task is genuinely new and practically relevant. But a referee should insist on data validation before acceptance, and nobody should cite the numbers as established fact until the leakage question is resolved. I would bring it to reading group mainly to discuss dataset construction and evaluation hygiene.","headline":"New task and two self-built datasets are the real contribution; the method is a plausible assembly, but the evaluation rests on random splits and unverified LLM captions, so the headline gains are not yet established.","tokens_in":20794,"tokens_out":1401,"would_cite":false,"duration_ms":13854,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MSAM, a cross-modal retrieval model for drone video, represents each video and caption as a set of Gaussian semantic embeddings and pools frames with text guidance; the paper reports top results on two new drone video-text datasets.","keywords":["drone video-text retrieval","cross-modal retrieval","multi-semantic learning","probabilistic embeddings","distribution-driven semantic learning","text-guided video pooling","aerial video-text datasets"],"falsifier":"Check caption quality and split leakage directly: ask independent annotators to flag factual errors in a random sample of the GPT-generated captions, and compute nearest-frame similarity between training and test partitions to see whether frames from the same scene appear on both sides. Either finding would overturn the benchmark premise.","tokens_in":19912,"feed_emoji":"🚁","tokens_out":5274,"duration_ms":44084,"temperature":0.7,"pith_summary":"The paper identifies drone video-text retrieval (DVTR) as a distinct task whose overhead viewpoint creates repetitive scenes and ambiguous object configurations that ground-view video-text methods are not built for. To address it, the authors build two new datasets, USRD and UMCRD, with five captions per video, and propose MSAM. MSAM's core move is to replace single-vector video and text representations with k probabilistic embeddings, so one video can match many paraphrases of the same scene; a text-guided pooling mechanism (CIFFP) then suppresses frames unrelated to the query. In reported experiments, MSAM outperforms all 13 baselines on both datasets for both text-to-video and video-to-text retrieval, including text-to-video R@1 improvements from 48.0 to 49.5 on UMCRD and 28.6 to 29.9 on USRD.","feed_headline":"New model sets best marks for drone video-text retrieval","feed_subtitle":"Treating captions and frames as probabilistic semantic distributions handles the ambiguity of overhead scenes.","key_machinery":"The load-bearing components are four. First, the adaptive semantic construction module maps text tokens and video frames to k mean/variance pairs, turning each sample into a Gaussian distribution. Second, the distribution-driven semantic learning term minimizes an asymmetric KL-like divergence between matched video and text distributions. Third, the diversity semantic term penalizes the Gram matrices of normalized embeddings for deviating from the identity matrix, keeping the k semantics decorrelated. Fourth, cross-modal interactive feature fusion pooling (CIFFP) computes frame-text attention, aggregates frames under text influence, applies a second text attention, and combines both branches","core_discovery":"On the paper's own terms, the central claim is that drone-specific retrieval requires modelling semantic uncertainty rather than deterministic point matching. Each video and caption is converted into a set of probabilistic embeddings whose means and variances are aligned by a distribution-driven learning term, and an orthogonal diversity term keeps the k embeddings semantically distinct. The cross-modal interactive feature fusion pooling mechanism uses a two-stage attention over frame-text similarities to reconstruct video features, filtering out background frames. The paper reports that this combination beats every compared method on its two self-built datasets, and takes this as evidence t","pith_inferences":["Editorial inference: the random (non-scene-disjoint) split may let near-duplicate frames from the same flight appear in both training and test sets, so the reported margins could narrow when evaluated with geographically or flight-level disjoint splits.","Editorial inference: with LLM-generated captions, the benchmark inherits any systematic caption biases; human verification of a caption subsample would quantify this.","Editorial inference: the probabilistic-embedding mechanism is not drone-specific in principle, so the same distribution-alignment and diversity losses could transfer to other repetitive-scene retrieval problems, such as satellite or vehicle-mounted imagery.","Editorial inference: the visualization cases where a near-synonym query ranks a different video first (e.g., 'speedy' cars) suggest caption semantics, not visual features, may be the next bottleneck; richer captions could shift results."],"forward_implications":["Dedicated drone retrieval mechanisms, rather than direct adaptation of ground-view models, can yield measurable gains on overhead footage.","Representing captions and videos as distributions rather than points helps when multiple videos share near-identical scenes yet have different valid descriptions.","Text-conditioned frame pooling can reduce the visual noise that average or self-attention pooling picks up in aerial video.","The two datasets give the community standard evaluation axes for a task that previously had none, assuming the promised code and data are released.","Each component contributes independently: CIFFP alone lifts the CLIP4Clip baseline on both datasets, and the semantic terms add further gains on the reported recall metrics."],"fun_headline_variants":["Drone retrieval models semantic ambiguity as probability","Probabilistic embeddings align drone videos and text","MSAM maps word-frame pairs to uncertain distributions","Adaptive mining of multi-semantic features for drone retrieval","Distribution-driven learning sharpens drone video-text match"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything hinges on the two new datasets honestly measuring retrieval: if the machine-generated captions contain systematic errors, or if frames from the same scene appear in both training and test sets, the reported gains overstate real-world performance.","fun_headline_variants_meta":{"raw":{"variants":["Drone retrieval models semantic ambiguity as probability","Probabilistic embeddings align drone videos and text","MSAM maps word-frame pairs to uncertain distributions","Adaptive mining of multi-semantic features for drone retrieval","Distribution-driven learning sharpens drone video-text match"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1186,"prompt_tokens":745,"completion_tokens":441,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":489,"tokens_out":441,"duration_ms":4582,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T09:23:48.530252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check caption quality and split leakage directly: ask independent annotators to flag factual errors in a random sample of the GPT-generated captions, and compute nearest-frame similarity between training and test partitions to see whether frames from the same scene appear on both sides. Either finding would overturn the benchmark premise.","supporting_citations":[],"review_version":1}