{"id":"fcd770d6-d0b3-4648-9433-7a4a6c5292be","arxiv_id":"2506.16082","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PR-DETR injects k-means-derived position anchors and an overlap-aware relation mask into a DETR decoder, improving dense video captioning on two benchmarks.","lead":"A new DETR-based model for dense video captioning adds explicit position and relation priors to improve event localization and caption coherence. It beats prior DETR baselines on ActivityNet Captions and YouCook2 with moderate gains, using no extra pretraining data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ActivityNet SODA_c regression undercuts the claimed consistent caption gains, and no ActivityNet ablations verify the priors' contribution.","rationale":"The reader identified the statistical correlation between location and semantics as the weakest assumption, but I see a more direct threat to the central claim. Even if the correlation is real, the paper must show that the priors improve both tasks across the claimed datasets. Table I's ActivityNet SODA_c is numerically lower than CM2, contradicting 'consistent improvements' in caption quality; the remaining caption deltas are small enough that sampling variability could flip them. No error bars or multi-seed runs are provided, and the component ablations are confined to YouCook2. The concrete test, ActivityNet ablations with seeds and bootstrap CIs, would settle whether the priors generalize. I also note Eq. 8's description of the overlap-aware distance is inconsistent with its definition (temporally connected events do not generally give 0), which is a separate correctness concern, but the empirical unsupportedness is the most load-bearing issue for the central claim.","tokens_in":16576,"tokens_out":8821,"duration_ms":100430,"concrete_test":"Run the Table III component ablation (Base, +P, +R, +P+R) on the ActivityNet Captions validation set with at least 3 random seeds, reporting mean±std for BLEU4, METEOR, CIDEr, SODA_c, Precision, Recall, and F1. Also compute paired bootstrap 95% confidence intervals for the Table I ActivityNet differences between PR-DETR and CM2. If the +R or +P+R gains over Base are not consistently positive across seeds on ActivityNet, or if the PR-DETR vs CM2 confidence intervals include zero, the central claim of simultaneous improvement is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the position and relation priors simultaneously improve localization and captioning, with consistent improvements over PDVC and CM2 on both datasets. Table I does not fully support this: on ActivityNet, PR-DETR's SODA_c (6.13) is lower than CM2's (6.18), so caption quality is not consistently improved. The other ActivityNet caption deltas are tiny (BLEU4 +0.20, METEOR +0.17, CIDEr +0.15), within typical run-to-run noise for these metrics, and no error bars, seeds, or significance tests are reported. The ablations that causally attribute gains to the two priors (Table III) are run only on the YouCook2 validation set; there is no Base/P/R/P+R ablation on ActivityNet. The mixed-dataset experiment (Fig. 5) compares full models and also lacks variance estimates. Consequently, the evidence that the priors, rather than the added feature-aggregator and relation-encoder parameters, drive the observed gains is incomplete, and the claimed simultaneous improvement over CM2 on ActivityNet is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PR-DETR proposes two explicit priors for DETR-style dense video captioning: (1) position-anchored queries, initialized from k-means clustering of ground-truth event locations and refined through slot-attention feature aggregation into video-specific event anchors; and (2) an event relation encoder that computes an overlap-aware pairwise distance between predicted event anchors and converts it into an attention mask used in decoder self-attention. The model is trained end-to-end with a proposal loss supervising the anchor generation. On ActivityNet Captions and YouCook2, the paper reports improved localization (precision/F1) and captioning (BLEU4, METEOR, CIDEr, SODA_c) over PDVC and CM2 baselines reproduced with CLIP features, with component ablations on YouCook2 validating the two priors. The paper frames these results as evidence that injecting position and relation priors improves event localization and caption quality simultaneously.","tokens_in":16789,"tokens_out":6519,"duration_ms":67543,"significance":"If substantiated, the paper would provide a clean empirical demonstration that explicit position and relation priors can reduce the optimization burden of DETR-like dense video captioning and improve both localization and caption coherence. The main strengths are the simple, interpretable priors; the consistent CLIP-feature reproduction of PDVC and CM2; and the YouCook2 ablations that attribute gains to each component. However, the absence of ActivityNet ablations, missing error bars, and the SODA_c regression on ActivityNet mean the general claim of simultaneous improvement on both datasets is not yet established. The significance is therefore moderate: the idea is plausible and useful, but the evidence is incomplete.","major_comments":[{"comment":"The caption-quality claim on ActivityNet is not fully supported: PR-DETR's SODA_c (6.13) is below CM2's (6.18), and the other caption deltas (BLEU4 +0.20, METEOR +0.17, CIDEr +0.15) are small and likely within run-to-run variation. Since no error bars or significance tests are reported anywhere (Tables I-III and Fig. 5), the paper has not demonstrated consistent caption improvement on ActivityNet, which weakens the abstract's claim of simultaneous improvement. Please report multiple seeds with standard deviations and a significance test for the key comparisons.","section":"Table I (Section IV-B)"},{"comment":"All component ablations are conducted only on the YouCook2 validation set. Without a Base/P/R/P+R ablation on ActivityNet, it is impossible to verify that the position and relation priors, rather than the additional feature-aggregator and relation-encoder parameters, are responsible for the gains on that dataset. Please include the same ablation on ActivityNet, or at least a Base versus full PR-DETR comparison there.","section":"Section IV-C (Tables III-VII)"},{"comment":"The relation prior's motivation rests on the claim that temporally connected events have more semantically related captions, which is supported only by a scatter plot (Fig. 3(b)) without a correlation coefficient or statistical test. Since Eq. (8) directly encodes this assumption, please quantify the correlation (e.g., Pearson or Spearman) on both datasets and show that the observed association is not dataset-specific.","section":"Section III-A and Eq. (8)"},{"comment":"The statement that training uses a YouTube-accessible subset with 'approximately 7% less' than the original video count creates a potential confound when comparing with PDVC† and CM2. Please state explicitly whether these reproduced baselines were trained and evaluated on exactly the same video subset, and provide the exact train/validation/test video counts for each dataset and method.","section":"Section IV-A (Datasets)"}],"minor_comments":[{"comment":"The section heading 'Predction Heads' contains a typo and should read 'Prediction Heads'.","section":"Section III-E"},{"comment":"The sentence 'The overlap-aware distance of two temporally connected events is 0, whereas overlapping or distant events are computed as close to 1' is inconsistent with Eq. (8), where overlapping events give beta > 0 and hence a positive logarithmic value; please clarify what 'temporally connected' means and how the formula maps to the described ranges.","section":"Section III-D, Eq. (8)"},{"comment":"Fig. 3(b) lacks axis labels and a legend; adding them, along with a fitted line and correlation coefficient, would make the claimed relationship between location correlation and semantic similarity visible.","section":"Fig. 3(b)"},{"comment":"In Table I, the best ActivityNet SODA_c value is CM2's 6.18, not PR-DETR's 6.13; the bold formatting should be corrected to avoid misleading the reader.","section":"Table I"},{"comment":"The statement 'too large or too small lambda_prop leads to performance degradation' is not strictly true for all metrics (METEOR at lambda_prop=2 is 6.59, higher than at lambda_prop=1); please qualify the conclusion with respect to the primary metric or explain the discrepancy.","section":"Section IV-C, Table VI"},{"comment":"Table V's first row (no checkmarks) should be labeled as Base+R to match Table III, making the component-wise comparison easier to follow.","section":"Section IV-C, Table V"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical contribution, but the experimental reporting needs to be tightened before acceptance. The most critical issues are the unverified cross-dataset generality of the ablations, the SODA_c regression on ActivityNet, and the data-subset ambiguity relative to the baselines. If the authors address these points adequately, the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-executed, modestly novel DETR extension for dense video captioning. The position-anchored query (k-means cluster centers from training ground truth as initial event anchors, refined with slot attention) and the overlap-aware relation mask are sensibly motivated and clearly described. The ablations on YouCook2 are the strongest part: each component adds something, and the full model beats the PDVC baseline by a clear margin on CIDEr (37.30 vs 29.20) with decent gains on BLEU4, METEOR, and SODA_c. The localization gains also look real on YouCook2. I trust the YouCook2 story.\n\nThe soft spots are about generalization and framing. The headline claim is that the priors improve localization and captioning simultaneously and consistently across both datasets. Table I does not show that on ActivityNet: SODA_c is lower than CM2 (6.13 vs 6.18), and the caption deltas over CM2 are tiny—BLEU4 +0.20, METEOR +0.17, CIDEr +0.15. Those are within run-to-run noise for these metrics, and there are no error bars or multiple seeds. The causal ablations are only on YouCook2; there is no Base/P/R/P+R ablation on ActivityNet. So we know the priors help on YouCook2, but we have no evidence they generalize to ActivityNet. The mixed-dataset experiment in Fig. 5 compares full models only, also without variance, so it doesn't fill that gap. The authors also use a ~7% smaller YouCook2 training subset, which is acknowledged but makes direct comparison with published PDVC/CM2 numbers slightly apples-to-oranges. The statistical motivation in Fig. 3(b) is a scatter plot with no correlation coefficient or test; that is a minor weakness, not fatal, since the ablations provide empirical support.\n\nCitation pattern is fine. DAB-DETR is not cited in the anchor-query discussion, but the contribution is the combination in video captioning, so that's a minor omission. No code release, which matters for reproducibility in this field.\n\nVerdict: this deserves peer review. It is a competent, clearly written paper with a genuinely new design and honest ablations, even though the ActivityNet claims need tempering. A serious referee should ask for ActivityNet ablations, error bars or seeds, and a clear statement that ActivityNet caption gains are not established. I would not desk-reject it; I'd send it out.","headline":"A solid DETR-based dense video captioning paper with genuinely useful priors, but the ActivityNet results don't support the 'simultaneous improvement' headline.","tokens_in":17325,"tokens_out":2073,"would_cite":false,"duration_ms":23114,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PR-DETR claims that injecting data-derived position and relation priors into a transformer-based dense video captioner improves both event localization and caption quality over implicit-learning baselines.","keywords":["dense video captioning","event localization","detection transformer","position-anchored query","relation prior","event relation encoder","video understanding","transformer decoder"],"falsifier":"Compute the rank correlation (e.g., Kendall's tau) between the pairwise location-correlation metric of Eq. (1) and the caption-similarity scores used in the paper's Fig. 3(b) on the validation sets; if the correlation is near zero or negative, the relation prior's motivation collapses. A second decisive check is to replace the learned relation mask with a randomly permuted mask and measure whether the reported CIDEr and SODAc gains survive; if they do, the explicit relation encoding is not carrying the claimed signal.","tokens_in":16383,"feed_emoji":"🎬","tokens_out":11457,"duration_ms":111832,"temperature":0.7,"pith_summary":"Dense video captioning asks a model to find every event in an untrimmed video and describe each one in language. The paper argues that current transformer-based systems learn event locations and event relations purely implicitly, which demands large training sets and leaves the model with ambiguous search regions and inconsistent captions. PR-DETR tries to establish that two data-derived priors fix this: position-anchored queries seeded from the clustering centers of real event locations, and a relation mask built from overlap-aware distances between predicted event boundaries and injected into decoder self-attention. On ActivityNet Captions and YouCook2, the paper reports that both priors improve localization and captioning over strong end-to-end baselines with the same visual features, and that combining them gives the best result.","feed_headline":"Explicit event priors lift dense video captioning on both fronts","feed_subtitle":"Position-anchored queries plus an overlap-aware relation mask beat pretrained baselines on the same features.","key_machinery":"Two components carry the argument. The position-anchored query turns the global statistics of event locations into a starting point: k-means centroids of ground-truth event centers and durations are projected through sinusoidal positional encoding into N event slots, K slot-attention iterations aggregate frame features into event-centric embeddings, and an MLP predicts offsets that turn centroids into scene-specific anchors $P = \\{(c_n, d_n)\\}_{n=1}^N$, supervised by a Hungarian-matched generalized-IoU proposal loss. The relation-enhanced decoder then makes event interaction explicit: at each layer it computes a pairwise overlap-aware distance $r(p_i, p_j) = (\\log(\\beta/d_i + 1), \\log(d_i/d_j))$ from current event boundaries, embeds it, and maps it through a $1\\times1$ convolution to a relation mask $\\tilde{R} \\in \\mathbb{R}^{N\\times N\\times M}$ that is added to $QK^\\top/\\sqrt{D}$ before softmax in self-attention. Static anchors $P_{sta} = P$ keep the position prior fixed during decoding while the event anchors $P$ update layer by layer, so the mask tells the decoder which queries are temporally connected and should exchange semantic context.","core_discovery":"The central claim is that event locations and event semantics in a long video are not arbitrary: long events tend to sit near the video center, and events that are temporally connected tend to share semantic content. A dense video captioner can exploit these regularities directly instead of learning them from scratch. Position-anchored queries are initialized at k-means centroids of ground-truth event boundaries, refined by slot-attention feature aggregation into scene-specific event anchors, and supervised by a proposal loss. A relation-enhanced decoder computes pairwise overlap-aware distances between anchors, encodes them into an attention mask, and adds this mask to self-attention so that interactions between temporally connected events are guided explicitly. With both priors, the model predicts more precise event boundaries and more coherent captions; the ablations attribute localization gains mainly to the position prior and captioning gains mainly to the relation prior.","pith_inferences":["Inference: the paper motivates the relation prior with a scatter plot rather than a quantified correlation, so a natural extension is to measure the rank correlation between the location metric and caption similarity per dataset; the prediction would be that the relation mask's benefit grows with that correlation.","Inference: because the position prior is fit to training-set event clusters, out-of-distribution videos with atypical event layouts are a stress test; one would expect the anchor initialization to help less or even bias localization on such data.","Inference: the relation mask is computed from anchors that are noisy early in decoding, so gating or annealing the mask as anchors refine is a plausible improvement the paper does not explore.","Inference: the same recipe—clustering-based position seeding plus a relation mask over outputs—could transfer to other set-prediction tasks with structured targets, such as temporal action detection or dense paragraph captioning, wherever output coherence matters."],"forward_implications":["Event localization should improve on structured videos, because seeded queries eliminate implausible proposals; the paper reports higher precision and F1 on both YouCook2 (F1 27.29 to 28.87) and ActivityNet Captions (F1 54.91 to 55.59) against the same-feature baseline.","Caption coherence should improve, because the relation mask strengthens interactions among events sharing context; on YouCook2 CIDEr rises from 29.20 to 37.30 and SODAc from 4.95 to 5.43.","Training-data demand should drop, because competitive results are reached without pretraining on millions of extra videos; the injected priors carry part of the learning burden.","Robustness to varied event distributions should rise; on a mixed ActivityNet-YouCook2 training set, the method improves SODAc from 4.87 to 5.33 and F1 from 26.69 to 27.91 over the base model.","The two priors are complementary rather than redundant; ablations show the position prior mainly helps localization, the relation prior mainly helps captioning, and their combination outperforms either alone."],"supporting_citations":[{"why":"Supplies the end-to-end transformer-based parallel-decoding baseline that PR-DETR extends and reproduces with the same visual features.","marker":"[10]"},{"why":"Introduces the detection-transformer set-prediction paradigm whose query and decoder design the method builds upon.","marker":"[21]"},{"why":"Provides the retrieval-augmented end-to-end baseline compared in the main tables.","marker":"[22]"},{"why":"Computes the language-model caption similarity used in the statistical analysis that motivates the relation prior.","marker":"[52]"},{"why":"Defines the dense video captioning task and supplies the ActivityNet Captions dataset used for evaluation.","marker":"[8]"},{"why":"Supplies the YouCook2 dataset used for evaluation.","marker":"[25]"},{"why":"Provides the frozen visual features used as input in all reported experiments.","marker":"[53]"},{"why":"Supplies the deformable-attention transformer encoder used to build multi-scale video representations.","marker":"[45]"}],"fun_headline_variants":["Priors sharpen video event localization and captions","Position and relation priors improve dense video captioning","Explicit event priors aid localization and coherence","PR-DETR: position and relation priors for captioning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that events that are temporally close or overlapping really do have semantically related captions; the paper supports this with one scatter plot and no statistical test, so if that correlation is weak or dataset-specific, the relation mask could stop helping.","fun_headline_variants_meta":{"raw":{"variants":["Priors sharpen video event localization and captions","Position and relation priors improve dense video captioning","Explicit event priors aid localization and coherence","PR-DETR: position and relation priors for captioning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00072,"raw_usage":{"total_tokens":3238,"prompt_tokens":956,"completion_tokens":2282,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":2218}},"tokens_in":572,"tokens_out":2282,"duration_ms":20871,"temperature":1.0,"reasoning_tokens":2218,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:43:41.194750+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the rank correlation (e.g., Kendall's tau) between the pairwise location-correlation metric of Eq. (1) and the caption-similarity scores used in the paper's Fig. 3(b) on the validation sets; if the correlation is near zero or negative, the relation prior's motivation collapses. A second decisive check is to replace the learned relation mask with a randomly permuted mask and measure whether the reported CIDEr and SODAc gains survive; if they do, the explicit relation encoding is not carrying the claimed signal.","supporting_citations":[{"cited_title":"End-to- end dense video captioning with parallel decoding,","cited_arxiv_id":null,"evidence_quote":"Supplies the end-to-end transformer-based parallel-decoding baseline that PR-DETR extends and reproduces with the same visual features."},{"cited_title":"Do you remember? dense video captioning with cross-modal memory retrieval,","cited_arxiv_id":null,"evidence_quote":"Provides the retrieval-augmented end-to-end baseline compared in the main tables."},{"cited_title":"Dense- captioning events in videos,","cited_arxiv_id":null,"evidence_quote":"Defines the dense video captioning task and supplies the ActivityNet Captions dataset used for evaluation."},{"cited_title":"Towards automatic learning of procedures from web instructional videos,","cited_arxiv_id":null,"evidence_quote":"Supplies the YouCook2 dataset used for evaluation."}],"review_version":1}