{"id":"609cf726-eb12-4d43-950d-f986f62dc4da","arxiv_id":"1908.03448","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A temporal action proposal system combining C3D features, a self-attention pyramid network, and boundary refinement reported 71.38% test AUC and second place in the ActivityNet 2019 challenge.","lead":"This technical report describes RapNet, a neural network that proposes action intervals in videos, and reports that the authors' submission placed second in the ActivityNet Challenge 2019. The system combines existing components, including a C3D encoder, a feature pyramid network with self-attention, and boundary refinement, and reports improved accuracy on the benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never ablates its novel self-attention / FPN components, so the reported AUC gains cannot be attributed to RapNet's relation-aware design.","rationale":"The reader identified the fixed 128-snippet resizing as the weakest assumption. That is a legitimate gap in analysis, but it only threatens the optimality of the pipeline, not the integrity of the central claim that the described system achieved the reported AUC and 2nd place. A more load-bearing concern is that the paper's main architectural novelty is never tested in isolation. The phrase 'which helps us achieve 2nd place' makes a causal attribution to RapNet; without ablations that remove or alter the self-attention and FPN components, the paper provides no evidence that these components are responsible for the results. The boundary adjustment ablations in Table 2 are real positive evidence that the post-processing steps matter, and the challenge server results are plausible, so I do not advocate rejecting the paper. But the missing ablation is a concrete, specific gap that should be filled before the relation-aware design is accepted as the contributor. The concrete test I propose would settle this directly in one experiment. Since the reader's verdict is Conditional, and my concern reinforces that condition rather than changing it, I recommend UNCHANGED. My disagreement with the reader is on which weakness is most load-bearing: I argue the unablated relation-aware contribution is more central to the paper's scientific claim than the temporal resizing choice.","tokens_in":3648,"tokens_out":14728,"duration_ms":140597,"concrete_test":"Run the identical training pipeline described in Section 2 (same anchors, loss, data, and hyperparameters) but with the self-attention modules removed, leaving a plain 1D FPN. Evaluate on the ActivityNet v1.3 validation split and report AR@AN and AUC. If the full RapNet AUC is within run-to-run variance (e.g., less than 0.2%) of the plain FPN, then the relation-aware component does not contribute to the reported 69.61% single-model AUC, undermining the paper's central causal claim that RapNet's relation-aware design helped achieve 2nd place.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims that the proposed Relation-Aware Pyramid Network (RapNet) 'helps us achieve 2nd place' in ActivityNet Challenge 2019 Task 1, and Section 2 presents the self-attention modules and 1D FPN as the core of the method. However, the only experimental ablation in the paper (Table 2) compares APG, +PEM, and +TAG; it never removes the self-attention modules or the FPN to measure their individual contributions. As a result, the 69.61% single-model AUC for APG could be produced by the anchor design, the ResNet-C3D features, or the FPN alone, while the supposedly novel relation-aware components may be adding little or nothing. If the relation-aware modules are inert, the paper's central claim that the proposed network is responsible for the competitive performance is unsupported, even if the reported benchmark numbers are authentic. The fixed 128-snippet resizing mentioned in the reader's verdict is a design choice that could affect absolute performance, but it does not challenge the attribution of the result to the proposed architecture; the missing ablation does.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a technical report describing the authors' submission to the ActivityNet Challenge 2019 temporal action proposal task. The proposed system, Relation-Aware Pyramid Network (RapNet), extracts snippet-level features with a fine-tuned ResNet-50/101 C3D backbone, resizes all videos to a fixed 128 x 256 feature map, and generates multiscale anchor-based proposals using a temporal feature pyramid network enhanced with self-attention modules. A two-stage boundary adjustment scheme (refined PEM followed by soft-NMS and watershed-based TAG refinement) is applied, and an ensemble over two backbones and two anchor configurations is used. The authors report 70.65% validation AUC for the single model with boundary adjustment, 71.51% validation AUC after ensemble, and 71.38% test AUC, claiming this result helped them achieve 2nd place in the challenge.","tokens_in":3918,"tokens_out":2503,"duration_ms":26640,"significance":"If the reported numbers hold, the system is clearly competitive at the top of a well-known benchmark, and the external test-server evaluation is a strength that rules out adaptation to the test set. The paper also provides an incremental ablation of the boundary-adjustment components (Table 2), and the feature-extraction and proposal-generation pipeline is described in enough detail to be broadly replicable. However, the central novelty claimed in the title and Section 2, namely the relation-aware (self-attention) and pyramid (FPN) design, is never isolated in the experiments, so the paper does not currently establish that its proposed architecture, rather than the anchor design, features, or post-processing, is responsible for the reported performance. The fixed-length resizing to 128 snippets is also a major design choice without supporting analysis. The paper is best assessed as an incomplete technical report rather than a fully supported research contribution.","major_comments":[{"comment":"The only ablation in the paper, Table 2, compares APG, +PEM, and +TAG, but it never removes the self-attention modules or the 1D FPN from the APG baseline. Since Sections 2 and Figure 1 present self-attention and FPN as the core of RapNet, the reported 69.61% AUC for APG cannot be attributed to these components; a baseline with the same anchors and feature map but without attention and FPN (e.g., plain temporal convolutions) is needed. Without such an ablation, the paper's central claim that RapNet's relation-aware design is responsible for the competitive result is unsupported.","section":"Sec. 2 and Table 2"},{"comment":"The fixed resizing of all videos to 128 snippets is a load-bearing preprocessing choice that is not analyzed. The manuscript gives no evidence that compressing arbitrarily long or short videos to 128 snippets preserves the boundary information that the proposal network and the boundary-adjustment stages rely on. A comparison with at least one other temporal length (e.g., 256 snippets) or a breakdown of recall by action duration would be necessary to rule out that this resizing degrades performance for a significant subset of actions.","section":"Sec. 1"},{"comment":"The paper provides no comparison with state-of-the-art methods on ActivityNet v1.3 validation (e.g., BSN, BMN, CTAP) and no error bars or number of runs. Consequently, the improvements attributed to boundary adjustment (0.74% and then 0.30% AUC in Table 2) could be within run-to-run variance, and the absolute values cannot be contextualized against published results. Reporting standard deviations over multiple training runs and a table of prior published AUC and AR@AN numbers is necessary to support the claimed significance of the increments.","section":"Secs. 3 and 4 (overall evaluation)"}],"minor_comments":[{"comment":"There are typos such as \"totem-poral\" in the abstract and \"proposasl\" in Section 1; the manuscript would benefit from a proofreading pass.","section":"Abstract and Sec. 1"},{"comment":"The loss notation is difficult to follow: the symbols T, M, N, and the index ranges (especially the T/2^i term) are not defined explicitly, and the relationship between the pyramid levels and the summation bounds is left implicit.","section":"Sec. 2, Eq. (1)"},{"comment":"The sentence \"In experiment, we find 12 anchors achieve the best performance\" reports a selection made on the validation set without specifying the search range or the criterion; this is a free parameter that should be documented for reproducibility.","section":"Sec. 2"},{"comment":"The typo \"degisn\" and the phrase \"with frames actioness\" obscure the description of the boundary-adjustment scheme; please clarify which features the PEM and watershed stages operate on.","section":"Sec. 3"},{"comment":"The ensemble description does not specify how the two backbone outputs and two anchor configurations are fused (e.g., score averaging, proposal merging, NMS thresholds), which is needed for reproducibility.","section":"Sec. 4"}],"recommendation":"major_revision","confidential_remarks":"This is a challenge technical report rather than a full research paper, and its contribution is largely incremental. The main risk is that the reported numbers are not backed by ablations of the novel components, making the title claim unsupported. If the authors can add the missing ablations and comparisons, the paper could become a useful benchmark report; in its current form it is not ready for publication in a serious venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a short technical report from the ActivityNet 2019 challenge, not a research paper, and it should be read that way. The system itself is a reasonable composition of known pieces: fine-tuned ResNet-C3D features, YOLO-style anchors, a temporal FPN with self-attention, BSN's PEM, TAG, and soft-NMS. What is new is the specific combination and the empirical result: 71.51% validation AUC and 71.38% on the challenge test server, with a claimed 2nd place. The numbers in Table 2 are internally consistent: each added post-processing component improves AR and AUC, which suggests the pipeline was not tuned to the test set. The paper also gives enough detail on anchors, loss weights, and label assignment that a reader could reproduce the approach in broad terms.\n\nThe main weakness is exactly where the stress-test note lands. The abstract and Section 2 sell RapNet as a relation-aware proposal network, with self-attention on the top-down path and a 1D FPN on the bottom-up path. But Table 2 only ablates APG, +PEM, and +TAG. It never removes the self-attention or the FPN, so 69.61% could come from the anchor design, the features, or the FPN while the self-attention adds little or nothing. That is a real gap. It is not a reason to call the benchmark numbers false; it is a reason to say the central attribution claim is unsupported by the evidence presented. If I had to guess, the relation-aware modules probably do help a bit, but the paper doesn't let me defend that bet.\n\nOther soft spots are smaller. There are no error bars, no training details like batch size, learning rate, epoch count, or snippet length L, and no comparison against published state-of-the-art methods (BMN, DBG, etc.). The fixed 128-snippet resizing is a design choice that could hurt performance on very long or very short actions, but the paper never claims to be optimal on that axis, so it is a minor concern. The test result comes from an external server, which is good; the lack of released code means the result is not independently re-runnable, but for a challenge report that is par for the course.\n\nOverall, this is a solid, useful technical report for someone building a temporal action proposal pipeline out of existing components. It is not a conceptual breakthrough and it is not a proof that relation-aware modules work. A serious editor would not desk-reject it outright if the venue is a workshop or a challenge-focused track; for a top conference or journal, the missing ablation alone would force major revision or rejection. I would bring it to a reading group as an example of an effective challenge submission, and I would encourage the authors to release code and run the missing ablation, but I would not cite it as evidence for the efficacy of self-attention in temporal proposals without that follow-up.","headline":"A competent, internally consistent challenge report whose named novelty (self-attention + FPN) is never ablated, so the reported gains cannot be attributed to the proposed design.","tokens_in":4415,"tokens_out":1645,"would_cite":false,"duration_ms":20860,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a temporal action proposal pipeline built around RapNet—a relation-aware pyramid network with self-attention and FPN—plus a two-stage boundary adjustment scheme and ensemble, reaches 71.51% validation and 71.38%…","keywords":["temporal action proposal","RapNet","self-attention","feature pyramid network","boundary adjustment","ActivityNet Challenge","video understanding","ensemble"],"falsifier":"Reproduce the recipe on ActivityNet v1.3 with the same train/validation split and compute AR@AN and AUC; if the single RapNet model does not approach the reported 69.61% validation AUC, or if the two-stage boundary adjustment does not add roughly one point of AUC, the central claim is not supported. A targeted test would bypass fixed-length resizing and feed variable-length representations to see whether the 128-snippet compression is hurting boundary recall.","tokens_in":3490,"feed_emoji":"🎬","tokens_out":6243,"duration_ms":59668,"temperature":0.7,"pith_summary":"The paper is a competition report for temporal action proposal, the task of generating candidate time intervals that likely contain actions. Its central claim is that a specific pipeline—RapNet's multiscale anchor-based proposals, followed by a two-stage boundary adjustment, then ensembling over two video encoders and two anchor sets—is a top-performing solution: the authors report 71.51% validation AUC and 71.38% test AUC, which they say placed second in ActivityNet Challenge 2019 Task 1. A sympathetic reader should care because temporal action proposals are the front end of video understanding systems, and the reported numbers are on a widely used benchmark with a public leaderboard.","feed_headline":"RapNet hits 71.38% test AUC and takes second in ActivityNet 2019","feed_subtitle":"Self-attention, a feature pyramid, and two-stage boundary adjustment make the difference.","key_machinery":"The central mechanism is RapNet, a temporal pyramid proposal generator: a 1D feature-pyramid network with six generators at different temporal scales, where self-attention is inserted on the top-down path to capture long-range snippet relationships and FPN on the bottom-up path supplies multiscale features. Anchors are chosen by K-means clustering over ground-truth durations, and each anchor predicts confidence, center and width, and IoU against ground truth. Proposal boundaries are then adjusted by a two-stage scheme using BSN's PEM module with frame-level actionness and TAG's watershed algorithm; redundant proposals are suppressed with soft-NMS.","core_discovery":"The paper reports that combining a temporal pyramid network with self-attention and FPN generates strong multiscale proposals: on ActivityNet v1.3 validation, RapNet alone gives 69.61% AUC, adding the PEM boundary-refinement stage brings it to 70.35%, and the further TAG watershed adjustment reaches 70.65% AUC. Ensembling ResNet-50 and ResNet-101 C3D backbones with 12 and 18 anchors raises the result to 71.51% validation AUC and 71.38% test-server AUC, which the authors state is second place in ActivityNet Challenge 2019 Task 1. The paper treats each boundary-adjustment stage as a measurable improvement and presents the ensemble as the final contribution.","pith_inferences":["The paper does not ablate self-attention, FPN, or anchor count separately, so a reader cannot attribute the AUC gains to any single RapNet component; a clean ablation is the natural next experiment.","The fixed 128-snippet representation is the most fragile part of the recipe; a variable-length or multi-resolution representation might improve boundary recall, especially for very short actions.","If the roughly one-point AUC gain from boundary adjustment is generic rather than specific to RapNet, it would transfer to other anchor-based proposal generators; that is testable by swapping base generators.","Because the test number is a single leaderboard evaluation from a competition, repeated training runs and multiple seeds would be needed to know the method's variance."],"forward_implications":["On the authors' reported numbers, the single-model pipeline already reaches 69.61% AUC, so the architecture is competitive before any ensembling.","Boundary adjustment is worth about one AUC point: PEM adds 0.74 points and the TAG watershed stage adds another 0.30 points.","Ensembling the two C3D backbones and two anchor sets adds about 0.86 AUC points over the single refined system, reaching 71.51% on validation.","The proposal generator and boundary-refinement stages are separable, meaning each component could be grafted onto other action detection systems independently."],"supporting_citations":[{"why":"Supplies soft-NMS, the redundant-proposal suppression step in the boundary adjustment scheme.","marker":"[1]"},{"why":"Provides Kinetics-pretrained weights that initialize the C3D backbone before fine-tuning on ActivityNet.","marker":"[2]"},{"why":"Provides ImageNet-pretrained weights used as the starting point for the ResNet backbone.","marker":"[3]"},{"why":"Supplies the ResNet residual architecture underlying the C3D video encoder.","marker":"[4]"},{"why":"Supplies the PEM boundary-refinement module and the frame-level actionness framing used in the first refinement stage.","marker":"[5]"},{"why":"Supplies the 1D feature pyramid network used on the bottom-up path of RapNet for multiscale proposals.","marker":"[6]"},{"why":"Supplies the anchor-based labeling and negative-instance screening strategy adapted for proposal prediction.","marker":"[7]"},{"why":"Supplies the C3D architecture that encodes snippet-level video representations.","marker":"[8]"},{"why":"Supplies the self-attention module inserted on RapNet's top-down path to model long-range snippet dependencies.","marker":"[9]"},{"why":"Supplies the watershed algorithm used with frame actionness to adjust proposal boundaries in the second refinement stage.","marker":"[10]"}],"fun_headline_variants":["RapNet takes 2nd in ActivityNet 2019 with 71.38% test AUC","Relation-aware pyramid nets hit 71.38% AUC for action proposals","RapNet: self-attention pyramid nets rank 2nd in ActivityNet 2019","RapNet achieves 71.38% AUC, second in ActivityNet 2019"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that resizing every video to a fixed 128-snippet representation preserves the temporal boundary cues needed to localize actions whose lengths vary widely.","fun_headline_variants_meta":{"raw":{"variants":["RapNet takes 2nd in ActivityNet 2019 with 71.38% test AUC","Relation-aware pyramid nets hit 71.38% AUC for action proposals","RapNet: self-attention pyramid nets rank 2nd in ActivityNet 2019","RapNet achieves 71.38% AUC, second in ActivityNet 2019"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000869,"raw_usage":{"total_tokens":3692,"prompt_tokens":804,"completion_tokens":2888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":420,"completion_tokens_details":{"reasoning_tokens":2793}},"tokens_in":420,"tokens_out":2888,"duration_ms":21282,"temperature":1.0,"reasoning_tokens":2793,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:12:25.217176+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the recipe on ActivityNet v1.3 with the same train/validation split and compute AR@AN and AUC; if the single RapNet model does not approach the reported 69.61% validation AUC, or if the two-stage boundary adjustment does not add roughly one point of AUC, the central claim is not supported. A targeted test would bypass fixed-length resizing and feed variable-length representations to see whether the 128-snippet compression is hurting boundary recall.","supporting_citations":[{"cited_title":"Bodla, B","cited_arxiv_id":null,"evidence_quote":"Supplies soft-NMS, the redundant-proposal suppression step in the boundary adjustment scheme."},{"cited_title":"Carreira and A","cited_arxiv_id":null,"evidence_quote":"Provides Kinetics-pretrained weights that initialize the C3D backbone before fine-tuning on ActivityNet."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides ImageNet-pretrained weights used as the starting point for the ResNet backbone."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the PEM boundary-refinement module and the frame-level actionness framing used in the first refinement stage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 1D feature pyramid network used on the bottom-up path of RapNet for multiscale proposals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the C3D architecture that encodes snippet-level video representations."},{"cited_title":"Vaswani, N","cited_arxiv_id":null,"evidence_quote":"Supplies the self-attention module inserted on RapNet's top-down path to model long-range snippet dependencies."}],"review_version":1}