{"id":"ab6dbe6a-8992-4be2-9beb-710c0ee4ac2f","arxiv_id":"1908.00707","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A multi-branch network with multi-dilation temporal convolutions and start, middle, and end point detection sets new state-of-the-art results for temporal action localization on THUMOS14 and ActivityNet-1.3.","lead":"This paper presents TSA-Net, a network that pinpoints when actions start and end in untrimmed videos by combining information across multiple time scales. It reports the best known accuracy on two standard benchmarks, suggesting better video search and summarization tools are within reach.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The THUMOS14 state-of-the-art claim is confounded by simultaneous changes in features and classifier; the architecture itself has more controlled support, so the verdict stays conditional.","rationale":"The reader's weakest_assumption identifies the cross-feature comparison as the load-bearing issue, and I agree that it is real. My stress-test adds that the classifier choice is a second, equally large confound: Table 7 shows classifier choice alone swings the reported number by more than 10 mAP points for the same proposal generator. This makes the headline 'new record' even less attributable to the architecture than the reader's feature-only framing suggests. However, the paper does contain a same-feature, same-classifier comparison on THUMOS14 (Ours vs BSN with two-stream features and UNet) and a closely matched ActivityNet comparison, both of which favor TSA-Net. Therefore the core architectural claim is not hollow; it is the specific state-of-the-art record that is overstated. Since the central claim can still be rescued by a controlled comparison, and since the paper also lacks code release and variance estimates, the appropriate verdict remains CONDITIONAL. My read does not move the reader's verdict, hence UNCHANGED.","tokens_in":14184,"tokens_out":4791,"duration_ms":46363,"concrete_test":"Reproduce Table 7 with the BSN proposal generator using the same P3D features and the same UNet classifier on THUMOS14, and compare its mAP@0.5 to TSA-Net(P3D, UNet). If BSN+P3D+UNet reaches or exceeds roughly 42.8 or comes within 1-2 points of 46.9, then the headline margin over TAL-Net is mostly feature/classifier gain, and the state-of-the-art claim should be restated as feature-matched only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that TSA-Net 're-calibrates new state-of-the-art' rests on the 46.9% THUMOS14 mAP@0.5 in Table 7. That number uses P3D features and the UntrimmedNet (UNet) classifier, whereas the quoted previous best of 42.8% (TAL-Net) uses I3D features and its own classifier. Two variables change at once, so the 4.1-point gap cannot be attributed to the proposed MDC/multi-branch architecture. The paper itself shows these confounds are large: switching the classifier from SCNN-cls to UNet changes Ours(P3D) from 36.6 to 46.9 at mAP@0.5, and switching features from two-stream to P3D changes Ours(UNet) from 41.5 to 46.9. A same-feature/same-classifier comparison that does exist (two-stream + UNet: Ours 41.5 vs BSN 36.9) supports the architecture, but it is not the headline claim. The ActivityNet result (Ours 34.62 vs BSN 32.84) is much more closely matched and is consistent, but the THUMOS14 'record' overstates what is attributable to TSA-Net.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes TSA-Net, a two-stage temporal action localization method. Its proposal-generation stage detects starting, mid, and ending critical points using multi-dilation temporal convolution (MDC) blocks arranged in three branches with different dilation rates, followed by average pooling; proposals are formed by pairing starting/ending points with mid-point filtering and scored by a learned compatibility function in a Bayesian-style product. The second stage reuses existing classifiers (UNet, SCNN-cls, and the classifier of [55]). The authors evaluate on THUMOS14 and ActivityNet-1.3 and claim state-of-the-art results, e.g., 46.9% mAP@0.5 on THUMOS14 versus 42.8% for the previous best.","tokens_in":14492,"tokens_out":6480,"duration_ms":58867,"significance":"The MDC block and multi-branch scale aggregation are simple and well-motivated, and the controlled experiments (two-stream features, same classifier) show consistent gains over BSN on both proposal generation (e.g., AR@50 from 37.46 to 42.83 with Soft-NMS) and final localization (mAP@0.5 from 36.9 to 41.5 with UNet). The ActivityNet comparison is also controlled and positive. If the reported gains are reproducible, the paper makes a useful contribution to temporal action localization. However, the headline claim of a new state-of-the-art record on THUMOS14 is not supported by a matched comparison, and some internal inconsistencies and missing statistical details reduce confidence.","major_comments":[{"comment":"The claim that TSA-Net 're-calibrates new state-of-the-art' on THUMOS14 is based on a comparison that changes two variables at once. The 46.9% mAP@0.5 is obtained with P3D features and the UNet classifier, whereas the quoted previous best of 42.8% (TAL-Net) uses I3D features and its own classifier. The same table shows a controlled comparison (Ours Two-Stream + UNet 41.5 vs BSN + UNet 36.9) that supports the architecture, but the headline margin of 4.1 points overstates the contribution. Please either compare with the same features/classifier or rephrase the state-of-the-art claim to identify the matched baselines.","section":"Abstract, §4.5, Table 7"},{"comment":"The AR values for the same configuration are inconsistent across tables. Table 1 reports Ours+Greedy-NMS at AR@50 as 41.40 and Ours+Soft-NMS as 42.83, while Tables 4-6 repeatedly report TSA-Net/multi-branch at AR@50 as 40.83. The text says 'Bayesian scores are used to rank proposals' in §4.4, but Table 6 indicates that the 40.83 value corresponds to the product P(s)P(e) without the compatibility function, while the full Bayesian score including φ gives 42.83. Please clarify precisely which scoring and NMS variant is used in each table, as the current presentation makes the ablation numbers difficult to reconcile.","section":"§4.3, §4.4, Tables 1, 4, 5, 6"},{"comment":"No variance estimates or multiple-seed results are provided. The ActivityNet gains are small (AUC 67.01 vs 66.26 on the test set; average mAP 34.62 vs 32.84), and without repeated runs or error bars it is not possible to assess whether the 'clear and consistent' improvement is statistically meaningful. At minimum, report mean and standard deviation over several training runs for the main comparisons.","section":"§4.2, Tables 2 and 8"}],"minor_comments":[{"comment":"The phrase 'sequence-to-sequence learning' and the citation to [20] are not illuminating; the model is a per-frame classification network. Consider removing or rewriting the sentence.","section":"§3.2"},{"comment":"Eq. (2) is called a Bayesian formulation, but it is a product of independently estimated probabilities multiplied by a learned compatibility score; please state that it is a heuristic scoring function.","section":"§3.3, Eq. (2)"},{"comment":"Table 7 lists IoU thresholds in descending order (0.7, 0.6, ..., 0.3); consider ordering them ascending for consistency with the text (0.3 to 0.7).","section":"Table 7"},{"comment":"The frame sampling strides (5 on THUMOS14, 16 on ActivityNet) are introduced as choices balancing complexity and fidelity, but no sensitivity analysis is given.","section":"§4.1"},{"comment":"Minor typos: 'staring' in the introduction; the phrase 'starting / mid-point / ending' appears with inconsistent hyphens in the abstract.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's controlled comparisons are the strongest evidence, and the architecture is plausible. The main issue is the overclaiming of the THUMOS14 state-of-the-art record based on a features-and-classifier confound. The inconsistency in AR numbers across tables should be resolved before publication. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the TSA-Net architecture is genuinely reasonable: MDC blocks with multiple dilation rates, multiple branches for different duration scales, and start/mid/end point detection with a learned compatibility function. The ablations on THUMOS14 with matching two-stream features show each piece earns its keep—MDC beats plain conv by about 9 points AR@50, multi-branch beats every single branch, and the compatibility function adds roughly 2 points. Second, the headline claim of a new state of the art on THUMOS14 (46.9% vs 42.8%) is not a like-for-like comparison. Ours uses P3D features and the UNet classifier; the quoted prior best (TAL-Net) uses I3D and its own classifier. The paper's own ablations show that swapping the classifier alone moves Ours(P3D) from 36.6 to 46.9, so you cannot attribute that gap to the temporal scale aggregation. This is a framing problem, not a fatal one: the same-feature comparison (two-stream + UNet: 41.5 vs BSN 36.9) and the ActivityNet comparison (34.62 vs 32.84, same features and classifier) are much cleaner and support the central claim that the architecture improves proposal quality and localization.\n\nWhat the paper does well is honest engineering. The design is clearly motivated, the ablations isolate the components, and the authors acknowledge they reuse existing classifiers to focus on proposal generation. What is missing is the thing that would make the headline trustworthy: a feature-matched head-to-head table at the state-of-the-art level, plus error bars or multiple seeds. No code is released, and several hyperparameters (threshold 0.9, delta 0.1, dilation sets) are fixed without sensitivity analysis. Those are standard weaknesses for a 2019 paper, but they still matter for reproducibility.\n\nWho is this for? Anyone working on temporal action localization or proposal generation will find the multi-dilation block and three-point pairing scheme useful, and the ablation study is worth reading on its own. The paper deserves a serious referee; the core method appears sound and the controlled comparisons are credible. My recommendation: send it to review, and ask the authors to add a feature-matched comparison for the headline result, report variance, and release code. With those changes the claims would match the evidence.","headline":"The architecture is solid and the controlled comparisons support it, but the headline THUMOS14 record compares P3D+UNet against an I3D baseline and overstates what TSA-Net alone delivers.","tokens_in":658,"tokens_out":936,"would_cite":true,"duration_ms":26446,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TSA-Net claims that ensembling temporal convolutions with different dilation rates—organized into branches tuned to short, middle, and long actions—plus start/mid/end point pairing, sets new state-of-the-art action localization results on…","keywords":["temporal action localization","untrimmed videos","action proposals","multi-dilation temporal convolution","boundary point detection","temporal scale aggregation","THUMOS14","ActivityNet-1.3"],"falsifier":"Train and evaluate TSA-Net on THUMOS14 with I3D features and compare mAP@0.5 with the prior best 42.8%; if the gap nearly disappears, the headline record is feature-driven rather than caused by the multi-scale architecture.","tokens_in":13996,"feed_emoji":"🎬","tokens_out":12321,"duration_ms":101543,"temperature":0.7,"pith_summary":"The paper sets out to show that temporal action localization in untrimmed videos is improved by making the network explicitly aware of action duration scale. Its proposal generator, TSA-Net, is built from multi-dilation temporal convolution (MDC) blocks—parallel 1-D convolutions with different dilation rates whose outputs are averaged—stacked in three branches tuned to short, middle, and long actions, and it detects three kinds of critical points (start, midpoint, end) instead of the usual two. On THUMOS14 the paper reports mAP@0.5 of 46.9% against a previous best of 42.8%, and on ActivityNet-1.3 average mAP rises from 32.84% to 34.62%. If the claim is right, boundary imprecision, a known bottleneck of two-stage localization, can be reduced without expensive larger kernels or video pyramids.","feed_headline":"46.9% mAP: multi-scale network sets action localization record","feed_subtitle":"Duration-tuned dilated filters and start–mid–end pairing beat prior bests on THUMOS14 and ActivityNet-1.3.","key_machinery":"The multi-dilation temporal convolution (MDC) block is the core object: within one block, three 1-D temporal convolutions share a kernel size but use dilation rates $d_1 < d_2 < d_3$, their outputs are averaged, and a skip connection is added so the block learns a residual. Two MDC blocks are stacked in each of three branches with dilation schedules (1,2,3), (1,3,5), and (1,5,7), giving receptive fields of 13 to 29 snippets, so each branch is receptive to a different action-duration scale. Branch outputs are average-pooled before a small predictor emits start, midpoint, and end probability sequences; proposals are formed by pairing start and end points, gated by the midpoint confidence, and ranked by a learned Bayesian compatibility function that scores a proposal from 96 sampled values of the three probability sequences.","core_discovery":"The paper's central claim is that boundary precision in two-stage temporal action localization is limited by a fixed temporal receptive field, and that ensembling dilation rates fixes it. TSA-Net detects three kinds of critical points—starting, midpoint, and ending—and pairs start/end candidates only when the detected midpoint is also confident, which the authors argue removes low-accuracy proposals that two-point boundary methods produce. The proposal generator uses parallel branches of stacked MDC blocks with different dilation schedules, each branch specialized to a duration range, and merges their outputs by average pooling before predicting critical-point probability sequences. On identical two-stream features, TSA-Net improves THUMOS14 AR@50 from 37.46% to 42.83% and mAP@0.5 from 36.9% to 41.5%; with P3D features it reports 46.9% mAP@0.5, and on ActivityNet-1.3 it raises average mAP from 32.84% to 34.62%.","pith_inferences":["The three-point design suggests a direct extension to streaming localization: a confident midpoint could act as a trigger, with a proposal committed only when a later ending point lands, reducing false positives in live video monitoring.","The paper only tests fixed dilation schedules; a natural next probe is to learn branch dilation rates or branch weights per video, to see whether the fixed schedules are near-optimal or merely adequate.","The same scale-aggregation reasoning may transfer to temporal action segmentation and dense video captioning, where boundary precision is also the limiting factor and where MDC blocks could replace larger kernels with cheaper dilated ones."],"forward_implications":["Higher precision at the same proposal budget: on THUMOS14 with two-stream features, AR@50 rises from 37.46% to 42.83%, so a video system can retrieve more true action segments with fewer candidate proposals.","Both short and long actions gain: the duration-stratified evaluation reports improvement on every quintile of action durations, not just the middle range.","Unseen classes transfer: on ActivityNet-1.3, AUC and AR@100 drop only slightly for action classes never seen in training, suggesting the scale-aggregation mechanism is not class-specific.","The MDC block is the source of the gain: replacing MDC blocks with parameter-matched standard 1-D convolutions drops AR@50 from 40.83 to 31.48, and any single dilation branch underperforms the three-branch ensemble."],"supporting_citations":[{"why":"Supplies the boundary point detection and Bayesian proposal-scoring formulation that TSA-Net extends by adding a mid-point and multi-scale branches.","marker":"[28]"},{"why":"Reports the previous best THUMOS14 mAP@0.5 (42.8%) used as the headline baseline and already applied dilated temporal convolutions to localization.","marker":"[6]"},{"why":"Contributes the structured start-course-end proposal decomposition and the compatibility-feature construction used for pairwise scoring.","marker":"[54]"},{"why":"Provides the SCNN classifier used in THUMOS14 comparisons and one of the proposal-generation baselines TSA-Net is measured against.","marker":"[36]"},{"why":"Provides the UntrimmedNet classifier used with TSA-Net proposals for THUMOS14 action classification results.","marker":"[43]"},{"why":"Provides the P3D features used for the top-line THUMOS14 results, including the 46.9% mAP@0.5 record.","marker":"[31]"},{"why":"Provides the two-stream snippet features on which the same-feature comparisons between TSA-Net and prior methods are run.","marker":"[45]"},{"why":"Defines the THUMOS14 benchmark and its temporal annotations, the primary testbed for the paper's claims.","marker":"[19]"},{"why":"Defines ActivityNet-1.3, the second benchmark where TSA-Net reports improved average mAP (34.62% vs 32.84%).","marker":"[16]"}],"fun_headline_variants":["46.9% mAP: TSA-Net's scale aggregation sharpens action boundaries","Three-point pairing: key to TSA-Net's 46.9% action localization record","Multi-dilation network improves action localization to 46.9% mAP","TSA-Net sets state-of-the-art on THUMOS14 and ActivityNet-1.3","Start, mid, end: TSA-Net's recipe for precise action localization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline comparison assumes that a result obtained with P3D features can be fairly compared with a previous result obtained with I3D features, so the claimed margin could reflect feature differences rather than the network alone.","fun_headline_variants_meta":{"raw":{"variants":["46.9% mAP: TSA-Net's scale aggregation sharpens action boundaries","Three-point pairing: key to TSA-Net's 46.9% action localization record","Multi-dilation network improves action localization to 46.9% mAP","TSA-Net sets state-of-the-art on THUMOS14 and ActivityNet-1.3","Start, mid, end: TSA-Net's recipe for precise action localization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001237,"raw_usage":{"total_tokens":5113,"prompt_tokens":1017,"completion_tokens":4096,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":3995}},"tokens_in":633,"tokens_out":4096,"duration_ms":29262,"temperature":1.0,"reasoning_tokens":3995,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:36:10.766875+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train and evaluate TSA-Net on THUMOS14 with I3D features and compare mAP@0.5 with the prior best 42.8%; if the gap nearly disappears, the headline record is feature-driven rather than caused by the multi-scale architecture.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the boundary point detection and Bayesian proposal-scoring formulation that TSA-Net extends by adding a mid-point and multi-scale branches."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports the previous best THUMOS14 mAP@0.5 (42.8%) used as the headline baseline and already applied dilated temporal convolutions to localization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the structured start-course-end proposal decomposition and the compatibility-feature construction used for pairwise scoring."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SCNN classifier used in THUMOS14 comparisons and one of the proposal-generation baselines TSA-Net is measured against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the UntrimmedNet classifier used with TSA-Net proposals for THUMOS14 action classification results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the P3D features used for the top-line THUMOS14 results, including the 46.9% mAP@0.5 record."},{"cited_title":"Idrees, A","cited_arxiv_id":null,"evidence_quote":"Defines the THUMOS14 benchmark and its temporal annotations, the primary testbed for the paper's claims."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines ActivityNet-1.3, the second benchmark where TSA-Net reports improved average mAP (34.62% vs 32.84%)."}],"review_version":1}