{"id":"2fbe1099-c882-4124-9745-40cbe7863e9a","arxiv_id":"2506.13320","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A video-only method, TA2Net, localizes audible collision frames on a new Audible623 benchmark by detecting sudden motion changes, achieving F1 0.616 and zero-shot transfer to repetitive counting.","lead":"This paper introduces a new computer vision task: finding the exact frames in a silent video where objects collide or make a sound, using only visual motion cues. It also releases a new benchmark dataset and a model, TA2Net, that reports strong results for timing these audible actions and can transfer to counting repeated actions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Audible623 labels are produced by silent visual inspection and exclude videos whose audible frames cannot be visually determined, so the benchmark bakes in the inflection prior and likely overstates how well TA2Net localizes genuinely audible actions.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the dataset labels come from silent visual inspection, and visually undeterminable videos are excluded. I agree with that assessment. The concern is not an internal inconsistency in the method but a construct-validity problem: the benchmark's annotation protocol operationalizes 'audible action' as 'visually identifiable collision-like motion', which is precisely the signal TA2Net is designed to exploit. The paper does provide real independent support: the code and dataset are released, ablations show the proposed components help, and the zero-shot repetitive-counting results on UCFRep and CountixAV are plausible falsifiable evidence that inflectional flow captures event timing. However, those results do not settle whether the frames localized are actually the ones that produce sound. The decisive missing check is audio-based validation of the labels. Without it, the headline performance numbers may not transfer to real dubbing scenarios, where audible actions are not guaranteed to be visually salient and silent visible collisions are common. The appropriate verdict remains conditional: the method is promising and reproducible, but the central empirical claim should be re-scoped or re-validated before being accepted at face value.","tokens_in":16071,"tokens_out":4373,"duration_ms":52434,"concrete_test":"Pick 100 Audible623 test videos and retrieve their original audio from Kinetics, UCF101, or YouTube. Have three annotators listen to the audio (without video) and mark frames containing a new audible impact; compute agreement with the published visual-only keyframes within ±2 frames. Also run the same listening protocol on 50 videos that were excluded by Audible623's visual screening. If frame-level agreement is below 0.8, or if excluded videos frequently contain clearly audio-localizable impacts, then the silent-visual annotation protocol has materially distorted the benchmark and the headline F1 should not be interpreted as audible-action localization accuracy. A cheaper automated cross-check is to run an audio onset detector (e.g., YAMNet or SuperFlux) on the original tracks and measure alignment with the keyframes, but human listening is the definitive check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — that TA2Net localizes audible action frames in silent video using inflectional flow — rests on Audible623 as ground truth. Section 3 states that annotators watch videos without audio and exclude videos if the temporal location of audible frames cannot be visually determined. This is the decisive weak point. Without audio, annotators cannot distinguish an audible collision from a silent or quiet one; every visually salient collision can be labeled 'audible', and any sound that lacks a clear visible inflection is excluded. The resulting label set is therefore a human estimate of visible collision timing, not an independent measurement of audibility. TA2Net's input is optical flow and its prior is a second derivative of motion, so the evaluation is confined to exactly the cases the prior is designed to detect. The reported F1 0.616, NME 3.462, and PME 0.744 are upper-envelope numbers for a pre-filtered, visually discriminable subset. The zero-shot counting results on UCFRep and CountixAV are genuine supporting evidence that inflectional flow captures repetitive motion, but they do not validate the audibility claim, because those datasets measure repetition, not sound production. The paper's own limitation section (Appendix F) only discusses optical-flow inaccuracy, not label construct validity. A benchmark whose construction guarantees visual discriminability cannot falsify the sound-from-collision premise, yet that falsifiability is needed for the claimed generalization to dubbing. This is a correctness risk, not a disagreement with consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Audible Action Temporal Localization (AATL), a task aimed at predicting frame-level timings of audible, collision-like actions in silent video. The authors propose TA2Net, which estimates 'inflectional flow' as the finite difference of optical flow across adjacent frames, uses a cross-kinematics attention module to combine image, motion, and inflection features, and adds a self-supervised spatial auxiliary loss with inter-video contrastive and intra-video smoothness terms. To train and evaluate the method, the authors collect Audible623, a new dataset of 623 videos with frame-level keyframe labels for audible actions. Experiments report state-of-the-art F1, NME, and PME on Audible623, zero-shot repetitive counting results on UCFRep and CountixAV, and qualitative spatial localization results. The central modeling premise is that audible collisions are typically associated with abrupt visual velocity changes, i.e., visual inflections.","tokens_in":16320,"tokens_out":6163,"duration_ms":63222,"significance":"The proposed task is novel and potentially useful for automated dubbing and video editing, and the inflectional-flow prior is a simple, physically motivated cue that goes beyond standard action recognition. The authors release code and data, provide ablations showing the contribution of each component, and demonstrate transfer to repetitive counting, which gives some evidence that the acceleration cue generalizes to periodic motion. The main risk is construct validity of the benchmark: if the ground truth is defined by human visual judgment of silent videos, then the evaluation may reward models that reproduce the annotators' reliance on visual motion inflections rather than models that localize genuinely audible events. The reported numbers should therefore be interpreted with caution, and the paper's central claims need stronger validation.","major_comments":[{"comment":"The ground-truth labels are produced by volunteers who watch videos without audio and are asked to visually determine the temporal location of audible frames; videos for which this cannot be done visually are excluded. This protocol makes the labels a measure of visual salience or collision visibility, not an independent measurement of audibility. Because TA2Net's input is optical flow and its prior is a second-order motion change, the evaluation is largely confined to the cases the prior is designed to detect, creating a partial circularity. The reported F1=0.616, NME=3.462, and PME=0.744 may therefore overstate the method's ability to localize genuinely audible events. The authors should either add an audio-based validation subset (e.g., videos with verified sound tracks where labels are checked against audio), report inter-annotator agreement and analyze the excluded videos, or explicitly reframe the task as 'temporal localization of visually detectable collisions' and adjust the title, abstract, and claims accordingly. The limitation section (Appendix F) discusses only optical-flow inaccuracy and does not address this label-construction issue.","section":"§3, Dataset Annotation"},{"comment":"All quantitative results are reported as single runs without error bars or statistical significance tests. Since the test set contains only 126 videos, the differences between the proposed method and the best baseline (e.g., F1 0.616 vs. 0.553 for TransRAC) may be within run-to-run variance. The authors should report mean and standard deviation over multiple random seeds, or at least provide a significance test, before claiming state-of-the-art performance on the new benchmark.","section":"§5, Tables 2–4"},{"comment":"The zero-shot counting results on UCFRep and CountixAV are presented as evidence of generalizability, but these datasets provide repetition counts, not labels for audible events. The CountixAV filtering criterion ('clear sound or background noise') does not turn repetition counts into audio ground truth. These results support the claim that acceleration peaks are useful for periodic motion analysis, but they do not validate the audibility claim. The paper should clearly separate this evidence from the core audibility argument and avoid implying that counting performance corroborates the sound-from-collision premise.","section":"§5.3 and Appendix A"},{"comment":"The term 'inflectional flow' is used for the finite difference of optical flow, which is an acceleration estimate (the second derivative of position with respect to time). A true velocity inflection point is where the acceleration changes sign, not where the acceleration magnitude is large; a sudden force during a collision is better characterized by a large acceleration or jerk. The computed quantity may still be a useful cue, but the conceptual framing 'inflections in velocity' is misleading. Please clarify the terminology and justify why the magnitude of the second temporal difference of optical flow, rather than a zero-crossing or a third-derivative measure, is the appropriate prior for collision timings.","section":"§4, Eq. (3) and terminology"},{"comment":"The definitions of NME and PME are incomplete. In Eq. (15), c_i is computed as the sum of predicted probabilities O_t, but no thresholding or binarization procedure is specified, making the count error ambiguous. In Eq. (16), PME depends on 'matched audible action frames' m_i, but the matching scheme between predictions and ground-truth frames is not defined. Without exact, reproducible definitions, the central quantitative comparisons cannot be independently verified. Please provide precise pseudocode or formal definitions of both metrics, including how predictions are converted to discrete frame labels and how matches are established.","section":"Appendix B, Eqs. (15)–(16)"}],"minor_comments":[{"comment":"The abstract and introduction claim the task is to identify 'spatio-temporal coordinates of audible movements,' but Fig. 2 restricts the scope to 'visible audible actions.' This restriction should appear in the formal problem formulation in §4, not only in a figure caption, so readers can assess the intended scope.","section":"Abstract and Fig. 2"},{"comment":"The attention is called 'image-to-motion' and 'image-to-inflection,' but the query comes from motion or inflection features and the key comes from image features, so the direction is actually motion-to-image and inflection-to-image. Please rename or clarify to avoid confusion.","section":"§4.1, Eqs. (5)–(6)"},{"comment":"The intra-video temporal regularization loss L_temp penalizes the second-order difference of the discriminative map (D_{i+2} + D_i − 2D_{i+1}), which enforces smoothness of the map's second difference rather than the map itself. This should be stated explicitly to avoid misrepresenting the constraint as standard total-variation smoothness.","section":"§4.2, Eq. (12)"},{"comment":"The reference for Xu et al. 2020b is incomplete; it lists volume and page numbers but no journal or conference name. Also, the 'Vaswani, A.' reference should be 'Vaswani et al.' to reflect the actual authorship of 'Attention is all you need.'","section":"References"},{"comment":"The caption says 'The light part of a_{i−1} illustrates...' but does not explain what the 'light part' refers to. Please specify whether it denotes a region of high acceleration magnitude or a specific direction of the flow field.","section":"Figure 5 caption"}],"recommendation":"major_revision","confidential_remarks":"The core concern is the construct validity of the Audible623 benchmark. If the authors can add an audio-validated subset or a human study comparing visual-only labels with audio-based labels, the central claim would be substantially strengthened. Otherwise, the paper's title and abstract overclaim 'audible' when the benchmark measures visually salient collisions. The counting transfer results are genuine but do not resolve this issue because they validate periodicity estimation, not audibility. I recommend major revision rather than rejection because the problem is potentially fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real contribution — a new task, a new dataset, a sensible kinematic prior — but the benchmark's annotation protocol makes the evaluation partially circular, and the paper doesn't engage with that. Still worth a serious referee.\n\nWhat's new: Audible Action Temporal Localization as a frame-level keyframe task, the Audible623 dataset (623 videos, ~6,262 action instances, frame-level labels), and TA2Net, which uses optical flow and its finite difference as an 'inflectional flow' prior. The method outperforms eleven baselines across four task families on the benchmark, and the zero-shot results on UCFRep and CountixAV are real supporting evidence that the motion prior generalizes to periodicity. Code and dataset are promised. That's concrete, reproducible work and deserves credit.\n\nThe soft spots are real, and the main one is the label construction. Section 3 says annotators watch videos without audio, judge whether each video contains at least one audible action, and exclude videos where the temporal location can't be visually determined. So the ground truth is a human estimate of visually identifiable collision timing, not an independent measurement of audibility. The model's input is optical flow, the prior is a second derivative of motion — the evaluation is confined to exactly the cases the prior is designed to detect. That doesn't falsify the 'sound-from-collision' premise; it bakes it in. The paper's own limitation section only mentions optical-flow inaccuracy, which suggests the authors don't see this as a threat. The right fixes are cheap-ish: report inter-annotator agreement, validate a subset with actual audio, and measure performance on excluded 'hard' videos. Also, all Tab. 2 and Tab. 4 numbers lack error bars, so it's hard to know how much of the 0.616 F1 edge is noise.\n\nI'm not as negative as the stress-test note. The paper explicitly scopes to 'visible audible actions' (Fig. 2), so within that scope the central argument mostly holds up. The counting results on UCFRep and CountixAV genuinely strengthen the inflection-flow prior, even if they don't validate audibility. The concern is that the abstract and introduction claim more than the benchmark can support.\n\nBottom line: this is a solid new-task paper with a usable dataset and a plausible method. A serious editor should send it to review; my recommendation would be major revision, mainly on dataset validation and uncertainty reporting. I'd cite it if I did video temporal localization; I'd also bring it to reading group. But I wouldn't take the F1 numbers at face value until the annotation protocol is clarified.","headline":"New task and dataset for timing visually obvious collisions; the method is reasonable and the zero-shot counting results are real, but the silent-visual annotation protocol makes the benchmark partially circular and the paper doesn't address it.","tokens_in":16882,"tokens_out":3877,"would_cite":true,"duration_ms":34511,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Silent video alone can time visible collisions that would make a sound, because the collision moment is encoded as an inflection in the object's motion.","keywords":["audible action temporal localization","inflectional flow","keyframe localization","video dubbing","self-supervised spatial localization","repetitive action counting","sound from collision","Audible623"],"falsifier":"Take a set of real videos with independently recorded audio and known collision times, compute the inflectional flow from optical flow, and check whether true collision frames are local peaks of inflectional-flow magnitude far more often than chance; if collisions frequently occur at frames with no velocity inflection, or optical flow is inaccurate under camera motion and occlusion, the central premise fails. A cheaper variant: measure the method's F1 only on the videos that the Audible623 annotators excluded as visually undeterminable, using audio ground truth; if performance collapses there, the claim that silent video alone carries the timing would be falsified.","tokens_in":15852,"feed_emoji":"💥","tokens_out":7975,"duration_ms":68501,"temperature":0.7,"pith_summary":"This paper introduces a task it calls audible action temporal localization: given a silent video, mark the exact frames at which a visible action would make a sound, such as a bat hitting a ball or a hammer striking a surface. The paper's central claim is that these sound moments can be read from the video itself, without any audio track, because collisions are sudden changes in force that show up as inflections in an object's velocity. To test that claim it builds TA2Net, which computes an 'inflectional flow' from the difference of consecutive optical-flow fields, combines it with image and motion features, and trains with a self-supervised spatial-localization auxiliary task. The paper also contributes Audible623, a 623-video benchmark with frame-level labels of audible actions, and reports that TA2Net reaches F1 0.616, NME 3.462, and PME 0.744 there, outperforming eleven methods from four task families. A sympathetic reader would care because, if true, silent video alone can drive automated dubbing, re-dubbing, and sound-aware editing without needing recorded audio.","feed_headline":"The bend in motion reveals the frame where a collision sounds","feed_subtitle":"A vision-only model times impact sounds on the new Audible623 benchmark and counts repetitions without hearing audio.","key_machinery":"Inflectional flow: a velocity-inflection prior computed from the difference of consecutive optical-flow fields, $a^+_{i-1} = v^+_i - v^+_{i-1}$, which approximates the second derivative of an object's position and is intended to mark the instant a collision force changes the object's state of motion. The claim-carrying mechanism is that this prior, combined with motion flow and original frames through a Cross-Kinematics Aggregation module, an attention operation that lets image features query motion and inflection features, gives the network a kinematic cue that is fine-grained enough to pick a single audible frame rather than an action interval. A self-supervised spatial auxiliary task, built from an inter-video contrastive loss and an intra-video smoothness loss, reinforces this by learning where motion is happening and produces a spatial localization map as a side output. These components together are what the paper credits for the jump from the appearance-only baseline (F1 0.406) to the full model (F1 0.616).","core_discovery":"On its own terms, the discovery is that the time of an audible collision is marked by a local inflection in the object's motion, and that this inflection can be computed directly from vision as the second derivative of position. The paper formalizes this as inflectional flow: with optical flow $v(t) = dx/dt$, the inflectional flow is $a(t) = d^2x/dt^2$, estimated in practice as the difference between two consecutive optical-flow fields. TA2Net feeds the original frames, motion flow, and inflectional flow into three encoders, aggregates them with cross-attention, and classifies each frame as sound or non-sound, with the same features also decoded into a spatial motion map. The authors state the load-bearing assumption explicitly: audible actions, specifically collisions, are typically induced by sudden changes in force acting on an object, and these manifest as velocity inflections. On Audible623 this yields frame-level timings with F1 0.616, a number-match error of 3.462 frames, and a position-match error of 0.744 frames, and the same model transfers to repetitive action counting without any counting-specific training.","pith_inferences":["Editorial inference: if inflectional flow is as general as the collision experiments suggest, the same second-derivative cue could mark other force-driven micro-events that are not audible, such as foot-strike in running, tool contact in surgery, or the moment a dancer's limb changes direction, turning the method into a general contact or phase detector.","Editorial inference: the benchmark's annotation protocol removes videos whose audible-frame location cannot be visually determined, so an immediate stress test is to run the same method on those excluded videos with audio-derived ground truth; the paper's reported advantage may shrink on visually ambiguous collisions.","Editorial inference: the spatial localization side output could serve as pseudo-label supervision for sound source localization in the audio-visual setting, letting a visual-only teacher bootstrap an audio-visual student; this is a natural next experiment the paper does not run."],"forward_implications":["A silent-video-only model can assign a precise frame to each impact sound, so dubbing and re-dubbing tools could place synthesized collision sounds without a reference audio track.","Because inflectional flow is a general kinematic quantity, the same network counts repetitive actions zero-shot on UCFRep and CountixAV, reaching first-place or second-place results on MAE and OBO without being trained for counting.","The spatial side maps localize moving and sound-source regions, and the paper shows the localization transfers to AVE videos without training there, pointing toward audio-free sound source localization.","Each added component matters: ablations show motion flow, inflectional flow, cross-kinematics aggregation, contrastive loss, and temporal smoothing each raise F1 on Audible623, so the reported gain is not from any single module alone."],"supporting_citations":[{"why":"Supplies GMFlow, the pretrained optical-flow estimator whose forward and backward flow fields are differenced to form the inflectional-flow prior.","marker":"(Xu et al., 2023)"},{"why":"Source of Kinetics videos used, together with UCF101, to construct the Audible623 dataset.","marker":"(Kay et al., 2017)"},{"why":"Source of UCF101 videos used both for Audible623 and for the UCFRep counting benchmark.","marker":"(Soomro et al., 2012)"},{"why":"Provides the RepNet baseline and the repetition-counting evaluation convention (MAE and OBO) used for comparison.","marker":"(Dwibedi et al., 2020)"},{"why":"Provides the TransRAC baseline and the multi-scale temporal-correlation transformer design that motivates the paper's spatio-temporal fusion stage.","marker":"(Hu et al., 2022)"},{"why":"Supplies the UCFRep dataset used to test zero-shot transfer to repetitive action counting.","marker":"(Zhang et al., 2020)"},{"why":"Supplies the CountixAV dataset used to test zero-shot transfer to repetitive action counting.","marker":"(Zhang et al., 2021)"}],"fun_headline_variants":["Vision-only model times collisions from motion inflections","Inflectional flow detects when collisions sound, no audio needed","New benchmark Audible623: timing sounds from motion alone","TA2Net tags impact frames via the second derivative of motion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that audible actions, especially collisions, are reliably accompanied by a sudden change in an object's motion that is visible in silent video, and that human volunteers can locate that frame by eye; if many real-world sounds occur without a visible jerk in the object's motion, or the annotation procedure discards exactly those videos where the timing cannot be seen, the reported accuracy rests on a selected subset rather than on the general sound-from-collision link.","fun_headline_variants_meta":{"raw":{"variants":["Vision-only model times collisions from motion inflections","Inflectional flow detects when collisions sound, no audio needed","New benchmark Audible623: timing sounds from motion alone","TA2Net tags impact frames via the second derivative of motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000277,"raw_usage":{"total_tokens":1678,"prompt_tokens":1000,"completion_tokens":678,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":611}},"tokens_in":616,"tokens_out":678,"duration_ms":6985,"temperature":1.0,"reasoning_tokens":611,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:04:25.937619+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of real videos with independently recorded audio and known collision times, compute the inflectional flow from optical flow, and check whether true collision frames are local peaks of inflectional-flow magnitude far more often than chance; if collisions frequently occur at frames with no velocity inflection, or optical flow is inaccurate under camera motion and occlusion, the central premise fails. A cheaper variant: measure the method's F1 only on the videos that the Audible623 annotators excluded as visually undeterminable, using audio ground truth; if performance collapses there, the claim that silent video alone carries the timing would be falsified.","supporting_citations":[{"cited_title":"Counting out time: Class agnostic video repetition counting in the wild","cited_arxiv_id":null,"evidence_quote":"Provides the RepNet baseline and the repetition-counting evaluation convention (MAE and OBO) used for comparison."},{"cited_title":"Context-aware and scale-insensitive temporal repetition counting","cited_arxiv_id":null,"evidence_quote":"Supplies the UCFRep dataset used to test zero-shot transfer to repetitive action counting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the CountixAV dataset used to test zero-shot transfer to repetitive action counting."}],"review_version":1}