{"id":"5dde887f-c7a8-4ab1-8025-515cbcbc438f","arxiv_id":"2502.01455","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Training a before/after classifier with a temporal reconstruction loss on motion-compensated saliency maps improves weakly supervised video segmentation of waste on a conveyor belt.","lead":"This paper introduces a weakly supervised method that adds a temporal consistency loss, using optical flow, to train a classifier whose saliency maps segment illegal objects in waste-sorting videos. The method improves mean IoU over GradCAM, Frame-to-Frame, and PuzzleCAM on the authors' SERUSO dataset, from 34.20 to 37.84.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported temporal gain may be inflated by evaluation leakage: the split is not stated to be video-level and the background median may use test frames; a video-level, training-only-background re-run is needed.","rationale":"I focused on the strongest claim, that training-time temporal coherence improves weakly supervised video segmentation over spatial-only CAMs. For that claim to hold, the 3.64 mIoU gap must reflect a genuine learning signal rather than evaluation leakage. The manuscript's protocol descriptions are incomplete in two places: the train/validation split is not stated to be video-level, and the background median in Section 4.1 is described as 'across all grayscale frames' without restricting to training frames. Both omissions are load-bearing because SERUSO is a small private dataset (3,682 images from 68 videos) and the temporal module is exactly the component that could exploit within-video redundancy or test-video statistics. The reader's verdict already conditions on missing error bars and possible background leakage; I partially agree, and the proposed video-level re-evaluation would settle whether the central claim survives. If the re-run preserves the gap, the contribution is credible; if not, the current numbers cannot support the claim. Since the concern is about missing protocol evidence rather than a demonstrated internal inconsistency, the CONDITIONAL verdict remains appropriate.","tokens_in":11964,"tokens_out":9197,"duration_ms":93084,"concrete_test":"Ask the authors to rerun Table 1 and Table 2 under a video-level split: use 80% of the 68 videos for training and the remaining 20% for evaluation, estimate each video's background only from training videos (or a separate empty-belt capture), repeat with at least 5 random video splits, and report mean ± std mIoU for all methods. If the mean Our-vs-PuzzleCAM gap remains positive and its confidence interval excludes zero, the claim survives; if the gap shrinks or crosses zero, the reported temporal benefit is not established as genuine.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison (Our 37.84 vs PuzzleCAM 34.20 on SERUSO, Table 1) is only meaningful if the evaluation protocol prevents test information from entering training or preprocessing. The paper does not establish this. Section 5.1 says the dataset was split into training and validation sets with an 80%/20% split, but it does not say the split is video-level. SERUSO consists of 68 videos and 3,682 images; if frames are split randomly, the same objects appear in both training and test, and the temporal module can memorize the specific tracks it is later evaluated on, inflating the temporal contribution. Section 4.1 also estimates the background as a pixel-wise median across all grayscale frames; computed per before/after video, this median is estimated from the very frames whose foreground masks are evaluated, injecting test-video statistics into the inputs of every method. Both effects can differentially favor the proposed temporal loss. The +3.64 mIoU gap therefore cannot be attributed to temporal coherence until re-evaluated under a leakage-free protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a weakly supervised video segmentation method for waste sorting. It trains an auxiliary classifier on before/after videos, derives segmentation masks from CAMs, and augments the training loss with a temporal reconstruction term that warps saliency maps from adjacent frames via optical flow and enforces agreement with the central frame, alongside the spatial consistency loss of PuzzleCAM. Experiments on the authors' SERUSO dataset and on ZeroWaste-w compare GradCAM, Frame-to-Frame, and PuzzleCAM, and an ablation isolates the spatial and temporal components. The central claim is that integrating temporal coherence during classifier training improves segmentation mIoU over spatial-only CAM methods, with the reported best result being 37.84 versus 34.20 for PuzzleCAM on SERUSO.","tokens_in":12227,"tokens_out":5574,"duration_ms":50628,"significance":"If the reported results survive a leakage-free evaluation, the contribution is a modest but useful addition to weakly supervised video segmentation: a training-time temporal coherence loss on CAMs, with a clean ablation suggesting that the spatial and temporal modules are complementary. The industrial motivation is clear and the external ZeroWaste-w comparison is a strength, as is the ablation isolating each loss component. However, the paper does not provide public code or data, and the empirical evidence currently lacks error bars, a precise description of the data split, and a specification of the background-estimation protocol. These gaps directly affect the credibility of the central temporal-coherence claim, so the significance of the result is not yet established.","major_comments":[{"comment":"The 80/20 split is described only as a split of the dataset, and the paper never states that the split is video-level. Since SERUSO consists of 68 videos and 3,682 images, a random frame-level split would place frames from the same video, and hence the same object instances and temporal tracks, in both training and validation. The temporal module is trained to reconstruct the central saliency map from adjacent frames, so it could memorize the very sequences used for evaluation. This would differentially inflate the reported temporal gains in Table 1. Please re-run the experiments under a video-level split (e.g., holding out whole videos) and state the exact split protocol, including how the 364 manually labeled test images are selected.","section":"§5.2, §5.1"},{"comment":"The background estimator is computed as the pixel-wise median 'across all grayscale frames,' without specifying whether this includes validation or test frames. If the median is computed over the entire dataset, the background and the foreground masks for each frame depend on statistics of the very frames being evaluated, including lighting changes and object presence. This leakage is common to all compared methods, but it can interact with the temporal module differently because the temporal module uses the masked frames for optical flow computation and warping. Specify that the background is estimated from training videos only, give the numerical threshold used to define 'significantly different,' and report the sensitivity of Tables 1 and 2 to that threshold.","section":"§4.1"},{"comment":"All quantitative claims rest on a single 80/20 split and, apparently, a single training run. The differences at stake are small (37.84 versus 34.20 mIoU for the main comparison; 29.23 versus 24.08 in the temporal-only ablation), and no error bars, number of seeds, or significance tests are reported. Without variance estimates, the reader cannot tell whether the temporal improvement is robust or split-dependent. Add results over multiple seeds and report mean ± standard deviation, or use a paired evaluation across multiple splits.","section":"Tables 1 and 2, §5.2"},{"comment":"The comparison omits the most relevant temporal-CAM baselines discussed in the paper itself, namely T-CAM [3] and CoLo-CAM [4]. The caption of Table 1 claims that the proposed model outperforms 'all other CAM-based methods,' but only GradCAM, Frame-to-Frame, and PuzzleCAM are evaluated. If T-CAM and CoLo-CAM are not applicable to the segmentation setting or to SERUSO, this should be stated explicitly; otherwise they should be included. This is directly relevant to the central claim that integrating temporal coherence during training is superior to post-hoc temporal aggregation.","section":"§2, Table 1"}],"minor_comments":[{"comment":"The phrase 'to be closed of the reconstructed one' should read 'to be close to the reconstructed one'; please proofread the manuscript for similar grammatical errors.","section":"§4.4, after Eq. (11)"},{"comment":"The symbol z is used both for the prediction vector in Eq. (3) and for the 'true label vector' in the sentence after Eq. (5). Use distinct symbols (for example, z for the prediction and y_true for the label vector) to avoid confusion.","section":"§4.2, Eqs. (3)–(5)"},{"comment":"The sentence 'by inverting the binary masks, we generated a new set of images containing only background elements' is imprecise about how the background-only images are constructed and whether they are used for classifier training, for evaluation, or both. Clarify this step and its role in the three-class formulation.","section":"§4.1"},{"comment":"The entry 'Impossible' in the ZEROWASTE column should be replaced with a dash or 'not applicable,' with a note in the text explaining that the after-class data in ZeroWaste-w are static and therefore do not support training the temporal module. The current label could be misread as a failed experimental result.","section":"Table 1"},{"comment":"The two-class formulation in Eq. (1) and Fig. 2a (illegal versus background) is not fully aligned with the three-class formulation introduced in Section 4.1 and Fig. 3b (before, after, background). Reconcile these notations so that the class labels Λ and Λ-hat are used consistently.","section":"Figures 2 and 3, Section 3"},{"comment":"The statement that code and dataset are 'available upon request' is not a public release and prevents independent verification. If the journal's policy requires reproducibility, provide a public link or a detailed implementation protocol so that the reported numbers can be checked.","section":"Abstract, §5.1"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the central idea is sound and the ablation is informative, but the evaluation protocol currently has two concrete leakage risks — the possibly frame-level data split and the possibly test-inclusive background median. These are fixable with a re-run under a video-level, training-only protocol. If the authors address these points, add variance estimates, and clarify the baseline selection, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper's real contribution is a training-time temporal-consistency loss for class activation maps: force the central frame's CAM to match the optical-flow-warped, max-fused CAMs of its neighbors. That is a clean and sensible extension of PuzzleCAM's spatial consistency, and the ablation supports the claim that temporal and spatial regularizers are complementary (None 24.08, Temporal only 29.23, Spatial only 34.20, Both 37.84 on SERUSO). The three-class background split is a neat trick against background bias, and Table 3 gives a nice demonstration that background removal makes the classifier generalize. The paper is clearly written and does a fair job situating itself relative to F2F and T-CAM.\n\nThe soft spots are concentrated in the evaluation protocol, and they are serious enough that I would not trust the headline number yet. Section 5.1 says the dataset was split 80/20 but never says the split is video-level. SERUSO is 68 videos, 3,682 frames; if frames are randomly assigned, the same physical objects appear in training and test, and a temporal module that consumes triplets can memorize the very tracks it is later tested on. That would inflate the +3.64 mIoU over PuzzleCAM. Section 4.1 estimates the background as the pixel-wise median over 'all grayscale frames' with no mention of a training-only estimate; if that median uses the full dataset, every method's input is contaminated by test statistics, and the temporal method likely benefits most. There are no error bars, only one split, and code/dataset are only 'available upon request.' Missing comparisons with T-CAM and CoLo-CAM matter less than the leakage concerns, but they should be added or explicitly dismissed.\n\nI'd send this to peer review, but the main ask to the authors is straightforward: rerun with video-level splits, training-only background estimation, multiple seeds, and report variance. If the gap shrinks or disappears, so be it. The method is still a reasonable engineering contribution, and the leakage issue is a fixable protocol problem, not a broken idea.","headline":"Legitimate temporal-consistency idea, but the evaluation may leak test statistics; rerun with video-level splits before trusting the numbers.","tokens_in":12690,"tokens_out":2765,"would_cite":false,"duration_ms":25515,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A temporal reconstruction loss between motion-compensated saliency maps improves weakly supervised video segmentation for waste sorting.","keywords":["weakly supervised video segmentation","class activation maps","temporal coherence","optical flow","waste sorting","PuzzleCAM","saliency maps","background removal"],"falsifier":"Recompute the background estimator using only the training split of SERUSO and rerun the full pipeline; if the mIoU gap over PuzzleCAM shrinks sharply, the reported gain came from test-set statistics rather than temporal coherence, and if shuffling frame order while keeping optical flow warping does not lower mIoU, the temporal reconstruction loss is not the active ingredient.","tokens_in":11802,"feed_emoji":"♻️","tokens_out":10222,"duration_ms":77171,"temperature":0.7,"pith_summary":"Industrial waste sorting needs to segment 'illegal' objects on a conveyor belt without pixel-level annotations. This paper claims that training a before/after video classifier to produce temporally consistent class activation maps, using optical flow to warp neighboring frames, yields more accurate weakly supervised segmentation masks than spatial-only CAM methods. On the SERUSO dataset the full method reaches 37.84 mIoU versus 34.20 for PuzzleCAM and 22.08 for GradCAM. The central move is a reconstruction loss during training that aligns the central frame's saliency map with motion-compensated maps of adjacent frames, combined with PuzzleCAM's spatial coherence and a background-removal preprocessing step.","feed_headline":"Motion-aligned saliency training lifts weak video segmentation","feed_subtitle":"Flow-warped neighboring frames make weak segmentation masks consistent, beating spatial-only CAMs on waste-sorting video.","key_machinery":"The load-bearing mechanism is a temporal reconstruction loss applied to class activation maps, the saliency maps that highlight the image regions most responsible for a classifier's prediction. For each triplet of consecutive frames $X_{t-1}, X_t, X_{t+1}$, the network computes normalized maps $M_{t-1}, M_t, M_{t+1}$ as the ReLU of the target class channel of the feature space. Optical flow between $X_t$ and each neighbor warps the lateral maps into the central frame's coordinates, and the two warped maps are fused by pixel-wise maximum into $M_t^{\\mathrm{fused}}$. The temporal loss $\\mathcal{L}_{\\mathrm{temporal}} = \\|M_t - M_t^{\\mathrm{fused}}\\|_1$ pushes the central map to agree with its motion-compensated neighbors. The total loss is $\\mathcal{L}_{\\mathrm{total}} = \\mathcal{L}_{\\mathrm{cls}} + \\mathcal{L}_{\\mathrm{p-cls}} + \\alpha\\mathcal{L}_{\\mathrm{spatial}} + \\beta\\mathcal{L}_{\\mathrm{temporal}}$, combining the classification losses with the PuzzleCAM spatial reconstruction loss $\\mathcal{L}_{\\mathrm{spatial}} = \\|f_t - f_t^{\\mathrm{puzzle}}\\|_1$ and the temporal reconstruction loss.","core_discovery":"The paper's central claim is that temporal coherence belongs in the training phase of a weakly supervised segmentation classifier, not only in post-processing. The authors train an auxiliary classifier to distinguish 'before' videos (with illegal objects) from 'after' videos (without them) and use its class activation maps as segmentation masks. They add a temporal module that warps the saliency maps of frames $X_{t-1}$ and $X_{t+1}$ into the central frame's coordinate system using optical flow, fuses them with a pixel-wise maximum, and penalizes the $\\ell^1$ distance between this fused map and the central frame's own map. Combined with PuzzleCAM's spatial reconstruction loss, this pushes the network to activate the same object regions consistently as they move. On the SERUSO dataset the combined model attains 37.84 mIoU on the 'before' class, and the ablation shows that temporal and spatial modules complement each other.","pith_inferences":["A natural test of the method's internal logic is to replace the temporal reconstruction loss with a loss comparing each frame to a randomly chosen other frame; if the gain persists, temporal consistency per se is not the active ingredient.","The background estimation step, based on the pixel-wise median of all grayscale frames with a fixed threshold, is not explicitly validated on a training-only split; an experiment that recomputes the background from the training set alone would clarify whether the reported gains are inflated by test-frame statistics.","The saliency maps produced here could serve as pseudo-labels to train a fully supervised segmentation network, following the authors' stated future work; this would measure whether temporal consistency during classifier training translates into better downstream masks."],"forward_implications":["On SERUSO, the full method reaches 37.84 mIoU on the 'before' class, versus 34.20 for PuzzleCAM and 22.08 for GradCAM.","The ablation shows 24.08 mIoU with no reconstruction loss, 29.23 with only the temporal module, 34.20 with only the spatial module, and 37.84 with both, indicating the two coherence mechanisms are complementary.","Removing backgrounds and classifying into before/after/background improves classifier transfer: a classifier trained on background-free images keeps high accuracy on both background-free and original test images, while one trained with backgrounds loses accuracy when backgrounds are removed.","The method cannot run on ZeroWaste-w because that dataset provides static 'after' images, so temporal training requires videos for both classes."],"supporting_citations":[{"why":"Supplies the before/after conveyor-belt scenario and the ZeroWaste-w dataset used for comparison.","marker":"[2]"},{"why":"Provides the spatial coherence module and the PuzzleCAM baseline that the method extends and beats.","marker":"[17]"},{"why":"Computes the optical flow used to warp adjacent-frame saliency maps into the central frame.","marker":"[36]"},{"why":"Defines class activation maps, the saliency mechanism that produces the segmentation masks.","marker":"[44]"},{"why":"Frame-to-Frame baseline that aggregates saliency maps with optical flow in post-processing, the comparison for training-time integration.","marker":"[23]"},{"why":"GradCAM baseline for saliency map generation in the comparison table.","marker":"[29]"}],"fun_headline_variants":["Temporal coherence in training sharpens weak video segmentation","Flow-warped saliency maps make weak video masks consistent","Train with optical flow for consistent weak video segmentation","Temporal warping beats spatial-only CAMs for weak video segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The background is estimated as the pixel-wise median of all grayscale frames under a fixed threshold, and the method assumes this cleanly separates foreground objects using only training frames without leaking object or lighting cues.","fun_headline_variants_meta":{"raw":{"variants":["Temporal coherence in training sharpens weak video segmentation","Flow-warped saliency maps make weak video masks consistent","Train with optical flow for consistent weak video segmentation","Temporal warping beats spatial-only CAMs for weak video segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000488,"raw_usage":{"total_tokens":2396,"prompt_tokens":932,"completion_tokens":1464,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":1397}},"tokens_in":548,"tokens_out":1464,"duration_ms":10561,"temperature":1.0,"reasoning_tokens":1397,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:13:57.126108+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the background estimator using only the training split of SERUSO and rerun the full pipeline; if the mIoU gap over PuzzleCAM shrinks sharply, the reported gain came from test-set statistics rather than temporal coherence, and if shuffling frame order while keeping optical flow warping does not lower mIoU, the temporal reconstruction loss is not the active ingredient.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the before/after conveyor-belt scenario and the ZeroWaste-w dataset used for comparison."},{"cited_title":"In: 2021 IEEE international conference on image processing (ICIP)","cited_arxiv_id":null,"evidence_quote":"Provides the spatial coherence module and the PuzzleCAM baseline that the method extends and beats."},{"cited_title":"Advances in Neural Information Processing Systems33, 15220–15231 (2020)","cited_arxiv_id":null,"evidence_quote":"Computes the optical flow used to warp adjacent-frame saliency maps into the central frame."},{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Frame-to-Frame baseline that aggregates saliency maps with optical flow in post-processing, the comparison for training-time integration."},{"cited_title":"In: Temporal-consistent CAMs 17 Proceedings of the IEEE international conference on computer vision","cited_arxiv_id":null,"evidence_quote":"GradCAM baseline for saliency map generation in the comparison table."}],"review_version":1}