{"id":"a490de6a-2b1b-4408-831f-6df4069e65a1","arxiv_id":"2411.15729","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"OccludeNet introduces a large occluded video dataset and a counterfactual training loss that masks the actor to improve occlusion robustness, with modest gains.","lead":"OccludeNet is a new video dataset with over 230,000 clips that adds various types of occlusions, such as moving objects, static barriers, and multi-view interactions, to action recognition benchmarks. The paper also proposes a 'causal' training loss that subtracts predictions on a version of the video with the actor removed, reporting small accuracy gains on occluded clips.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The backdoor adjustment in Eq. (4) is never computed; CAR's loss uses a single actor-erased counterfactual, and this erasure does not hold O, B, F fixed. The causal claim is therefore unsupported, and gains are explainable by the loss itself.","rationale":"The reader's weakest assumption correctly identifies that the counterfactual video does not necessarily realize do(A=a) while holding O, B, F fixed. My stress-test agrees with that, but sharpens it into a more direct internal inconsistency: even if the counterfactual were a perfect intervention, the algorithm never computes the backdoor adjustment of Eq. (4) because it does not sum over the occlusion variable o. The loss in Eq. (8) is a contrastive regularizer that directly encourages original predictions to match the original-minus-counterfactual softmax difference, so the reported gains are a direct consequence of the objective. This does not change the reader's REJECT verdict; it reinforces it. I set agreement_with_reader to 'partial' because I emphasize the missing sum in Eq. (4) rather than the counterfactual-correspondence issue alone, though both concerns target the same unsupported causal claim.","tokens_in":12962,"tokens_out":5210,"duration_ms":46414,"concrete_test":"One decisive check: re-run the CAR fine-tuning protocol but replace the counterfactual logits c_i with logits from the same video after erasing a random background region (matched in pixel count and segmentation tool). If accuracy on OccludeNet-D remains within 1 point of CAR, the actor-specific erasure is not doing the claimed causal work. Separately, quantify whether the erasure leaves O and B fixed: use the provided segmentation masks to compute mean L2 pixel change between original and counterfactual videos in the occluder region and in the background region; if either change exceeds a small threshold (e.g., 5% of the region's norm), the intervention is confounded. These two checks settle whether Eq. (4) and the do(A=a) assumption are actually realized.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that CAR implements backdoor-adjusted causal inference (Eq. 4) and that its robustness gains follow from this causal correction. This claim is not supported because the method never computes Eq. (4). The backdoor adjustment requires a sum over occlusion states o with P(o); instead, the algorithm generates exactly one counterfactual video per training clip by removing the actor (Section IV-C, Fig. 12) and forms a logit difference p_i - c_i (Eq. 7). No averaging over o appears anywhere in the training procedure or the ablation. The single counterfactual video also does not realize do(A=a) with O, B, F held fixed: erasing the actor removes actor pixels that are part of the visible scene, reveals background that was previously occluded, and in OccludeNet-D the occluder is centered on the actor's bounding box, so part of the occluder is erased as well. The causal graph in Eq. (2) posits that O influences B, F, and A, so an intervention on A must not change O; the paper provides no analysis showing that the segmentation-based erasure leaves O, B, and F approximately invariant. Moreover, the training loss in Eq. (8) explicitly incentivizes the model to make original predictions P match Y, the softmax of (p_i - c_i). This is a direct supervised contrast, so any improvement in accuracy is exactly what the objective asks for, independent of whether the counterfactual has causal validity. The empirical gains in Tables IV and V are modest (roughly 0.3-4.6 points) and are reported without error bars; they are consistent with a generic regularization effect. The paper's headline causal contribution therefore rests on an untested and, as implemented, incorrect correspondence between Pearl's do-operator and a single actor-masked video.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OccludeNet, a large occlusion-focused action recognition dataset with four subsets: OccludeNet-D (233,769 synthetic dynamic-occlusion clips built from Kinetics-400), OccludeNet-S (256 real static-occlusion clips), OccludeNet-I (345 single-view interactive clips), and OccludeNet-M (1,242 multi-view interactive clips). The authors benchmark eight action recognition models on these subsets and analyze which action classes are most affected by occlusion. They also propose Causal Action Recognition (CAR), which models an occluded scene with an SCM, invokes backdoor adjustment, and trains a model by comparing original predictions with predictions on an actor-erased counterfactual video. The empirical results show small but mostly positive accuracy gains for CAR over baselines on several subsets.","tokens_in":13204,"tokens_out":3427,"duration_ms":32662,"significance":"The dataset contribution is potentially useful: OccludeNet-D is large, covers a Kinetics-scale class set, and the paper provides a broad benchmark comparison of eight models, along with an analysis of class-specific sensitivity to occlusions. The data and code are promised publicly, which is a practical asset for the occlusion robustness community. However, the causal claims are the central novelty of the paper, and they are not supported by the implementation or the experiments. The backdoor adjustment is never computed, the counterfactual video does not realize the required intervention, and the training loss directly optimizes the quantity that is later reported as evidence for the causal model. Without the causal framing, the contribution reduces to a dataset plus an auxiliary supervision loss with modest empirical gains, which changes the significance of the work substantially.","major_comments":[{"comment":"The paper claims to apply backdoor adjustment, but Eq. (4) is never computed anywhere in the method or the experiments. The training pipeline in Section IV-C and Fig. 12 generates one actor-erased counterfactual per clip and forms the logit difference p_i - c_i in Eq. (7); there is no sum over occlusion states o with weights P(o), no estimate of P(o), and no ablation that implements Eq. (4). The statement in the abstract and conclusion that CAR 'uses backdoor adjustment' is therefore unsupported by the presented method.","section":"Section IV-B, Eq. (4)"},{"comment":"The counterfactual video does not faithfully realize the intervention do(A=a) while holding O, B, and F fixed. According to Eq. (2), O influences A, B, and F, so an intervention on A must not change O. Erasing the actor with segmentation removes actor pixels that are part of the visible scene, reveals background that was previously hidden behind the actor, and in OccludeNet-D the occluder is centered on the actor's bounding box (Fig. 3), so part of the occluder is erased as well. The paper provides no analysis demonstrating that the erasure leaves O, B, and F approximately invariant, which is a precondition for the causal interpretation of Eq. (5).","section":"Section IV-B, Section IV-C, Fig. 12"},{"comment":"The training loss is circular with respect to the causal claim. Eq. (5) defines the treatment effect as the difference between original and counterfactual predictions, and Eq. (8) explicitly trains the model so that the original softmax P moves toward Y, the softmax of (p_i - c_i). Any accuracy improvement on Tables IV and V is exactly what this supervised objective asks for, regardless of whether the counterfactual has any causal validity. Reporting these gains as evidence for the SCM or for 'causal influence of unoccluded actor features' is therefore circular.","section":"Section IV-C, Eq. (8)"},{"comment":"The test sets for OccludeNet-I and OccludeNet-M are very small: a 15% test split of 345 clips gives roughly 52 test clips for O-I, and 15% of 1,242 gives roughly 186 test clips for O-M. Table V reports point improvements of 0.93-4.64 percentage points without confidence intervals or significance tests. Differences of this size on these sample sizes are within sampling noise, so the claims of consistent robustness gains from CAR are not statistically supported.","section":"Table V"}],"minor_comments":[{"comment":"There are typos and formatting errors, including 'textitasize' in the Introduction and 'availibale' in the abstract; the dataset name is also inconsistently capitalized as OccludeNet and OCCLUDENET.","section":"Abstract and Section I"},{"comment":"The applications list is malformed: item 3 ends without a main verb, and item 4 begins with a dangling number ('...varied occlusions, 4) OccludeNet enables...'). Please renumber and complete each sentence.","section":"Section VI, Applications"},{"comment":"The comparison with actor-mask training lacks implementation details: the mask construction procedure, the auxiliary loss form, the number of training epochs, and the choice of backbone are not specified. The qualitative explanation that the mask auxiliary signal 'often lacks continuity' is vague and should be either quantified or removed.","section":"Section V-B, Table VII"},{"comment":"The caption states that 'ten classes with the largest drops on OccludeNet-D were selected at random'; random selection without a fixed seed or repeated trials makes this comparison difficult to interpret. Please specify the selection protocol.","section":"Fig. 14 caption"},{"comment":"The notation in Eq. (5) is unclear: P(A=A,O=O) and P(do(A=a),O=O) mix feature variables with the prediction P, and the expectation over ~A is not defined in terms of the actual counterfactual generation process. Please clarify the notation or move the causal formalism to a setting where it is well-defined.","section":"Section IV-C, Eq. (5)"}],"recommendation":"reject","confidential_remarks":"The paper would be substantially stronger if resubmitted as a dataset and benchmark paper, with the causal framing removed or heavily qualified. As it stands, the central novelty---the causal mechanism of CAR---is not implemented or tested, and the empirical evidence for the method is weakened by small test sets and the circularity of the training objective. These are load-bearing issues rather than presentation problems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: OccludeNet itself is a real contribution—a large synthetic occlusion dataset (233k clips) plus smaller real subsets, with per-class analysis that action recognition people will use. The CAR method, though, is where the paper falls apart. The causal framing doesn't survive contact with the code. Eq (4)'s backdoor adjustment is never computed; the algorithm generates one actor-erased video per clip and subtracts logits (Eq 7), then trains Eq (8) to match the corrected softmax. That single erasure doesn't realize do(A=a) with O, B, F held fixed—erasing the actor also removes occluder pixels in the D subset and reveals background. More importantly, the loss explicitly maximizes the difference between original and counterfactual predictions, so any accuracy gain is literally the training objective, not independent evidence for a causal effect. The paper cites [25],[26] for counterfactual debiasing; the method is a re-application, not a new causal result. Gains are small (0.3–4.6 points), on tiny test sets (O-I ~52 clips, O-M ~186) without error bars. This looks like a generic regularizer.\n\nWhat's done well: the dataset construction is thoughtful—different occlusion types, multi-view, careful manual screening, occluder masks from COCO, and a solid benchmark across eight models. The class-correlation analysis (which actions drop most under occlusion) is a useful empirical finding. If the causal sections were stripped out, the dataset could stand alone as a resource paper.\n\nThe paper deserves peer review because the dataset is substantial and the claims need refereeing. I'd recommend major revision: either provide a real backdoor adjustment with proper interventions and error bars, or drop the causal language and present CAR as a simple counterfactual regularizer. As it stands, the central causal claim is unsupported.","headline":"A genuinely useful occlusion dataset wrapped in a causal story the implementation does not support.","tokens_in":13897,"tokens_out":3277,"would_cite":true,"duration_ms":29286,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By training a video backbone on the divergence between original predictions and predictions from actor-erased counterfactuals, the paper claims to isolate and reinforce the causal contribution of unoccluded actor features, making action…","keywords":["action recognition","occlusion robustness","causal inference","counterfactual reasoning","backdoor adjustment","video dataset","structural causal model","class correlation profiling"],"falsifier":"Train CAR with counterfactual videos in which a random rectangular patch or a blurred region is removed instead of the segmented actor, matching the same occluder statistics; if the accuracy gain matches CAR's gain, the causal interpretation is not needed to explain the improvement. A second check is to measure the pixel difference outside the actor mask between original and counterfactual frames — if erasing the actor also alters background or occluder pixels, the modeled intervention was not actually realized.","tokens_in":12664,"feed_emoji":"🎬","tokens_out":6495,"duration_ms":56991,"temperature":0.7,"pith_summary":"The paper addresses the fact that standard action-recognition video benchmarks are mostly occlusion-free, so models trained on them degrade when actors are partially hidden. It builds OccludeNet, a large-scale dataset spanning 424 action classes with four kinds of occlusion — dynamic tracking, static scene, single-view interactive, and multi-view interactive — mixing synthetic and real footage. On this dataset it measures which classes suffer most, finding that actions with low scene relevance and partial body visibility drop hardest. To counter the degradation it proposes Causal Action Recognition (CAR), which models the occluded scene as a causal graph, treats occlusion as a confounder between actor features and predictions, and trains a model to minimize the divergence between its prediction on the original video and its prediction on a counterfactual video with the actor removed. The paper claims this redirects attention to the unoccluded actor and improves accuracy on occluded and even unoccluded benchmarks.","feed_headline":"Causal loss cuts occlusion damage in action recognition","feed_subtitle":"Actor-erasure counterfactuals and a 424-class occluded dataset narrow the accuracy gap on hidden actors.","key_machinery":"The load-bearing object is the structural causal model over four feature groups — actor A, background B, context or motion F, and occlusion O — feeding a prediction P, with O influencing all three other factors. From this graph the paper derives the interventional prediction via back-door adjustment over O, and implements it operationally as counterfactual video generation: the actor is segmented and removed while the rest of the frame is kept, producing a counterfactual video whose logits are subtracted from the original logits to form a corrected softmax prediction. The training objective is cross-entropy plus a KL-divergence term weighted by a hyperparameter, which the paper interprets as a supervised signal for the treatment effect of actor features. The mechanism is what makes the method parameter-free at inference: the loss reshapes the backbone so that the original classifier itself becomes occlusion-robust, so no extra module is needed when deploying.","core_discovery":"The paper's central claim is that the drop in prediction quality under occlusion can be counteracted by supervising the model with a causal effect estimate: the difference between the softmax prediction on the original video and the softmax prediction on a counterfactual video in which the actor has been erased. Treating occlusion as a confounder of actor features and the prediction, CAR applies the back-door adjustment of causal inference and approximates the interventional prediction by feeding a counterfactual video into the same backbone. The training loss combines cross-entropy with a Kullback-Leibler term that pulls the original prediction toward a corrected prediction formed by subtracting counterfactual logits from original logits. The paper reports consistent top-1 and top-5 gains over baselines on the OccludeNet subsets and on unoccluded Kinetics-400 with a fine-tuned video transformer, and class activation maps show attention shifting from occluders to the actor.","pith_inferences":["My inference: the subtraction-of-logits operation is mathematically equivalent to training on a mask-sensitivity signal, so the observed gains might be reproducible with any strong erasure, such as random patches, blur, or background replacement, rather than requiring a causal interpretation.","My inference: a testable extension of the paper's logic is that the KL term should matter most when the actor is genuinely occluded; if it is just a regularizer, gains should appear equally on unoccluded data with no occlusion in the counterfactual.","My inference: the paper leaves implicit that its counterfactual 'erase the actor' operation cannot hold background and occluder fixed in the dynamic-occlusion subset, because the occluder is placed over the actor bounding box and is erased together with the actor; measuring the pixel change outside the actor mask would test whether the intervention was actually realized."],"forward_implications":["If the causal adjustment is doing what the paper claims, then any video backbone can be made more occlusion-robust by fine-tuning with the counterfactual KL loss, without adding inference-time cost.","The class-correlation analysis predicts which action classes will gain most from such training: low-scene-relevance, body-part-dependent actions should benefit most, while background-driven classes gain little.","Synthetic tracking occlusions transfer to real occluded footage, so the synthetic subset can serve as a scalable training source for real-world occlusion robustness.","The dataset itself provides a graded benchmark in which occlusion degree, area ratio, and duration ratio can be varied independently, allowing future methods to be compared along each axis."],"supporting_citations":[{"why":"Supplies the do-calculus and back-door adjustment formulas that the causal intervention in Eqs. (3)-(4) implements.","marker":"[11]"},{"why":"Provides the K-400-O and UCF-101-O occlusion benchmarks that motivate the dataset and supply comparison numbers.","marker":"[6]"},{"why":"Kinetics-400 is the source video collection for the dynamic-occlusion subset and the standard unoccluded benchmark.","marker":"[3]"},{"why":"Provides the four occluder object categories and their foreground masks used to generate synthetic tracking occlusions.","marker":"[30]"},{"why":"UniFormerv2 is the backbone used for the main CAR fine-tuning experiments and accuracy comparisons.","marker":"[40]"},{"why":"StillMix is a foreground-background bias mitigation baseline that CAR compares against on the dynamic-occlusion subset.","marker":"[9]"},{"why":"FAME is a motion-aware foreground-background merging baseline that CAR compares against.","marker":"[10]"},{"why":"The grounded segmentation and tracking tool is used to produce counterfactual videos by erasing the actor.","marker":"[32]"}],"fun_headline_variants":["Counterfactual erasure boosts action recognition under occlusion","Causal action recognition with actor-erasure counterfactuals","OccludeNet: causal method improves occluded action recognition","Erase actor, apply counterfactual loss: action recognition wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that cutting the actor out of the video realizes the causal intervention 'make the actor absent' while leaving background, occluder, and motion cues untouched; the paper assumes this correspondence without testing it.","fun_headline_variants_meta":{"raw":{"variants":["Counterfactual erasure boosts action recognition under occlusion","Causal action recognition with actor-erasure counterfactuals","OccludeNet: causal method improves occluded action recognition","Erase actor, apply counterfactual loss: action recognition wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000456,"raw_usage":{"total_tokens":2275,"prompt_tokens":916,"completion_tokens":1359,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":1290}},"tokens_in":532,"tokens_out":1359,"duration_ms":10708,"temperature":1.0,"reasoning_tokens":1290,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:58:45.458468+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CAR with counterfactual videos in which a random rectangular patch or a blurred region is removed instead of the segmented actor, matching the same occluder statistics; if the accuracy gain matches CAR's gain, the causal interpretation is not needed to explain the improvement. A second check is to measure the pixel difference outside the actor mask between original and counterfactual frames — if erasing the actor also alters background or occluder pixels, the modeled intervention was not actually realized.","supporting_citations":[{"cited_title":"Causal inference,","cited_arxiv_id":null,"evidence_quote":"Supplies the do-calculus and back-door adjustment formulas that the causal intervention in Eqs. (3)-(4) implements."},{"cited_title":"Revealing the unseen: Bench- marking video action recognition under occlusion,","cited_arxiv_id":null,"evidence_quote":"Provides the K-400-O and UCF-101-O occlusion benchmarks that motivate the dataset and supply comparison numbers."},{"cited_title":"Microsoft COCO: common objects in context,","cited_arxiv_id":null,"evidence_quote":"Provides the four occluder object categories and their foreground masks used to generate synthetic tracking occlusions."},{"cited_title":"Uniformerv2: Unlocking the potential of image vits for video under- standing,","cited_arxiv_id":null,"evidence_quote":"UniFormerv2 is the backbone used for the main CAR fine-tuning experiments and accuracy comparisons."},{"cited_title":"Mitigating and evaluating static bias of action representations in the background and the foreground,","cited_arxiv_id":null,"evidence_quote":"StillMix is a foreground-background bias mitigation baseline that CAR compares against on the dynamic-occlusion subset."},{"cited_title":"Motion-aware contrastive video representation learning via foreground-background merging,","cited_arxiv_id":null,"evidence_quote":"FAME is a motion-aware foreground-background merging baseline that CAR compares against."}],"review_version":1}