{"id":"0c1f248a-19bc-465d-955e-0246a0ca724d","arxiv_id":"2505.08173","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A two-stage causal intervention method (TSCNet) improves tail-class accuracy in Vision Transformer long-tailed classification by combining patch and feature-level backdoor adjustment with adaptive counterfactual augmentation.","lead":"This paper proposes TSCNet, a two-stage causal debiasing method for long-tailed image classification with Vision Transformers. It reports accuracy gains on CIFAR-100-LT and VireoFood-172, especially for tail classes, by combining background confounder interventions with counterfactual augmentation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage-1 'backdoor adjustment' is not the quantity claimed: S is derived from X, and Eq. (2) uses one random confounder sample per image without an inference-time average, so the reported gains are not yet attributable to the causal intervention.","rationale":"I read the paper as proposing a two-stage debiasing method whose central empirical claim rests on Table 1. The reader's weakest assumption correctly targets Eq. (2), but I would sharpen it: the construction of S from X already breaks the causal graph, and the one-sample approximation is not a Monte Carlo estimate unless inference averages over S. This matters because the HCRL module's entire contribution is framed as backdoor adjustment; if the test above shows single-sample and averaged inference differ, the reported Acc@t is not the causal quantity claimed. The paper also lacks code and reports no repeated runs, so the magnitude of the Table 1 gains cannot be checked. A secondary concern is Table 1's H2T row on VireoFood-172 (Acc@h 0.630 below Acc@t 0.876), which suggests at least one misreported baseline; the comparison table should be audited before the outperformance claim is accepted. None of this calls the authors' integrity into question; it means the empirical claim is conditional on a concrete check of the inference protocol. Therefore I keep a conditional verdict, but with the additional test above rather than code release alone.","tokens_in":13628,"tokens_out":9193,"duration_ms":91427,"concrete_test":"Take the CIFAR-100-LT ratio 0.02 TSCNet-ViT configuration. At inference, evaluate the stage-1 model under three protocols: (a) one randomly sampled confounder patch as the paper appears to describe; (b) the averaged predicted distribution over 100 dictionary patches; (c) no confounder patch. If (a) differs from (b) by more than the run-to-run standard deviation, the one-sample approximation in Eq. (2) fails. Additionally, replace the confounder dictionary with patches sampled from random training images; if accuracy is unchanged, the dictionary content is not acting as a semantic confounder and the causal interpretation is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Equation (1) is the standard backdoor formula only if S is a genuine confounder in the graph of Fig. 3. In Section 4.2, however, S is constructed per image by detecting the subject and applying a mask to x_i, so the dictionary entries are deterministic functions of the observed image; they cannot play the role of an exogenous confounder S -> X. The stronger problem is Eq. (2). The identity P(Y|do(X)) = sum_S P(Y|X,S)P(S) requires an expectation over S, but the implementation replaces it with one sampled confounder patch: P(Y|do(X)) ≈ P(Y|f(E(X),E(s_k))). A single random draw gives P(Y|X,S=s_k), not the marginal. Training with random s_k minimizes the expected conditional log-loss, not the log of the expected conditional, so the two objectives are not equivalent. The paper never specifies whether inference averages over s_k, uses one sample, or omits the patch; without that, Table 1's numbers cannot be identified with the backdoor-adjusted distribution. The ablation in Table 2 shows that adding random background patches helps, but that is consistent with standard input augmentation and does not validate the causal reading.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TSCNet, a two-stage method for long-tailed image classification with Vision Transformer backbones. In the first stage (HCRL), the method extracts class-agnostic background patches from training images to form a confounder dictionary and applies what it calls backdoor interventions at the patch level and the global feature level to reduce semantic bias. In the second stage (CLBC), it performs Fourier-based counterfactual augmentation with an adaptively adjusted per-class perturbation strength to calibrate logit bias caused by the long-tailed data distribution. The paper reports results on CIFAR-100-LT at imbalance ratios 0.01, 0.02, and 0.1 and on VireoFood-172, claiming state-of-the-art accuracy, especially with the VPT backbone, and includes an ablation study plus attention and confusion visualizations.","tokens_in":13904,"tokens_out":4170,"duration_ms":46010,"significance":"If the causal interpretation were fully supported, the paper would offer a modular two-stage recipe that is orthogonal to many existing long-tailed methods and that appears to improve tail-class accuracy on ViT backbones. The strengths are the clear ablation design in Table 2, the evaluation on two long-tailed benchmarks, and the attempt to make the intervention mechanism applicable to Transformer-based features. The paper is also honest enough to report head-class and tail-class accuracies separately, so the reader can see that the gains are not merely a redistribution of overall error. However, the central causal claims are currently not established: the backdoor adjustment is replaced by a single-sample approximation without an inference-time expectation, and the confounder dictionary is constructed from the observed images themselves, which conflicts with the exogenous-confounder graph in Figure 3. These issues are load-bearing because the paper's contribution is specifically a causal debiasing method, not just a heuristic augmentation scheme.","major_comments":[{"comment":"The single-sample approximation in Eq. (2) is not equivalent to the backdoor adjustment in Eq. (1). The identity P(Y|do(X)) = sum_S P(Y|X,S)P(S) requires an expectation over S, but the implementation uses one randomly sampled confounder patch s_k per image, giving P(Y|f(E(X),E(s_k))). Training with a random s_k minimizes E_s log P(Y|X,s), whereas the backdoor-adjusted predictor is log E_s P(Y|X,s); these are different objectives. The paper does not state whether inference averages over multiple s_k, uses one sample, or omits the patch entirely. Unless the inference procedure is specified and shown to approximate the expectation, the improvements attributed to patch-level intervention in Table 2 are compatible with standard input augmentation and do not validate the causal reading.","section":"§4.2, Eq. (2)"},{"comment":"The confounder S is not exogenous as required by the graph S -> X, S -> Y. In §4.2, each dictionary entry s_i is obtained by detecting the main subject of x_i and applying a mask, so S is a deterministic function of the observed image X. With S = h(X), the back-door path X <- S -> Y is not a standard confounding structure, and the condition for backdoor adjustment using S is not met. The authors should either redefine S as a separate variable sampled from an image-independent distribution, or explicitly state that the construction is an approximation and justify why deterministic masks can stand in for the confounder distribution P(S).","section":"§4.2, Confounder Dictionary and Figure 3"},{"comment":"The feature-level intervention is not derived. Eq. (4) asserts an NWGM approximation without specifying which network layer it applies to, and Eq. (5) replaces E_c[g(c)] by a weighted sum of prototype features, but g(c) is never defined and Eq. (6) uses an inconsistent indexing (1/N_i sum over N). It is therefore unclear how the learned weights mu_i correspond to P(S) or how the parameterized form in Eq. (5) represents the backdoor-adjusted conditional P(Y|X,S). Since the feature-level module contributes to the +F ablation gains in Table 2, this missing derivation is a load-bearing gap.","section":"§4.2, Eqs. (4)-(6)"},{"comment":"The central empirical claim that TSCNet \"outperforms existing methods\" rests on single-run comparisons with no error bars, repeated seeds, or significance tests. The margins in the ablation study (e.g., 0.864 vs. 0.877 for +I+F+C vs. +I+F+C+R on CIFAR-100-ratio0.02) are of a size that can easily be run-to-run variation for ViT training. Please report mean and standard deviation over at least three runs, or otherwise provide a statistical justification that the reported gaps are stable.","section":"§5.1 and Table 1"}],"minor_comments":[{"comment":"The variable VLoL appears in Eq. (13) but is not defined, and the update rule text says the strength increases \"from easy to difficult\" while the equations increase L_e^c when accuracy is high and decrease it otherwise; please clarify the intended update and specify bounds on L_e^c.","section":"§4.3, Eqs. (11)-(13)"},{"comment":"The text says \"S andB acts as confounder,\" but B is never defined, and the notation in Figure 3 uses D for the distribution confounder; please make the notation consistent.","section":"§4.1"},{"comment":"The summation indices are inconsistent: the first expression uses 1/N sum over j, while the final expression uses a single sampled k; please rewrite the equation with consistent notation.","section":"§4.2, Eq. (2)"},{"comment":"The H2T row on VireoFood-172 reports Acc@h = 0.630 and Acc@t = 0.876, which is qualitatively different from all other rows and may be a typo; please verify the reported values.","section":"Table 1"},{"comment":"There are several typos and broken references: \"Feature-level Invention model\" in §5.4, \"conterfactual\" after Eq. (10), \"Eq.6\" in the sentence above Eq. (5), and the citation \"[48]\" is not expanded in the bibliography.","section":"§5.4 and §4.2"},{"comment":"The term \"sparse mechanism shift (SMS)\" is introduced in the Introduction but is never defined or used in the method sections; please either define it and connect it to the two-stage design or remove it.","section":"§1"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical gains are plausible, but the causal mechanism is the main selling point and the current formulation does not support it. If the authors can reframe the method as a heuristic multi-scale debiasing strategy with a clearly specified inference procedure, or if they can provide a rigorous justification of the single-sample approximation and the confounder construction, the paper could become publishable. Without code or repeated-run statistics, I would be reluctant to rely on the exact margins in Table 1. Also, the paper uses CCIM, CaDeT, and GOAT, which come from emotion recognition, trajectory prediction, and vision-language navigation, as direct baselines for long-tail classification; the authors should justify this transfer in the experimental section."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nShort version: TSCNet reports real gains on long-tailed ViT classification, and the two-stage design is a sensible engineering contribution. But the causal machinery does not do what the paper says it does. The backdoor adjustment in Eq. (1)-(2) is not a backdoor adjustment as implemented.\n\nThe paper's legitimate contribution is identifying that existing causal long-tail methods (TDE, xERM) were built for CNNs and underperform on ViT. The HCRL stage — concatenating random background patches at the token level and using a prototype dictionary at the feature level — plus a counterfactual Fourier amplitude mixing stage with adaptive strength is a plausible recipe. The ablation shows each piece helps, and the gains over LPT, Gpaco, etc. are non-trivial (e.g., 0.887 vs 0.861 on CIFAR100-LT ratio 0.01 with VPT). That is worth taking seriously.\n\nSoft spots, in order of importance:\n\n1. The causal claim is unsupported. S is constructed from X by masking the subject, so it is a function of X, not an exogenous confounder. Eq. (1) requires S to be independent of X and to block the backdoor path; that is not the case. Even granting the graph, Eq. (2) replaces the expectation over P(S) with a single random draw and no mention of inference-time averaging. Training with a random s_k minimizes the expected conditional log-loss, which is not the same as the log of the expected conditional. The ablation in Table 2 shows random background patches help, but that is consistent with regularization or input augmentation; it does not validate the causal interpretation.\n\n2. Reproducibility. No code, no error bars, no repeated runs. Some hyperparameters (e.g., dictionary size l, thresholds) are missing. The table has an apparent error: H2T's Acc@t on VireoFood (0.876) exceeds Acc@h (0.630) and Acc@all is 0.798, which cannot be a simple average. That needs correcting.\n\n3. Baseline choice. CCIM, CaDeT, and GOAT are taken from other tasks and applied without adaptation; that weakens the claim that TSCNet beats state-of-the-art causal methods.\n\nThe causal framing is overclaimed, but the method and empirical results are not frivolous. The paper would benefit from a rewrite that describes the interventions as learned augmentations or regularizers, not as causal adjustments, and from a serious effort to release code and error bars.\n\nWho this is for: anyone working on long-tailed recognition with ViT or on causal claims in computer vision. It deserves a serious referee — the empirical core is strong enough to survive a critical reading, and the causal gap is fixable in presentation. My recommendation: engage with it, but keep the causal claims honest.\n\nBest","headline":"Real gains on long-tailed ViT, but the causal backdoor story does not hold up under inspection.","tokens_in":14429,"tokens_out":2572,"would_cite":true,"duration_ms":25045,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that TSCNet, a two-stage causal intervention framework, removes both semantic and distributional biases in long-tailed image classification and outperforms existing methods on ViT backbones.","keywords":["long-tailed image classification","vision transformer","causal inference","backdoor adjustment","counterfactual augmentation","logits bias calibration","fine-grained visual representation","class imbalance"],"falsifier":"On CIFAR-100-LT, train TSCNet with the confounder dictionary replaced by random noise patches of the same shape while keeping all other settings fixed; if tail accuracy does not drop below the reported dictionary version, the dictionary's semantic content is not doing the causal work. A complementary check is to replace the single-sample approximation in Eq. (2) with the full dictionary average: if tail accuracy does not improve or match, the causal graph or the approximation is mis-specified.","tokens_in":13443,"feed_emoji":"🎯","tokens_out":7989,"duration_ms":68408,"temperature":0.7,"pith_summary":"Long-tailed image classifiers trained on imbalanced data tend to lock onto head-class cues and misclassify visually similar tail classes. The paper argues that existing causal long-tail methods, built for CNNs, fail to transfer to Vision Transformers because ViT's global feature representation hides the associations between fine-grained features and predictions. It proposes TSCNet, which first performs backdoor interventions at both patch and feature levels to stop the model leaning on class-agnostic background information, then constructs a counterfactual balanced distribution to calibrate the logits. If the claim holds, ViT-based classifiers can reduce the head-tail performance gap without sacrificing head-class accuracy, and the approach transfers across CNN and transformer backbones.","feed_headline":"Two-stage causal method lifts tail accuracy in vision transformers","feed_subtitle":"The method removes background and distribution bias in transformers for long-tailed image classification.","key_machinery":"The load-bearing object is a structural causal model over image $X$, semantic confounder $S$, data-distribution confounder $D$, and prediction $Y$, with two backdoor paths $X \\leftarrow S \\rightarrow Y$ and $X \\leftarrow D \\rightarrow Y$. TSCNet is the two-stage framework that intervenes on these paths: patch-level intervention stacks random confounder patch embeddings with the image tokens, feature-level intervention uses a k-means++ prototype dictionary of confounder features pooled through normalized weighted geometric mean, and counterfactual logits bias calibration generates tail samples by interpolating Fourier amplitude spectra under a per-class strength $L_c^e$ that rises or falls by 0.1 depending on whether the class accuracy clears a threshold. Together these components are what let the model remove semantic bias at the representation level and distribution bias at the logit level.","core_discovery":"The central claim is that two distinct confounders drive long-tail errors—semantic factors such as background (S) and the imbalanced data distribution itself (D)—and that both can be neutralized with causal interventions suited to ViT. In the hierarchical causal representation learning stage, TSCNet masks out detected subjects, builds a confounder dictionary of class-agnostic patch images, and approximates the backdoor adjustment $P(Y|\\mathrm{do}(X))$ by concatenating a randomly sampled confounder patch at the token level and by integrating over confounder prototypes at the feature level. In the counterfactual logits bias calibration stage, it perturbs the Fourier amplitude spectrum of tail-class images with an adaptively adjusted strength per class, creating a counterfactual balanced distribution that recalibrates the decision boundary. On CIFAR-100-LT with ratio 0.01, TSCNet with VPT reaches 0.887 overall accuracy and 0.830 tail accuracy, and on VireoFood-172 it reaches 0.875, both above the compared baselines, while head-class accuracy stays around 0.93.","pith_inferences":["If the causal graph is right, the same two-stage intervention—background confounder dictionary plus counterfactual distribution—could be applied to other transformer-based vision tasks where background is a known confounder, such as scene classification or fine-grained recognition; the paper does not test this.","The one-sample approximation in Eq. (2) implies a testable scaling prediction: sampling several confounder patches per image or enlarging the dictionary should improve tail accuracy up to a saturation point; the paper reports only fixed dictionary settings.","Because the second stage perturbs only Fourier amplitude and keeps phase intact, the method implicitly claims phase carries the causally relevant class information; replacing amplitude interpolation with phase interpolation should degrade performance if that claim is true."],"forward_implications":["Causal long-tail methods on ViT should intervene on representations, not only calibrate logits; the paper's comparisons show TDE and xERM gain little on ViT while TSCNet improves tail accuracy substantially.","Head-class performance need not be sacrificed to improve tail classes: TSCNet reports head accuracy around 0.93 while tail accuracy rises to 0.83 on CIFAR-100-LT ratio 0.01 and to 0.82 on VireoFood-172.","The approach is backbone-agnostic: the same two stages improve ResNet50, plain ViT, and VPT, suggesting the intervention mechanism transfers rather than depending on one architecture's training recipe.","Adaptive per-class counterfactual strength outperforms fixed perturbation strengths, so tuning augmentation intensity by observed class accuracy is part of the method's effect."],"supporting_citations":[{"why":"Supplies the TDE baseline and the head/middle/tail evaluation protocol used for comparison; the paper argues this logits-only causal calibration transfers poorly to ViT.","marker":"[Tang et al., 2020]"},{"why":"Supplies the xERM cross-domain baseline whose ViT behavior motivates the paper's diagnosis of similar-class confusion.","marker":"[Zhu et al., 2022]"},{"why":"Supplies LPT, the strongest prompt-tuning baseline TSCNet must beat on VPT.","marker":"[Dong et al., 2023]"},{"why":"Supplies LiVT, the ViT long-tail baseline used in comparison.","marker":"[Xu et al., 2023]"},{"why":"Supplies Gpaco, a contrastive long-tail baseline used in comparison.","marker":"[Cui et al., 2023]"},{"why":"Provides the Fourier amplitude perturbation technique that underlies counterfactual generation in CLBC.","marker":"[Lv et al., 2022]"},{"why":"Provides the Grad-CAM tool used to detect main subjects and build the class-agnostic confounder dictionary.","marker":"[Selvaraju et al., 2020]"},{"why":"Contributes the VireoFood-172 dataset used for the main long-tail food recognition benchmark.","marker":"[Chen and Ngo, 2016]"},{"why":"Supplies the k-means++ algorithm used to build the feature-level confounder prototype dictionary.","marker":"[Bahmani et al., 2012]"}],"fun_headline_variants":["Causal two-stage method cuts ViT long-tail errors","TSCNet: remove background and distribution bias for long-tail ViT","Backdoor and counterfactual calibration improve ViT tail accuracy","Multi-scale causal intervention removes ViT long-tail confounders","Fourier perturbation and backdoor adjustment fix long-tail ViT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the background patches it masks out of images are exactly the class-irrelevant cues the model latches onto, and that looking at one randomly chosen background patch per image is enough to undo their influence; if the real spurious cues are not in those masks, the causal fix misses the true bias.","fun_headline_variants_meta":{"raw":{"variants":["Causal two-stage method cuts ViT long-tail errors","TSCNet: remove background and distribution bias for long-tail ViT","Backdoor and counterfactual calibration improve ViT tail accuracy","Multi-scale causal intervention removes ViT long-tail confounders","Fourier perturbation and backdoor adjustment fix long-tail ViT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001288,"raw_usage":{"total_tokens":5297,"prompt_tokens":1020,"completion_tokens":4277,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":4189}},"tokens_in":636,"tokens_out":4277,"duration_ms":28567,"temperature":1.0,"reasoning_tokens":4189,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:01:21.770946+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On CIFAR-100-LT, train TSCNet with the confounder dictionary replaced by random noise patches of the same shape while keeping all other settings fixed; if tail accuracy does not drop below the reported dictionary version, the dictionary's semantic content is not doing the causal work. A complementary check is to replace the single-sample approximation in Eq. (2) with the full dictionary average: if tail accuracy does not improve or match, the causal graph or the approximation is mis-specified.","supporting_citations":[{"cited_title":"Long-tailed classification by keeping the good and removing the bad momentum causal ef- fect","cited_arxiv_id":null,"evidence_quote":"Supplies the TDE baseline and the head/middle/tail evaluation protocol used for comparison; the paper argues this logits-only causal calibration transfers poorly to ViT."},{"cited_title":"Cross-domain empirical risk min- imization for unbiased long-tailed classification","cited_arxiv_id":null,"evidence_quote":"Supplies the xERM cross-domain baseline whose ViT behavior motivates the paper's diagnosis of similar-class confusion."},{"cited_title":"Lpt: Long-tailed prompt tuning for image classification","cited_arxiv_id":null,"evidence_quote":"Supplies LPT, the strongest prompt-tuning baseline TSCNet must beat on VPT."},{"cited_title":"Generalized para- metric contrastive learning","cited_arxiv_id":null,"evidence_quote":"Supplies Gpaco, a contrastive long-tail baseline used in comparison."},{"cited_title":"Grad-cam: visual explanations from deep networks via gradient-based localization","cited_arxiv_id":null,"evidence_quote":"Provides the Grad-CAM tool used to detect main subjects and build the class-agnostic confounder dictionary."},{"cited_title":"Deep-based ingredient recognition for cooking recipe re- trieval","cited_arxiv_id":null,"evidence_quote":"Contributes the VireoFood-172 dataset used for the main long-tail food recognition benchmark."}],"review_version":1}