{"id":"d7272f75-b183-49a5-b08b-b219faae932e","arxiv_id":"2412.13516","paper_version":4,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The authors argue that a causal transition matrix P(Ŷ|do(Y),X) for instance-dependent label noise is identifiable when a noise-sensitive component X2 of the input is recovered, and they train a framework that separates X into X1 and X2.","lead":"This paper proposes a causal graph for instance-dependent label noise that splits each input into a noise-resistant and a noise-sensitive component, and defines a 'causal transition matrix' P(Ŷ|do(Y),X) claimed to be identifiable. The authors combine co-teaching, a decorrelation loss, and policy gradient to train a classifier that infers clean labels, reporting gains over many baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's proof fails: conditioning on X2 opens a collider path, so the claimed d-separations do not hold; the identifiability guarantee is unsupported.","rationale":"The reader's weakest_assumption correctly identifies the flawed d-separation step in Theorem 1. My analysis confirms and sharpens this: the collider X2 opens a backdoor path in both conditional independences used in the proof, so the two equalities in Theorem 1 do not follow from the graph. This is the single most load-bearing concern because the paper's novelty and theoretical guarantee rest entirely on Theorem 1; without it, there is no identifiability result for the causal transition matrix, and the method's advertised consistency guarantee is unsupported. The concrete d-separation test is decisive because it checks the exact independences used in the proof against the graph topology; no other concern (e.g., missing error bars, best-of-10 reporting, or absence of code) is as fundamental, as those affect empirical credibility but not the core theoretical claim. Since the reader already recommended REJECT and this stress-test reinforces that verdict, no change is needed.","tokens_in":18732,"tokens_out":6065,"duration_ms":48728,"concrete_test":"Run an automated d-separation check on the graph in Fig. 2(a), using the structure described in the section 'Causal Viewpoint for Denoising': edges X→X1, X→X2, X1→Y, Y→X2, Y→Ŷ, Z→X2, Z→Ŷ, X2→Ŷ. For G_Ȳ, delete the edge X1→Y and test whether Ŷ ⊥ X1 | {Y, X2}; for G_Y, delete edges Y→X2 and Y→Ŷ and test whether Ŷ ⊥ Y | X2. If either test returns 'dependent' (as predicted by the collider path X1←X→X2←Z→Ŷ), the proof of Theorem 1 is invalid. This can be implemented in pgmpy's d_separation method or verified by hand tracing the open path after conditioning on X2.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central theoretical claim is Theorem 1: P(Ŷ|do(Y),X) is identifiable if the noise-predictive factor X2 is recovered. The proof relies on two d-separation assertions. First, in G_Ȳ (the graph with incoming edges of Y removed), it claims Ŷ ⊥ X1 | Y, X2. But in the proposed graph, X2 is a collider on the path X1 ← X → X2 ← Z → Ŷ: it receives edges from both X and Z. Conditioning on a collider opens the path, so X1 and Ŷ are d-connected given X2 (regardless of Y); the asserted independence is false. Second, in G_Y (the graph with outgoing edges of Y removed), it claims Ŷ ⊥ Y | X2. The collider path Y ← X1 ← X → X2 ← Z → Ŷ connects Y to Ŷ once X2 is conditioned on, so this independence also fails. Consequently, the equalities P(Ŷ|do(Y),X) = P(Ŷ|do(Y),X2) and P(Ŷ|do(Y),X2) = P(Ŷ|Y,X2) are not justified by the graph. Theorem 1 is therefore unproven, and the paper's central identifiability guarantee collapses. Even if the method works empirically, the theoretical contribution—relaxing instance independence with a guaranteed identifiable causal transition matrix—is unsupported. A secondary issue, also raised by the reader, is that Theorem 1 presupposes recovery of X2, but no identifiability result for X2 from (X,Ŷ) is provided; X2 is learned via a policy-gradient objective that could converge to any representation minimizing the same cross-entropy, making the guarantee conditional on the fitted network.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a causal-graph view of instance-dependent label noise in which the observed instance X is decomposed into a noise-resistant component X1 and a noise-sensitive component X2, with an unobserved latent Z affecting X2 and the noisy label Y-hat. On this basis it defines a 'causal transition matrix' P(Y-hat | do(Y), X), claims in Theorem 1 that this object is identifiable once X2 is recovered, and designs an end-to-end training framework combining co-teaching, a transition model, a decorrelation regularizer, and a policy-gradient objective for X2. Experiments on FashionMNIST, SVHN, CIFAR-10/100, Food101, and Clothing1M report competitive accuracy. The main theoretical claim, however, is not supported: the d-separation steps in the proof of Theorem 1 fail under the paper's own graph, and the recovery of X2 is not established as an identification result.","tokens_in":19197,"tokens_out":8835,"duration_ms":79472,"significance":"If the identifiability claim were valid, the paper would offer a meaningful relaxation of the instance-independent transition-matrix assumption, and the proposed framework is a plausible engineering contribution: it is end-to-end trainable, avoids generative models, and the experiments are extensive, including real-world noise and an ablation. The paper also deserves credit for explicitly modeling three noise-generation scenarios in one graph and for discussing limitations. Unfortunately, the central theoretical guarantee is unsound: the proof of Theorem 1 misapplies d-separation, and the 'identifiability' is conditional on a learned representation whose recovery is not guaranteed. The empirical results, while often strong, do not compensate for the absence of a valid identifiability theorem, so the paper's headline contribution is not established.","major_comments":[{"comment":"The proof asserts two conditional independences that are false in the graph of Figure 2(a). First, it claims Ŷ ⊥ X1 | Y, X2 in G_Ȳ. But the path X1 ← X → X2 ← Z → Ŷ contains the collider X2, and conditioning on X2 opens this path, so X1 and Ŷ are d-connected given (Y, X2); conditioning on Y does not block the path. Second, it claims Ŷ ⊥ Y | X2 in G_Y, and the same collider path Y ← X1 ← X → X2 ← Z → Ŷ connects Y to Ŷ once X2 is conditioned on. Therefore the equalities P(Ŷ|do(Y),X) = P(Ŷ|do(Y),X2) and P(Ŷ|do(Y),X2) = P(Ŷ|Y,X2) are not justified by the graph, and Theorem 1 is unproven.","section":"Causal Viewpoint for Denoising, Theorem 1 proof"},{"comment":"The theorem is conditional on 'recovering' the noise predictive factor X2, but no identifiability result for X2 from the observed distribution (X, Ŷ) is provided. In the framework, X2 is the output of a policy network g2 trained by the policy-gradient loss of Eq. (7), whose reward is computed from the same cross-entropy LCE used by the transition model. Any representation that minimizes that cross-entropy could serve as X2, and the paper gives no argument that the optimized g2(X) coincides with the true noise-sensitive component of the proposed causal graph. The claimed identification is therefore a property of the fitted network, not of the data-generating process.","section":"Theorem 1 statement and Section 'Training Framework for Denoising'"},{"comment":"The decorrelation loss is written as Reg(X1) = −Σ_i \\tilde{y}_{x1} log softmax(1_k). Since 1_k is the all-one vector, softmax(1_k) is the uniform distribution and log softmax(1_k) is a constant vector that does not depend on the model parameters. As written, the loss does not push the transition model's output toward a uniform distribution over class labels; it merely penalizes the sum of the output logits (or is constant if \\tilde{y}_{x1} is already normalized). If the intended objective is a cross-entropy between the model's prediction and the uniform distribution, the first argument should be softmax(\\tilde{y}_{x1}), not the raw output. As published, Eq. (3) does not implement the decorrelation between X1 and Ŷ that the causal-graph assumptions require.","section":"Eq. (3), 'Training Framework for Denoising'"},{"comment":"The paper motivates T_cau = P(Ŷ|do(Y),X) as an approximation of the true transition matrix P(Ŷ|Y,X), but no statement or proof connects the interventional distribution to the observational transition matrix used in loss correction. Intervening on Y removes incoming edges of Y, whereas the transition matrix is a conditional distribution given Y and X; equality or a bounded approximation error between T_cau and T(X) is never shown. Even if Theorem 1 were correct, it would identify a do-probability, and the paper would still need an argument that training with T_cau yields consistent clean-label inference.","section":"Causal Viewpoint for Denoising, definition of T_cau"}],"minor_comments":[{"comment":"The table header reads 'Results on SVNH'; the dataset name should be 'SVHN'.","section":"Table 2"},{"comment":"The sentence 'we approa the problem from a causal viewpoint' is missing 'ch' in 'approach'.","section":"Conclusion"},{"comment":"The paragraph beginning 'What our framework does?' refers to 'Theories 1 and 2'; the standard terminology is 'Theorems 1 and 2'.","section":"Analysis of the Framework"},{"comment":"The ablation 'Ours w.o/ pg' outperforms the full method on SYM-80 (75.8 vs 71.5) and IDN-20 (91.3 vs 91.0); the text's claim that the ablation achieves the best performance in the symmetric and asymmetric scenarios is not consistent with all entries in the table and should be qualified.","section":"Table 1"}],"recommendation":"reject","confidential_remarks":"The d-separation flaw in Theorem 1 is decisive: it invalidates the paper's central identifiability guarantee, and the issue is not a local presentational fix because the collider structure is part of the proposed causal graph. The additional gap between the interventional 'causal transition matrix' and the observational transition matrix further weakens the theoretical contribution. The experimental study is substantial, but the manuscript's advertised theoretical claim is not defensible in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's main idea is to split the instance X into a noise-resistant part X1 and a noise-sensitive part X2, and define a causal transition matrix P(Ŷ|do(Y),X) that is claimed to be identifiable once X2 is recovered. That graph and definition are genuinely new relative to CausalNL and NPC, and the end-to-end training framework avoids VAEs, which is a nice practical simplification. The synthetic experiments are broad, report standard deviations, and the ablation 'w.o/pg' suggests the separation model alone helps.\n\nThe problem is the proof of Theorem 1. In G_Ȳ the paper asserts Ŷ ⊥ X1 | Y, X2, but in the very graph they propose, X2 is a collider on the path X1 ← X → X2 ← Z → Ŷ. Conditioning on X2 opens that path, so the independence is false. The second independence, Ŷ ⊥ Y | X2 in G_Y, fails the same way via Y ← X1 ← X → X2 ← Z → Ŷ. Therefore the equalities P(Ŷ|do(Y),X)=P(Ŷ|do(Y),X2) and P(Ŷ|do(Y),X2)=P(Ŷ|Y,X2) are not justified. The theorem is unproven, and with it the paper's central claim of identifiable instance-dependent transition matrices.\n\nA secondary issue: Theorem 1 is conditional on recovering X2, but the paper gives no identifiability result for X2; X2 is learned by a policy-gradient objective that minimizes the same cross-entropy, so the guarantee is at best conditional on the fitted network. The real-world results are reported as best of last 10 epochs without error bars, and code is not released, so the empirical evidence is weaker than the synthetic tables suggest.\n\nThe framework may still work as a heuristic—the ablation numbers are surprisingly strong—but the theory section does not support what the abstract promises. This paper deserves a serious referee because the problem is important and the causal graph is a real attempt to relax instance independence, but the referee should be told to check Theorem 1 carefully. I would not expect acceptance as is; a corrected proof, or an honest reframing that drops the identifiability claim, would be needed.","headline":"New causal graph and training framework for instance-dependent label noise, but the central identifiability proof is invalid because conditioning on the collider X2 opens a backdoor path.","tokens_in":19706,"tokens_out":4000,"would_cite":false,"duration_ms":32298,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Instance-dependent label noise becomes identifiable when each input is split into a noise-resistant component and a noise-sensitive component and the latter is recovered.","keywords":["causal transition matrix","instance-dependent label noise","label-noise learning","causal graph","identifiability","do-calculus","noise-resistant component","policy gradient"],"falsifier":"Build a synthetic dataset with known clean component $X_1$ and known noise component $X_2$, generate $Y$ from $X_1$ and $\\hat{Y}$ from $(Y,X_2)$ with a known transition, then train the framework without revealing $X_2$. If the estimated $P(\\hat{Y}\\mid do(Y),X)$ deviates systematically from the known $P(\\hat{Y}\\mid Y,X_2)$, or if the learned $X_2$ fails the conditional independence $\\hat{Y}\\perp X_1\\mid Y,X_2$ on held-out data, the central identifiability claim would be falsified.","tokens_in":18541,"feed_emoji":"🧩","tokens_out":8028,"duration_ms":65912,"temperature":0.7,"pith_summary":"Noisy labels are usually handled by assuming the probability of a wrong label does not depend on the instance, an assumption that fails in practice. This paper replaces that assumption with a causal story: an unobservable cause $Z$ can corrupt the image, the annotator, or both, so each input $X$ is split into a noise-resistant component $X_1$ that drives the clean label and a noise-sensitive component $X_2$ that drives the noise. The central claim is that the ``causal transition matrix'' $P(\\hat{Y}\\mid do(Y),X)$, the noisy-label distribution after intervening on the clean label, is identifiable once $X_2$ is recovered, and it approximates the true instance-dependent transition matrix. A training framework realizes the graph with co-teaching, a decorrelation penalty, and a policy-gradient model for $X_2$. The paper itself notes that its main limitation is that the clean label is only partially observable through confidence sampling, which can strain the method at very high noise rates.","feed_headline":"A causal graph makes instance-dependent label noise learnable","feed_subtitle":"Splitting each input into a noise-resistant and a noise-sensitive part turns the transition matrix into an identifiable target.","key_machinery":"The central object is the causal transition matrix $T_{cau}=P(\\hat{Y}\\mid do(Y),X)$: the distribution of the noisy label after an intervention forces the clean label, conditioning on the instance. The paper's graph gives it a two-part structure; $do(Y)$ removes the incoming edges of $Y$, so $X_1$, the noise-resistant parent of $Y$, drops out of the transition, while conditioning on $X_2$, the noise-sensitive component, converts the intervention distribution into the observational one $P(\\hat{Y}\\mid Y,X_2)$. The proof machinery is d-separation in the manipulated graphs $G_{\\bar{Y}}$ and $G_Y$, using the backdoor criterion. The training machinery is a separation model that extracts $X_1$, a policy model $\\pi$ that outputs $X_2$, a transition model $f_{tran}$ with Gumbel-Softmax merging, co-teaching to make $Y$ partially observable, and a policy-gradient reward that pushes $X_2$ to reduce prediction error on noisy labels.","core_discovery":"On the paper's own terms, the discovery is that the transition matrix does not have to be assumed instance-independent if the data generation is modeled causally. In the proposed graph, an unobservable variable $Z$ affects one or both of the input $X$ and the noisy label $\\hat{Y}$; $X$ is therefore separated into $X_1$, which is independent of $Z$ and determines $Y$, and $X_2$, which interacts with $Z$ and $Y$ to produce $\\hat{Y}$. Theorem 1 states that $P(\\hat{Y}\\mid do(Y),X)=P(\\hat{Y}\\mid do(Y),X_2)=P(\\hat{Y}\\mid Y,X_2)$, so the instance-dependent causal transition matrix is identifiable whenever the noise predictive factor $X_2$ is recovered. Theorem 2 states that $P(Y\\mid do(X_1))=P(Y\\mid X_1)$, so a classifier built on $X_1$ alone is unbiased. These two results together say that learning with instance-dependent label noise can be reduced to recovering the two components of $X$, and the paper proposes an end-to-end framework that does this by treating $X_2$ as the action of a policy optimized by reward.","pith_inferences":["The identifiability guarantee is a reduction: it says the causal transition matrix is identified once $X_2$ is recovered, but the paper gives no separate proof that the fitted policy network recovers the true $X_2$; we would read Theorem 1 as conditional on that recovery.","The authors' own ablation without the policy model remains competitive in most settings, which suggests to us that much of the accuracy gain may come from learning a noise-resistant $X_1$ rather than from the estimated causal transition matrix; an experiment with known $X_2$ would separate the two effects.","If the reduction view is right, instance-dependent label-noise learning becomes a missing-component problem, and the same causal split could be tried for other weakly supervised settings such as partial labels, crowd-sourced annotations, or noisy medical labels, though the paper does not test those.","A skeptical check would be to verify the key conditional independence $\\hat{Y}\\perp X_1\\mid Y,X_2$ on the learned components; in the paper's own graph this independence is what makes Theorem 1 go through."],"forward_implications":["Instance-dependent label noise can be handled without the instance-independent assumption $P(\\hat{Y}\\mid Y,X)=P(\\hat{Y}\\mid Y)$, as long as the noise-sensitive component of the input can be learned.","A clean classifier can be trained from the noise-resistant component alone, because $P(Y\\mid do(X_1))=P(Y\\mid X_1)$ removes the confounding effect of the latent noise variable.","The causal graph unifies three previously separate noise scenarios, since $Z$ may affect the instance, the annotation, or both, and all three are captured by the same transition model.","The same component-splitting idea transfers to semi-supervised denoising pipelines, where the paper's adaptation improves or matches a strong baseline on the CIFAR-N datasets."],"supporting_citations":[{"why":"Supplies the backdoor adjustment and do-calculus that define and identify the causal transition matrix.","marker":"Pearl 1995"},{"why":"The canonical instance-independent transition assumption that the paper relaxes and compares against.","marker":"Natarajan et al. 2013"},{"why":"Baseline loss-correction method that estimates and uses the transition matrix to train on noisy labels.","marker":"Patrini et al. 2017"},{"why":"Co-teaching, used in the framework to make the unobservable clean label Y partially observable.","marker":"Han et al. 2018"},{"why":"CausalNL, the closest causal baseline, which assumes posterior instance independence that this method removes.","marker":"Yao et al. 2021"},{"why":"Policy-gradient machinery used to train the noise-sensitive component X2 when its gradient through the transition model is blocked.","marker":"Sutton and Barto 2018"}],"fun_headline_variants":["Causal graph identifies instance-dependent noise transition","Splitting inputs resolves label-noise identifiability","Two-part input makes transition matrix identifiable","Causal decomposition uncovers label-noise mechanism"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each observed instance really splits into a noise-resistant part and a noise-sensitive part with the exact independencies drawn in the paper's causal graph, and that the noise-sensitive part can actually be recovered from the noisy data; if either condition fails, the causal transition matrix is not identifiable.","fun_headline_variants_meta":{"raw":{"variants":["Causal graph identifies instance-dependent noise transition","Splitting inputs resolves label-noise identifiability","Two-part input makes transition matrix identifiable","Causal decomposition uncovers label-noise mechanism"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00115,"raw_usage":{"total_tokens":4817,"prompt_tokens":1045,"completion_tokens":3772,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":3716}},"tokens_in":661,"tokens_out":3772,"duration_ms":26625,"temperature":1.0,"reasoning_tokens":3716,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:03:10.228526+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a synthetic dataset with known clean component $X_1$ and known noise component $X_2$, generate $Y$ from $X_1$ and $\\hat{Y}$ from $(Y,X_2)$ with a known transition, then train the framework without revealing $X_2$. If the estimated $P(\\hat{Y}\\mid do(Y),X)$ deviates systematically from the known $P(\\hat{Y}\\mid Y,X_2)$, or if the learned $X_2$ fails the conditional independence $\\hat{Y}\\perp X_1\\mid Y,X_2$ on held-out data, the central identifiability claim would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the backdoor adjustment and do-calculus that define and identify the causal transition matrix."},{"cited_title":"S.; Ravikumar, P","cited_arxiv_id":null,"evidence_quote":"The canonical instance-independent transition assumption that the paper relaxes and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline loss-correction method that estimates and uses the transition matrix to train on noisy labels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CausalNL, the closest causal baseline, which assumes posterior instance independence that this method removes."}],"review_version":1}