{"id":"ab23f043-e3e5-41e5-957d-70ce6b521065","arxiv_id":"2504.18026","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Concept Preference Optimization, a DPO-based loss for concept bottleneck models, improves task accuracy and noise robustness over binary cross-entropy.","lead":"This paper proposes a new loss, Concept Preference Optimization, that trains concept bottleneck models by comparing each image's concept label against a model-sampled alternative, aiming to reduce the damage from mislabeled concepts. The authors report accuracy gains over binary cross-entropy on three datasets, with the strongest gains under added concept noise.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CPO's noise-robustness proof omits the sampling-gradient term; the exact gradient of Eq. 5 can exceed BCE's, so Proposition 4.2 and Theorem 4.3 fail as stated.","rationale":"The reader's weakest-assumption analysis is exactly right. The omitted dependence of the sampling distribution on theta is not a minor technical slip: it changes the sign and magnitude of the CPO gradient in meaningful regimes, and the counterexample at p=0.1 invalidates Proposition 4.2 as stated. Because Proposition 4.1, Proposition 4.2, and Theorem 4.3 form the theoretical backbone of the claim that CPO is 'inherently less sensitive' to concept noise, this gap is load-bearing. I also note the RandInt asymmetry as a separate confound for the empirical headline, but the proof gap is sufficient to justify conditional acceptance. The paper still has substantial independent value: broad experiments across three datasets, no added parameters, low computational overhead, and released code. A corrected derivation and a RandInt ablation could plausibly restore the central claims, so rejection is not warranted; the appropriate action is to require those fixes before full acceptance.","tokens_in":30526,"tokens_out":9087,"duration_ms":103149,"concrete_test":"Independently re-derive the gradient of Eq. 5 without treating c'~pi_theta as fixed: compute d/dtheta E_{c'~pi_theta}[-log sigma(log pi_theta(c|x) - log pi_theta(c'|x))] for a single binary concept, and evaluate the norm inequality in Proposition 4.2 at p in {0.1, 0.2, ..., 0.9}. If |g_CPO| exceeds |g_BCE| at any p, the proposition and Theorem 4.3 fail for the exact objective.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is that CPO is inherently less sensitive to concept noise than BCE. That claim rests on Proposition 4.2, whose proof in App. C.1 (Eqs. 25-26) computes the expected gradient of Eq. 5 as if the sampled concept c'~pi_theta(c|x) were exogenous. But L_CPO is an expectation over pi_theta, and the event c'=c contributes through the probability pi_theta(c|x) even though the integrand is constant there. For a single binary concept with p=pi_theta(c|x), the exact logit-space gradient for c=1 is g_CPO = -(1-p)^2 + p(1-p)(log p + log 2), whereas the paper's formula is -(1-p)^2 and BCE's gradient is p-1. At p=0.1, |g_CPO| ~= 0.955 > |g_BCE| = 0.9, directly contradicting the bound in Proposition 4.2. Proposition 4.1 similarly drops the p*log(2) contribution from the c'=c event. Since Theorem 4.3 inherits Proposition C.1, the 'inherently less sensitive' conclusion is not established by the written derivation. A separate, empirical threat is that App. A.1 gives RandInt augmentation to CPO-trained models but not to the CBM BCE baseline, so the headline comparisons may reflect augmentation rather than the loss; the theory gap alone is sufficient to make the central claim unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Concept Preference Optimization (CPO), a DPO-style loss for training Concept Bottleneck Models, and claims that CPO directly optimizes the concept posterior, is inherently less sensitive to concept-label noise than binary cross-entropy (BCE), and consistently outperforms BCE on CUB, AwA2, and CelebA with and without added noise. The theoretical component derives simplified forms of the CPO objective, a gradient bound relative to BCE, and a theorem about noise robustness; the empirical component compares CPO-trained CBMs and CEMs against BCE-trained baselines and ProbCBMs on task accuracy, concept AUC, intervention performance, uncertainty quantification, and noisy-label settings. The paper also includes a streaming-data experiment and releases code.","tokens_in":30815,"tokens_out":5068,"duration_ms":51581,"significance":"If the central claims were established, CPO would be a valuable, low-overhead drop-in loss for CBMs: it requires no extra parameters, adds modest compute, and appears to improve task accuracy and intervention behavior. The paper has concrete strengths: it ships code, evaluates across three datasets and several architectures, probes structured and uniform noise, includes an uncertainty study, and connects the loss to control-as-inference, which is a useful pedagogical framing. However, the central theoretical claim that CPO is 'inherently less sensitive' to concept noise rests on a gradient derivation that drops a theta-dependent term, and the headline empirical comparison is potentially confounded by the use of RandInt augmentation for CPO-trained models but not for the standard BCE baseline. The significance is therefore conditional on repairing the proof and deconfounding the experiments.","major_comments":[{"comment":"This comment is complete.","section":"Sec. 4.2 / App. C.1, Eqs. (25)-(26), Prop. 4.2"},{"comment":"This comment is complete.","section":"Prop. 4.1 / App. C.1, Eq. (6)"},{"comment":"This comment is complete.","section":"Sec. 4.2 / App. C.2, Theorem 4.3"},{"comment":"This comment is complete.","section":"App. A.1 / Table 1 / Figs. 5-6"}],"minor_comments":[{"comment":"This comment is complete.","section":"App. C.1, Eq. (11)"},{"comment":"This comment is complete.","section":"App. H, first paragraph"},{"comment":"This comment is complete.","section":"App. F.1"},{"comment":"This comment is complete.","section":"Table 1"},{"comment":"This comment is complete.","section":"App. A / App. I"},{"comment":"This comment is complete.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The core theoretical claim is not currently supported: the omitted theta-dependent term in App. C.1 is not a cosmetic issue, since it can flip the bound in Proposition 4.2. I would also urge the editor to require the authors to deconfound the RandInt augmentation before the empirical head-to-head comparison is taken at face value. The paper's empirical scope and released code are strengths, so I see this as a fixable major revision rather than a rejection, but the 'inherently less sensitive' claim should be removed or very precisely qualified if the proof cannot be repaired."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes Concept Preference Optimization (CPO), a DPO-style objective for training concept bottleneck models, and shows consistent gains on CUB, AwA2, and CelebA across clean, noisy, and structured-noise settings, with code released. The novelty is real: ProbCBM and SCBM amortize posteriors over a latent variable, while CPO applies preference optimization directly to concept predictions. The empirical work is broad—noise levels, interventions, streaming data, uncertainty calibration—and the method is simple and cheap. Worth taking seriously.\n\nThe soft spots are where the central claim lives. The proof that CPO is inherently less sensitive to noise rests on the App. C.1 gradient derivation, and that derivation is not right. L_CPO is an expectation over c' ~ pi_theta; the event c'=c gives a constant log sigma(0), but the probability of that event is pi_theta(c|x), which depends on theta. Dropping it misses a term. For a single binary concept, the exact logit-space gradient at pi=0.1 is roughly -0.955, which is larger in magnitude than BCE's -0.9, directly contradicting Proposition 4.2. Theorem 4.3 inherits the problem, so the 'inherently less sensitive' conclusion is not established as written. That does not mean the method is bad—empirically it appears to work—but the theory needs either a corrected derivation or a weaker claim.\n\nSecond, the headline empirical comparison has a confound: CPO-trained models receive the RandInt augmentation (25% random concept interventions) while the plain CBM BCE baseline does not (App. A.1). That is a meaningful extra training signal, and some of the gain could come from it. The paper mentions this but never ablates it. A simple fix: run the BCE baseline with the same RandInt setting.\n\nThird, Proposition 4.2 and Theorem 4.3 assume deterministic noise, while the experiments use random label flipping. That mismatch makes the theory support the empirical story less cleanly than the paper suggests.\n\nThe uncertainty analysis is the weakest part only in that it is mostly qualitative; the quantitative occlusion analysis shows CPO spreads uncertainty more evenly than BCE, but the aggregate differences are modest. The streaming-data experiment is exploratory but fine.\n\nIf I were advising the authors: correct or explicitly weaken the gradient theorem, add a BCE+RandInt ablation, and restate the noise theory for the actual noise model used. As it stands, the method is promising and the empirical pattern is consistent, but the central theoretical claim is unverified.","headline":"CPO is a genuinely new, DPO-style loss for concept bottleneck training with consistent empirical gains, but the proof that it is inherently noise-robust has a real gradient-derivation gap and the main comparison is confounded by RandInt.","tokens_in":31380,"tokens_out":4643,"would_cite":true,"duration_ms":43102,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A preference-optimization training loss makes concept bottleneck models robust to mislabeled concepts.","keywords":["concept bottleneck models","label noise","preference optimization","direct preference optimization","interpretability","uncertainty estimation","concept mislabeling","robustness"],"falsifier":"Train a logistic concept predictor on a single binary concept with noisy labels and compute the exact expected update of the CPO loss, including the case where the sampled concept equals the empirical one. At model confidence 0.1, if this update has larger norm than the BCE update, the bound in Proposition 4.2 is violated.","tokens_in":2022,"feed_emoji":"🛡️","tokens_out":2053,"duration_ms":82644,"temperature":0.7,"pith_summary":"Concept Bottleneck Models (CBMs) make neural predictions explainable by routing them through human-understandable concepts, but they are normally trained with binary cross-entropy (BCE), which assumes the concept labels are correct. This paper argues that concept labels are frequently wrong, whether from annotation errors, subjective judgments, or data augmentations that occlude concepts, and shows this can cut task accuracy by up to 25%. It proposes Concept Preference Optimization (CPO), an adaptation of Direct Preference Optimization that compares each empirical concept label with a concept sampled from the model's own policy, and claims CPO directly optimizes the concept posterior while being inherently less sensitive to label noise than BCE. Empirically, CPO-trained CBMs and Concept Embedding Models outperform their BCE counterparts on CUB, AwA2, and CelebA, both with clean labels and with up to 40% of concept labels flipped. A reader should care because CPO offers a drop-in training loss that makes interpretable models usable in noisy, high-stakes settings.","feed_headline":"Preference loss shields concept models from noisy labels","feed_subtitle":"CPO beats binary cross-entropy on CUB, AwA2, and CelebA, clean or noisy, with no extra parameters.","key_machinery":"The central object is the CPO loss (Eq. 5), an online Direct Preference Optimization objective in which the empirical concept $c$ is treated as preferred over a concept $c'\\sim\\pi_\\theta$ sampled from the current policy, with the reward reparameterized as the log-ratio of the policy to a prior. Under a uniform prior this becomes a log-likelihood term that contributes a gradient only when $c'\\neq c$, so the model updates only when it disagrees with the label, the mechanism that gives CPO its conservative, noise-attenuating updates. The supporting machinery is a control-as-inference identity: introducing an optimality variable $o$ with $p(o=1|x,c)=\\exp(r^*(x,c))$ yields the posterior $\\pi(c|o=1,x)\\propto \\pi_0(c|x)\\exp(\\beta^{-1}r^*(x,c))$, showing that optimizing Eq. 1 (and hence Eq. 5) directly approximates the concept posterior. This is what distinguishes CPO from BCE and from probabilistic CBM variants that amortize a latent variable's posterior instead.","core_discovery":"The paper's central claim is that the CPO objective, defined as $-E_{(x,c)\\sim\\mu,\\,c'\\sim\\pi_\\theta}[\\log\\sigma(\\log(\\pi_\\theta(c|x)/\\pi_0(c|x))-\\log(\\pi_\\theta(c'|x)/\\pi_0(c'|x)))]$, replaces the BCE concept loss in a CBM and directly optimizes the concept posterior rather than merely fitting noisy empirical labels. With a uniform prior, the loss is proportional to $-\\log \\pi_\\theta(c|x)$ whenever the policy-sampled concept $c'$ differs from the empirical concept $c$, and a constant otherwise, so CPO's expected gradient is BCE's gradient weighted by the model's confidence $(1-\\pi_\\theta(c|x))$. The paper proves that this expected gradient norm is bounded by BCE's and that, under a fixed level of label noise, CPO's gradient stays closer to the noise-free gradient than BCE's does. It also shows, through a control-as-inference optimality variable, that optimizing this objective is equivalent to amortizing the posterior $\\pi(c|o=1,x)$, which explains the improved uncertainty estimates and intervention behavior. The empirical section reports that CPO consistently improves task accuracy and concept AUC over BCE on CUB, AwA2, and CelebA with and without added noise, including at 40% concept-flip rates.","pith_inferences":["The confidence-weighted gradient suggests CPO should also dampen the effect of augmentation-induced concept corruption; a head-to-head comparison of no-augmentation versus random-crop training would isolate this mechanism from label-flip noise.","Because the proof's robustness guarantee omits a parameter-dependent term, CPO's advantage may shrink precisely when the model is least confident; annealing the prior strength $\\beta$ during training is a natural testable fix.","The same preference-over-labels construction could extend to structured annotator disagreement in medical or clinical concept labeling, where noise is correlated rather than random flips.","CPO's posterior-amortization view suggests it could replace the likelihood term in probabilistic CBM variants, yielding calibrated uncertainty without needing a latent variable bottleneck."],"forward_implications":["On clean data, CPO improves CBM task accuracy on CUB from 0.753 to 0.800 and concept AUC from 0.937 to 0.952, with no added parameters and roughly 0.05 extra minutes per epoch.","Under random concept-label flips at rates up to 40%, CPO-trained CBMs are the least affected models on CUB, AwA2, and CelebA, and at 40% noise they can still outperform more complex BCE-trained baselines.","CPO-trained models retain their intervenability under noise, with interventions ordered by CPO's uncertainty estimates improving task accuracy faster than BCE or ProbCBM interventions.","CPO with a non-uniform prior can update a CBM from streaming concept labels without drifting from the joint-trained policy, outperforming BCE-based streaming updates.","CPO's uncertainty estimates respond to occlusion: uncertainty rises when the target object is blocked, whereas BCE-trained models tend to stay overconfident."],"supporting_citations":[{"why":"Supplies the Direct Preference Optimization objective that CPO adapts to an online concept-training form.","marker":"[Rafailov et al., 2023]"},{"why":"Defines the Concept Bottleneck Model architecture and the BCE training paradigm that CPO replaces.","marker":"[Koh et al., 2020]"},{"why":"Control-as-inference tutorial whose derivations are used to equate CPO with amortized posterior approximation.","marker":"[Levine, 2018]"},{"why":"Shows the maximum-entropy RL objective equals amortized posterior inference, the bridge used in Section 4.3.","marker":"[Haarnoja et al., 2017]"},{"why":"Probabilistic CBM baseline that amortizes a latent variable's posterior, contrasted with CPO's direct concept posterior.","marker":"[Kim et al., 2023]"},{"why":"Provides the Concept Embedding Model architecture and the CelebA concept-set setup used in experiments.","marker":"[Espinosa Zarlenga et al., 2022]"},{"why":"CUB bird dataset with concept annotations on which most main results are reported.","marker":"[Wah et al., 2011]"},{"why":"AwA2 animal-attribute dataset used to validate CPO on a different domain.","marker":"[Xian et al., 2019]"},{"why":"CelebA face-attribute dataset used for the incomplete-concept evaluation.","marker":"[Liu et al., 2015]"}],"fun_headline_variants":["CPO loss slashes concept mislabeling impact","Preference loss beats binary entropy for concept models","Direct posterior optimization outperforms cross-entropy in CBMs","CPO keeps concept models accurate despite noisy labels","Preference loss makes concept models noise-proof"],"cache_read_input_tokens":33408,"weakest_assumption_plain":"The proof that CPO is always less sensitive to noise ignores part of the update from cases where the model's random choice matches the label; when the model is uncertain, the ignored part can make CPO's update larger than BCE's, so the guaranteed robustness bound does not always hold.","fun_headline_variants_meta":{"raw":{"variants":["CPO loss slashes concept mislabeling impact","Preference loss beats binary entropy for concept models","Direct posterior optimization outperforms cross-entropy in CBMs","CPO keeps concept models accurate despite noisy labels","Preference loss makes concept models noise-proof"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000771,"raw_usage":{"total_tokens":3441,"prompt_tokens":1002,"completion_tokens":2439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":2366}},"tokens_in":618,"tokens_out":2439,"duration_ms":16556,"temperature":1.0,"reasoning_tokens":2366,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:27:16.771517+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a logistic concept predictor on a single binary concept with noisy labels and compute the exact expected update of the CPO loss, including the case where the sampled concept equals the empirical one. At model confidence 0.1, if this update has larger norm than the BCE update, the bound in Proposition 4.2 is violated.","supporting_citations":[{"cited_title":"Caltech ucsd birds","cited_arxiv_id":null,"evidence_quote":"CUB bird dataset with concept annotations on which most main results are reported."}],"review_version":1}