{"id":"80982302-68c0-496f-bd2f-bf7a9ad0f6a3","arxiv_id":"2505.23968","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Model owners can covertly suppress confidence in targeted groups via Mirage, and Confidential Guardian audits calibration with zero-knowledge proofs to catch such tampering.","lead":"A new attack called Mirage lets a model owner quietly lower a model's confidence on chosen inputs, so the model abstains more often for certain people. The paper's defense, Confidential Guardian, uses zero-knowledge proofs to audit whether confidence scores are honestly calibrated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The ZKP proves calibration of whichever model the provider commits, but nothing binds production queries to that committed model; a dishonest provider can pass an audit with a clean model and serve a Mirage model, so the claimed guarantee over 'the deployed model' is unproven.","rationale":"The reader already identified the missing binding of production inference to the audited model, so my concern agrees with theirs. I keep CONDITIONAL because the Mirage attack and the ZKP-calibration protocol are real, empirically supported contributions, but the headline 'prevents abuse' claim is not established without either an explicit threat-model restriction (the provider is trusted to serve the committed model) or an additional cryptographic/attestation binding layer. The revision should state this assumption and either add binding or soften the claim from 'prevents' to 'certifies a given model under an honest-serving assumption.'","tokens_in":26737,"tokens_out":7242,"duration_ms":86493,"concrete_test":"Implement the two-model scenario: (1) run Algorithm 1 with a clean, calibrated model M_clean on Dref covering the attack region, and verify that the proof passes; (2) deploy a Mirage-trained model M_attack on the same task with the same uncertainty region; (3) query the deployed API and confirm that reported confidence and abstentions come from M_attack while the auditor's verification of M_clean succeeds. If this succeeds, the central prevention claim fails unless a binding mechanism is added, e.g., TEE attestation or per-query ZKP over the committed parameters; implementing one of those and rerunning the scenario would show whether the guarantee can be restored.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Algorithm 1 (§5.2) begins with 'P commits to a model M' and proves bin-wise ECE < α over Dref. Soundness of the ZKP implies only that the committed M passes; it does not imply that M is the model answering real queries. The paper's threat model is a dishonest institution that controls the service, and the abstract claims reported confidence scores 'genuinely originate from the deployed model,' yet no mechanism—per-query proof, attestation, commitment binding, signed transcript—connects the audited model to deployment. A provider can submit a clean, well-calibrated model to the audit, obtain a passing proof, and then deploy a Mirage-trained model (or any other discriminative abstention rule) for actual users. Section 6 experiments never test this mismatch: the model audited is the same model attacked. The reference-coverage limitation is acknowledged in the Limitations, but the binding problem is not mentioned, and it is equally necessary for the headline result. Fixable with trusted-execution attestation or per-inference ZKP over the same committed parameters, but without it Confidential Guardian is an audit of an arbitrary committed model, not a prohibition on abuse.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies a new threat in which a dishonest model provider deliberately suppresses a model's confidence scores on a chosen input region, thereby covertly triggering abstention for targeted individuals while keeping overall accuracy intact. It introduces Mirage, a fine-tuning objective that combines cross-entropy outside the target region with a KL-divergence regularizer inside it, and reports experiments on Gaussian, CIFAR-100, UTKFace, Adult, and Credit showing that accuracy is preserved while calibration error increases. As a defense, the paper proposes Confidential Guardian, a zero-knowledge proof protocol that computes bin-wise expected calibration error (ECE) on a reference dataset so that an auditor can verify that a committed model is calibrated within a public threshold, without revealing model parameters. The paper also proves a constructive theoretical result (Lemma 4.1) showing that feed-forward networks can be augmented to add arbitrary logit shifts in a selected bounded region, and it reports ZKP runtime and communication benchmarks for the audited models.","tokens_in":26990,"tokens_out":4138,"duration_ms":46892,"significance":"If the central claims hold, this is a valuable contribution: Mirage is a simple and apparently effective demonstration that abstention mechanisms can be abused by the model owner, a threat model that is under-studied relative to external poisoning adversaries. Confidential Guardian is a sensible use of zero-knowledge proofs of inference, and the paper is commendable for inheriting security from published UC-secure building blocks, for making the code available, and for being explicit about several limitations. The empirical attack results in Table 1 and Figures 3--5 are convincing as evidence that targeted confidence suppression is feasible without accuracy loss. However, the paper's headline claim that the framework 'prohibits' abuse and that reported confidence scores 'genuinely originate from the deployed model' is not supported by the protocol as written, and the detection evaluation does not provide the operating characteristics needed to substantiate the claim that Confidential Guardian reliably detects Mirage.","major_comments":[{"comment":"The paper claims that Confidential Guardian ensures reported confidence scores 'genuinely originate from the deployed model,' but the protocol only audits the model that the prover commits to. Algorithm 1 begins with the prover committing to a model M and then proves calibration of that committed model over Dref; soundness of the ZKP does not bind M to the model answering production queries. Since the threat model is a dishonest institution that controls the service, a provider can pass the audit with a clean, well-calibrated committed model and then serve a Mirage-trained model (or any other discriminatory abstention rule) in production. Section 6 never tests this mismatch: the model audited is the same model attacked. This is a load-bearing gap: either the protocol must be extended with a mechanism that binds deployed inference to the committed parameters (e.g., per-query ZKPs over the same commitment, trusted-execution attestation, or signed transcripts), or the paper must substantially weaken its claim from 'prohibiting abuse' to 'auditing an arbitrary committed model.'","section":"§5.2, Algorithm 1; Abstract"},{"comment":"The evaluation of Confidential Guardian as a detector is incomplete. Table 1 reports ECE and calibration error in the targeted bin, but the paper never specifies the threshold α used, nor reports detection rates, false-positive rates, ROC/AUC, or any other operating-characteristic metric. Figure 7 is described as showing that reference coverage affects 'detection performance,' but the figure only shows reliability diagrams and no quantitative detection result. Without a concrete decision rule and its sensitivity/specificity trade-off, the claim that Confidential Guardian 'effectively prevents' and 'reliably detects' Mirage is not substantiated. The authors should fix a threshold (or a family of thresholds), report detection and false-positive rates on the same reference distribution used by the audit, and quantify detection performance under the coverage degradation simulated in Appendix D.2.","section":"§6, Table 1; Figure 7"},{"comment":"The paper explicitly acknowledges in Section 5.1 and in the Limitations that miscalibration can arise from non-adversarial sources and that the method cannot attribute detected calibration failures to Mirage. This is appropriate caveating, but it conflicts with the strong language used in the abstract, the title, and Section 7, where the framework is said to 'prevent' confidence manipulation and to 'ensure' that abstentions are based on genuine uncertainty. The conclusion should be restated to match the acknowledged scope: Confidential Guardian detects a class of calibration anomalies under specified coverage and calibration assumptions; it does not by itself establish that abstention is legitimate or that manipulation has been prohibited.","section":"§5.1; Limitations"}],"minor_comments":[{"comment":"The ZKP runtime for CIFAR-100 is reported as '<333' without units; this is presumably seconds, but the entry should state the unit explicitly and, ideally, the exact measured value or a confidence interval.","section":"Table 1"},{"comment":"The notation is confusing: Equation (6) uses M to index bins, while Algorithm 1 and the surrounding text use M to denote the model; the paper should use distinct symbols for the number of bins and the model.","section":"§5.2, Algorithm 1"},{"comment":"The bullet 'Efficiency of Confidential Guardian in proving the ZK EEC constraint' appears to contain a typo: 'EEC' should be 'ECE.'","section":"§6, bullets"},{"comment":"Lemma 4.1 states that logits can be shifted by an arbitrary non-negative constant vector c in the selected region; the proof is constructive, but the relationship between this general logit shift and the specific confidence-reduction behavior needed for Mirage could be clarified, since adding an identical constant to all logits leaves softmax probabilities unchanged and class-dependent shifts may alter accuracy.","section":"§4.1, Lemma 4.1"},{"comment":"The Limitations paragraph correctly identifies reference-set coverage as a necessary condition for detection; it would be helpful to state explicitly that the ZKP guarantees nothing about regions not covered by Dref, since Algorithm 1 only computes statistics over the provided reference points.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The deployment-binding issue is the main correctness concern: the protocol's soundness does not connect the audited model to the served model, so the paper's strongest claim is currently unproven. The detection evaluation also needs a concrete decision rule and error rates. Both are fixable in revision, either by adding a binding mechanism or by substantially narrowing the claims, so I do not recommend rejection. The paper would also benefit from one of the authors (or an independent reader) double-checking that the Lemma 4.1 construction's 'arbitrary non-negative constant vector' is compatible with the confidence-reduction claim in Section 4.1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first paper I've seen that treats the model owner as the adversary against abstention mechanisms. The Mirage attack is credible—label-smoothing-style KL target keeps accuracy while suppressing confidence in a chosen region—and the idea of auditing calibration inside a ZKP is a natural but non-obvious integration. The empirical attack results across six datasets are enough to believe Mirage works as described. The ZKP cost numbers for tabular and synthetic models are real measurements, and the image-scale numbers are honestly labeled estimates.\n\nThe weakest spot is the one the stress-test note flags: Algorithm 1 proves the bin-wise ECE of whatever model the prover commits, but nothing in the protocol ties that committed model to the model answering production queries. The abstract claims confidence scores 'genuinely originate from the deployed model,' and the threat model is a dishonest institution that controls the service. As written, the provider can pass an audit with a clean, calibrated model and serve a Mirage-trained model (or any other discriminatory abstention rule) to actual users. Section 6 never tests that mismatch; the audited model is the same model attacked. The reference-coverage limitation is stated in the Limitations section, and the paper's own experiments (Figure 7) show detection degrades as coverage shrinks. The binding problem is not mentioned there, and it is equally load-bearing for the headline guarantee. Fixable, with attestation, per-inference ZKP over the same committed parameters, or signed inference transcripts, but without it Confidential Guardian is an audit of an arbitrary committed model, not a prohibition.\n\nOther soft spots are minor by comparison. There is no concrete alpha, ROC, or false-positive evaluation for detection—just raw calibration metrics that show Mirage inflates ECE. That is sufficient to establish feasibility but not to support 'prevent' language. The theory in Lemma 4.1 looks sound; the proof is constructive and fine.\n\nWho is this for: people working on trustworthy ML, selective classification, and verifiable ML. It deserves a serious referee. I would send it to review, but I would expect the authors to either add a deployment-binding mechanism or substantially soften the 'prevents misuse' claim, and to add a threshold-based detection experiment.","headline":"Clear new threat model and a solid Mirage attack, but the audit protocol only certifies a committed model, not the deployed one, so the 'prevents abuse' claim needs either a binding mechanism or a softer statement.","tokens_in":27504,"tokens_out":2581,"would_cite":true,"duration_ms":24416,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that a model owner can covertly suppress confidence for chosen inputs and that an auditor can catch this by verifying calibration inside a zero-knowledge proof.","keywords":["Mirage attack","abstention abuse","model calibration","expected calibration error","zero-knowledge proof","verified inference","selective classification","discrimination via uncertainty"],"falsifier":"Train a model, run Mirage on a demographic region, then audit it with a reference dataset that contains no points from that region: if the zero-knowledge audit returns 'well-calibrated' while the deployed model abstains on that group at a markedly higher rate than on others, then the paper's claim that miscalibration reliably reveals artificial uncertainty is refuted.","tokens_in":26560,"feed_emoji":"🛡️","tokens_out":7563,"duration_ms":66164,"temperature":0.7,"pith_summary":"The paper sets out to show that the abstention mechanisms designed to make machine learning safer can be turned into a covert instrument of discrimination: a model owner can retrain the model so that it reports low confidence on a chosen group of inputs, triggering abstention for exactly the people the owner wants to disadvantage, while overall accuracy stays high. To make this concrete, it introduces Mirage, a training objective that pushes confidence in a targeted region toward a correct-class-biased uniform distribution but never flips labels. The paper then argues that this attack is detectable in principle because artificially suppressed confidence shows up as miscalibration: in the targeted region, the model's stated confidence falls below its actual accuracy. The proposed defense, Confidential Guardian, is a zero-knowledge proof protocol in which the provider proves that the expected calibration error on an auditor-supplied reference dataset is below a public threshold, without revealing the model's parameters. If the proof passes, an auditor can treat reported abstentions as reflecting genuine uncertainty; if the calibration violation is detected, the model is flagged for scrutiny.","feed_headline":"Zero-knowledge check exposes biased AI abstention","feed_subtitle":"Mirage hides discrimination as uncertainty; Confidential Guardian proves the miscalibration without revealing the model.","key_machinery":"The load-bearing object is the zero-knowledge proof of well-calibratedness: a protocol in which the provider commits to the model weights, the auditor supplies a reference dataset, and both parties jointly evaluate the expected calibration error (ECE) inside a cryptographic proof, revealing only a binary pass/fail outcome. The proof first runs a zero-knowledge proof of correct inference for every reference point, so the confidence scores used in the audit are guaranteed to come from forward passes of the committed model; it then uses zero-knowledge arrays to bin points by confidence and to aggregate counts, confidence, and accuracy per bin; finally it proves that every bin satisfies |acc - conf| <= alpha times count for the public threshold alpha. On the attack side, the counterpart machinery is Mirage's KL-divergence loss toward the target distribution t_epsilon, which equals epsilon + (1 - epsilon)/C for the true label and (1 - epsilon)/C for every other label, making the model underconfident in the chosen region while preserving the argmax.","core_discovery":"The central claim is that artificially induced uncertainty is both easy to create and detectable in a confidentiality-preserving way, and that these two facts together close the loophole. On the attack side, the paper proves a constructive lemma: for any feed-forward network and any axis-aligned region, one can augment the network with analytically designed neurons so that logits inside the region are shifted by an arbitrary non-negative constant while all other outputs are unchanged, which lowers confidence without changing labels. Mirage instantiates this in trainable form by adding a KL-divergence term that pulls the output distribution toward a target that is almost uniform but slightly biased toward the true label. On the defense side, the paper's claim is that a zero-knowledge proof of well-calibratedness, where the committed model is run on the auditor's reference dataset and bin-wise expected calibration error is computed inside the proof, certifies that confidence scores genuinely come from the deployed model and that no calibration deviation beyond a public threshold alpha exists. The experiments show Mirage keeping accuracy essentially flat while raising ECE, and the proof protocol detecting the deviation at practical cost for small models.","pith_inferences":["A natural extension the paper leaves implicit is to make the proof output the location of the worst calibration bin, not just a pass/fail bit, so auditors can direct follow-up data collection toward the affected input region.","The confidentiality guarantee is only as strong as the link between the audited model and the served model; nothing in the protocol stops a provider from proving calibration on one checkpoint and routing production traffic through another, so binding deployment to the committed model is a second, unstated requirement.","The same calibration-in-a-proof machinery could catch opposite manipulations, such as backdoors that raise confidence on trigger inputs, because any systematic distortion of confidence leaves a footprint in the reliability diagram.","Because the proof only checks an aggregate threshold, an attacker who spreads suppressed confidence thinly across many bins might hide under alpha; testing whether Mirage can be adapted to stay within every bin's budget would sharpen the threat model."],"forward_implications":["A lender, employer, or insurer can deny service to a chosen demographic by lowering model confidence for that group, and accuracy-based fairness audits will not flag it because predicted labels stay correct.","An auditor who holds a reference dataset covering the suspected region can verify whether abstentions are justified without ever seeing the model's weights or training data, protecting the provider's intellectual property.","Audit coverage, not model accuracy, becomes the decisive regulatory resource: the protocol's detection power falls as the reference dataset undersamples the targeted region, so standards for what counts as an adequate reference set are required.","The attack and defense share a single dial: lower epsilon gives the attacker stronger confidence suppression but a larger calibration gap, so the same metric that empowers Mirage also exposes it.","The audit is practical for small and tabular models but currently too expensive for large vision models, so deployment of the defense in high-stakes settings depends on faster zero-knowledge inference."],"supporting_citations":[{"why":"Defines the abstention/selective-prediction framing that the paper shows can be abused by model owners.","marker":"El-Yaniv et al., 2010"},{"why":"Supplies the accuracy-based fairness auditing baseline that Mirage is designed to evade.","marker":"Hardt et al., 2016"},{"why":"Provides temperature scaling, used to calibrate the baseline model so that later miscalibration can be attributed to Mirage.","marker":"Guo et al., 2017"},{"why":"Supplies the expected calibration error metric that Confidential Guardian computes inside the zero-knowledge proof.","marker":"Naeini et al., 2015"},{"why":"Wolverine, the boolean/arithmetic circuit zero-knowledge proof building block whose security the protocol inherits.","marker":"Weng et al., 2021a"},{"why":"Mystique, a zero-knowledge proof of correct inference used to verify forward passes on the committed model.","marker":"Weng et al., 2021b"},{"why":"ZKLLM, another zero-knowledge proof of correct inference that can serve as the Finf subroutine.","marker":"Sun et al., 2024"},{"why":"Provides the zero-knowledge array machinery used to track bin membership, counts, confidence, and accuracy.","marker":"Franzese et al., 2021"},{"why":"Introduces label smoothing, the non-adversarial construction that Mirage's target distribution resembles.","marker":"Szegedy et al., 2016"},{"why":"Shows external availability attacks on abstention, the complementary threat model that motivates Mirage as an insider attack.","marker":"Lorenz et al., 2023"}],"fun_headline_variants":["Zero-knowledge proof detects fake AI uncertainty","Mirage attack and proof-based defense for model confidence","Verifiable abstinence: guarding against biased AI uncertainty","Cryptographic guard exposes artificially suppressed AI confidence","Proof-based audit catches dishonest model abstention"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole defense rests on the reference dataset actually covering the targeted region and on the model that is audited being the model that is served; if the attacker can pick a region the auditor's data never reaches, or can pass the audit with one model and run another in production, artificial uncertainty goes undetected.","fun_headline_variants_meta":{"raw":{"variants":["Zero-knowledge proof detects fake AI uncertainty","Mirage attack and proof-based defense for model confidence","Verifiable abstinence: guarding against biased AI uncertainty","Cryptographic guard exposes artificially suppressed AI confidence","Proof-based audit catches dishonest model abstention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1350,"prompt_tokens":945,"completion_tokens":405,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":335}},"tokens_in":561,"tokens_out":405,"duration_ms":4004,"temperature":1.0,"reasoning_tokens":335,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:37:59.983502+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a model, run Mirage on a demographic region, then audit it with a reference dataset that contains no points from that region: if the zero-knowledge audit returns 'well-calibrated' while the deployed model abstains on that group at a markedly higher rate than on others, then the paper's claim that miscalibration reliably reveals artificial uncertainty is refuted.","supporting_citations":[],"review_version":1}