{"id":"a8643f61-bc54-4510-8f23-2fcba4e505fa","arxiv_id":"2412.05767","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Adding a loss-variance penalty to adversarial training reduces membership inference leakage on CIFAR-10/100 while keeping accuracy mostly intact.","lead":"This paper proposes a training tweak called DeMem that aims to stop machine learning models from memorizing sensitive training examples, while keeping them robust against adversarial attacks. The authors report lower privacy-leakage rates on image benchmarks with only small accuracy costs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The loss-variance penalty likely reduces LiRA TPR by compressing the loss distribution rather than by de-memorizing high-risk samples; if so, DeMem's central privacy claim is an artifact of the attack metric.","rationale":"The central claim is that DeMem reduces privacy leakage while preserving robustness. The reported evidence for privacy is entirely LiRA TPR at low FPR (§4.2, Table 3). Because low-FPR LiRA thresholds the upper tail of the per-example loss distribution, a regularizer whose only demonstrated effect is to compress that distribution can lower the metric without changing what the model actually remembers. The gradient computation in §3.3 makes this concrete: the penalty amplifies updates on high-loss examples (coefficient 1 + 2λ(ℓ_i − μ) > 1), which is the opposite of 'selectively targeting high-risk samples' by restricting their influence. The paper provides no direct memorization-score comparison on robust models: Table 1 is a baseline CIFAR-100 correlation, not a before/after DeMem measurement on PGD-AT or TRADES. A rank-invariant attack or Eq. (1) measurement would distinguish real dememorization from loss-tail suppression. If the check fails, the paper's main contribution—privacy without robustness loss—collapses; hence the recommended verdict is REJECT rather than the reader's CONDITIONAL, unless the authors supply the missing evidence.","tokens_in":8132,"tokens_out":5646,"duration_ms":59525,"concrete_test":"Retrain PGD-AT and PGD-AT+DeMem on CIFAR-10 with identical budgets and evaluate both with (a) a LiRA variant that uses per-class rank-transformed losses or logits instead of raw loss, and (b) leave-one-out or influence-estimation memorization scores (Eq. 1) on a random subset of 1,000 training points. If rank/logit-based TPR at 0.1% FPR is comparable between the two models, or if memorization scores do not decrease under DeMem, then the reported privacy gain is an artifact of loss-variance compression rather than de-memorization.","verdict_should_be":"REJECT","load_bearing_attack":"Section 3.3 adds Ψ(B) = (1/N) Σ_i (ℓ_i − μ)^2 to the training loss. Its gradient contribution for sample i is 2λ(ℓ_i − μ)∇ℓ_i, so high-loss samples receive larger gradients and are fitted more strongly, not 'restricted' as claimed. The privacy metric is the load-bearing problem: LiRA scores members by per-example loss, and TPR at 0.1% FPR thresholds the extreme high-loss tail of the non-member distribution. Minimizing loss variance directly compresses that tail, so fewer members exceed the fixed-FPR threshold. Table 3's improvements may therefore be a mechanical consequence of loss homogenization. The only evidence linking loss to memorization is Table 1, computed on a non-robust CIFAR-100 baseline; no measurement shows that DeMem reduces Eq. (1) memorization under PGD-AT/TRADES. The paper's central claim requires showing reduced actual memorization, not merely reduced loss-tail separation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DeMem, a regularizer that adds a minibatch loss-variance penalty to adversarial training losses (PGD-AT, TRADES) with the stated goal of selectively de-memorizing high-risk samples and thereby reducing membership privacy leakage without significantly harming natural or robust accuracy. The authors justify the penalty by a measured Spearman correlation between per-sample loss and memorization scores on CIFAR-100, and evaluate privacy leakage with LiRA TPR at low FPR thresholds. Experiments on CIFAR-10 and CIFAR-100 show that adding DeMem reduces LiRA TPR while producing only small drops in robust accuracy, and the paper claims this achieves a better privacy-robustness balance than differential privacy.","tokens_in":8353,"tokens_out":4777,"duration_ms":45782,"significance":"If the central claim were established, DeMem would be a simple and scalable heuristic for reducing membership leakage in robust models, and the evaluation protocol (LiRA TPR at 0.1% and 0.001% FPR, 128 models per setting, mean over 10 models) is a rigorous and appropriate way to quantify privacy risk. The paper also raises an interesting question about why DP degrades robust models by examining sample-level memorization. However, the significance is currently undercut by a mismatch between the proposed mechanism and its actual gradient behavior, and by the absence of a direct comparison with DP on the privacy metric. The empirical phenomenon may be real, but the paper's interpretation needs substantial revision before the claims can be accepted.","major_comments":[{"comment":"The proposed penalty Ψ(B) = (1/N) Σ_i (ℓ_i − μ)^2 is described as selectively restricting high-risk samples, but its gradient contribution for sample i is 2λ(ℓ_i − μ)∇ℓ_i. Samples with loss above the batch mean receive amplified gradients and are therefore fitted more strongly, while samples below the mean are fitted more weakly. This is the opposite of the stated mechanism. Please either correct the mechanism description or provide direct evidence (e.g., memorization scores from Eq. (1) before and after DeMem under PGD-AT/TRADES) that the penalty reduces memorization of high-loss samples.","section":"§3.3 (definition of Ψ(B))"},{"comment":"The abstract and §3.1 claim DeMem achieves a better privacy-robustness balance than differential privacy, but Table 3 reports TPR only for PGD-AT/TRADES with and without DeMem. The DP-SGD baseline ('Base' in Table 2) is never evaluated with LiRA, so the central comparison against DP is not demonstrated. Please add TPR at the same FPR thresholds for the DP-SGD baseline on both datasets.","section":"Table 3 / §4.2"},{"comment":"The only evidence that loss is a reliable proxy for memorization comes from Spearman correlations (0.501 on training, 0.784 on test) computed on a non-robust CIFAR-100 baseline using memorization scores from Feldman and Zhang. DeMem is applied to PGD-AT and TRADES models, and adversarial training substantially changes the loss distribution. Without measuring the loss-memorization correlation under adversarial training, the proxy assumption is unsupported in the target setting.","section":"§3.2, Table 1"},{"comment":"LiRA, the primary privacy metric, scores members and non-members by per-example loss, and TPR at 0.1% FPR thresholds the extreme high-loss tail of the non-member distribution. Since the variance penalty directly compresses the training loss distribution, the observed TPR drop may be a mechanical consequence of loss homogenization rather than a reduction in actual memorization. Please report a direct memorization measure (e.g., Eq. (1) estimated via influence functions or leave-one-out on a subsample) or an attack metric that is not purely loss-threshold based.","section":"§2.4 and §3.3"}],"minor_comments":[{"comment":"The phrase 'without compromising both nature and robust accuracy' should read 'natural and robust accuracy'; the same typo appears in the abstract's parenthetical example.","section":"Introduction"},{"comment":"The text says \"'Base' refers to PGD-AT with DP-SGD configured identically to Table 2,\" but Table 2 does not list a DP-SGD configuration. Please clarify the DP-SGD hyperparameters (noise multiplier, clipping norm) used for the Base rows and state whether the same Base is used for both PGD-AT and TRADES comparisons.","section":"§4.2, Table 2"},{"comment":"The abstract claims evaluation across 'multiple training methods and datasets' and later mentions 'model structures,' but the experiments use only ResNet-50 with PGD-AT and TRADES. Please either add experiments with another architecture or soften the claim.","section":"Abstract"},{"comment":"Figures 2 and 3 are referenced in the text but their construction is not fully described; please add explicit axis labels, the evaluation protocol (e.g., which attack models are used), and the range of λ and ε values shown.","section":"Figures 2 and 3"},{"comment":"Several references have incomplete bibliographic details (e.g., [20] and [25] lack proper venue/pagination), which should be corrected for publication.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core empirical observation may be salvageable if the authors reframe DeMem as a loss-variance regularizer and demonstrate that it reduces membership leakage through a mechanism other than simply compressing the loss tail. The missing DP baseline in Table 3 is a straightforward but essential addition. I recommend major revision rather than rejection because the experimental protocol is otherwise careful and the question is important."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this paper adds a loss-variance penalty to adversarial training and shows it reduces LiRA membership attack TPR at low FPR on CIFAR-10/100. The effect is real enough to take seriously, but the paper's framing as selective de-memorization of high-risk samples does not match what the penalty actually does. The central comparison against DP-SGD's privacy level is also missing, so the headline claim of a better privacy-robustness balance is not demonstrated.\n\nWhat is genuinely useful: the diagnostic that DP disproportionately harms low-memorization samples is a nice observation, using precomputed memorization scores and per-bin LiRA TPR. It gives a concrete reason why naive DP + adversarial training fails. And a one-line regularizer that cuts LiRA TPR at 0.1% FPR from 20.29% to 12.39% on CIFAR-10, with only a 0.93% robustness drop, is practically interesting. The experimental setup (128 models, LiRA, multiple methods and two datasets) is solid, though narrow.\n\nThe soft spots are real. First, Table 3 never reports privacy leakage for the DP-SGD baseline, so the abstract's claim of a better balance than DP is unsubstantiated. You cannot conclude DeMem beats DP on the privacy-robustness frontier without DP's TPR. Second, the mechanism is misdescribed. The penalty Ψ(B) = (1/N)Σ(ℓ_i − μ)^2 does not 'selectively target' high-risk samples; it compresses all losses toward the batch mean. The gradient for a high-loss sample is 2λ(ℓ_i − μ)∇ℓ_i, which is larger in magnitude, so it actually fits those samples harder, not 'restricts' them. Third, the only loss-memorization correlation (Table 1) comes from a non-robust CIFAR-100 baseline; there is no evidence that DeMem reduces the Eq. (1) memorization score under PGD-AT or TRADES. What it likely does is homogenize loss values, which directly shrinks the tail that LiRA's low-FPR thresholds operate on. That may be a legitimate privacy defense, but it is not the de-memorization story told here. Fourth, robustness drops are downplayed: TRADES + DeMem on CIFAR-100 drops robust accuracy from 18.93% to 16.77%, which is about two points, not negligible. Finally, there are inconsistencies between text and tables (e.g., the claimed DP natural accuracy of 20.35% vs. 31.27% in Table 2), and no code is provided.\n\nThese are addressable. The paper deserves a serious referee, but the revision needs: DP-SGD privacy numbers, a direct or indirect measurement of memorization under DeMem (even on a subset), a corrected mechanism description, and honest treatment of robustness costs.\n\nFor whom? Researchers working on privacy-robustness trade-offs may find the regularizer worth testing. I would not cite it in its current form, but it is worth a careful read.","headline":"DeMem is a simple loss-variance penalty that empirically lowers LiRA TPR in adversarial training, but the paper overstates the mechanism and omits the key DP-SGD privacy comparison.","tokens_in":8835,"tokens_out":3138,"would_cite":false,"duration_ms":32661,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68P27"],"pacs":[],"model":"deepseek-v4-flash","headline":"A minibatch loss-variance penalty reduces membership-inference leakage in adversarially trained models.","keywords":["adversarial robustness","membership inference attack","memorization","differential privacy","privacy-robustness trade-off","loss variance regularization","LiRA","DeMem"],"falsifier":"Compute leave-one-out memorization scores on a PGD-AT or TRADES model for CIFAR-10 and correlate them with per-sample losses; if the Spearman correlation is near zero or negative, the loss proxy DeMem relies on is absent. Alternatively, measure per-sample memorization before and after DeMem: if high-loss samples retain their memorization while aggregate privacy improves, the proposed mechanism is not what is doing the work.","tokens_in":7935,"feed_emoji":"🔒","tokens_out":7728,"duration_ms":64208,"temperature":0.7,"pith_summary":"Adversarially trained models are known to leak membership information, and the standard privacy remedy, differential privacy, destroys the robustness that adversarial training is meant to provide. This paper argues that the damage is uneven: DP disproportionately hurts low-memorization, low-risk samples that carry most of the model's accuracy, while high-memorization samples drive privacy leakage. To target only the risky samples, the authors propose DeMem, a regularizer that penalizes the variance of per-sample losses within each minibatch, pushing high-loss samples toward the batch mean. On CIFAR-10 with PGD-AT, DeMem lowers the LiRA membership-inference attack's true-positive rate at 0.1% false-positive rate from 20.29% to 12.39% while robust accuracy drops from 35.96% to 35.03%, with similar gains on CIFAR-100 and with TRADES. The claim is that privacy in robust models can be improved substantially without the accuracy collapse caused by DP.","feed_headline":"Loss penalty cuts membership-inference leaks in robust models","feed_subtitle":"On CIFAR-10 PGD-AT, LiRA TPR at 0.1% FPR falls from 20.29% to 12.39% with under 1% robust-accuracy loss.","key_machinery":"The central object is the Sample-wise Dememorization Penalty, the variance of per-sample losses within a training minibatch: $\\Psi(B)=\\frac{1}{N}\\sum_{i=1}^{N}\\left(\\ell(x_i,\\theta)-\\frac{1}{N}\\sum_{j=1}^{N}\\ell(x_j,\\theta)\\right)^2$. It is added to the original training loss as $L_{\\mathrm{total}}(\\theta)=L(\\theta)+\\lambda\\Psi(B)$. The penalty shrinks the spread of losses, pulling high-loss (high-memorization) samples toward the batch mean so the model retains less information about them. Its rationale rests on the measured Spearman correlation between loss and memorization (0.501 train, 0.784 test on CIFAR-100), which lets loss stand in for the computationally expensive leave-one-out memorization score. The paper uses LiRA TPR at low FPR as the privacy metric and applies the penalty on top of PGD-AT and TRADES.","core_discovery":"The paper's central claim is that the privacy leakage introduced by adversarial training can be reduced by selectively de-memorizing high-risk samples, and that a simple variance penalty on minibatch losses accomplishes this. The authors first show, using precomputed memorization scores from prior work, that applying DP-SGD to PGD-AT cuts accuracy mostly among samples with low memorization scores, whereas samples with high memorization scores are the main source of membership leakage. They then note that loss correlates with memorization (Spearman 0.501 on training and 0.784 on test data for a baseline CIFAR-100 model) and define DeMem as adding $\\lambda$ times the variance of the losses in a minibatch to the training objective. With this regularizer added to PGD-AT and TRADES, membership-inference attack success at strict false-positive rates drops markedly on CIFAR-10 and CIFAR-100 while natural and robust accuracies remain nearly unchanged. The authors take this as evidence that the privacy-robustness tension in adversarial training can be eased without DP's global performance penalty.","pith_inferences":["One extension the paper leaves implicit: because the loss-memorization proxy is not specific to adversarial training, DeMem should also reduce membership leakage under standard training, which would be straightforward to test.","Another extension: since DeMem already suppresses memorization of high-risk samples, combining it with DP-SGD might allow smaller noise for a fixed privacy budget, softening DP's accuracy collapse.","A caveat the paper does not address: the loss-memorization correlation was measured only on a baseline CIFAR-100 model, so if it weakens under adversarial training, the mechanism would need re-validation even if aggregate privacy gains persist."],"forward_implications":["Adding DeMem to PGD-AT lowers LiRA TPR at 0.1% FPR from 20.29% to 12.39% on CIFAR-10 while robust accuracy drops from 35.96% to 35.03%.","On CIFAR-100, PGD-AT + DeMem reduces TPR at 0.1% FPR from 67.23% to 64.65%, and TRADES + DeMem from 52.60% to 47.80%, with robust accuracy losses under about two points.","Increasing the dememorization strength $\\lambda$ strengthens privacy protection while robustness declines only slightly, giving a tunable privacy-robustness knob.","The privacy gains hold across tested adversarial perturbation magnitudes, so DeMem is not tied to one attack configuration.","DeMem is a drop-in regularizer for adversarial training methods, so it can be appended to future robust training schemes without changing their core objectives."],"supporting_citations":[{"why":"supplies precomputed memorization scores for CIFAR-100 and establishes the long-tail memorization phenomenon used to frame the analysis.","marker":"[13]"},{"why":"LiRA is the membership-inference attack used to measure privacy leakage at low false-positive rates.","marker":"[21]"},{"why":"defines the memorization score that the paper uses to attribute privacy risk to individual samples.","marker":"[12]"},{"why":"PGD-AT is the primary adversarial training method DeMem is added to and compared against.","marker":"[14]"},{"why":"TRADES is the second adversarial training method used to demonstrate DeMem's generality.","marker":"[15]"},{"why":"DP-SGD is the differential-privacy baseline whose accuracy collapse motivates DeMem.","marker":"[11]"},{"why":"provides prior evidence that differential privacy degrades robust generalization, which the paper analyzes at sample level.","marker":"[10]"},{"why":"establishes that adversarial training increases vulnerability to membership inference, the problem DeMem targets.","marker":"[6]"}],"fun_headline_variants":["DeMem trims privacy leaks without hurting adversarial robustness","Variance penalty on losses cuts membership leaks in robust models","Targeting high-risk samples eases privacy-robustness tension","DeMem: selective de-memorization keeps robust models private","Loss variance regularizer slashes MIA success on CIFAR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a sample's loss reliably tracks how much the model memorizes that sample under adversarial training, so shrinking the spread of losses in a minibatch actually de-memorizes the high-risk samples; this correlation is measured only on a baseline CIFAR-100 model, not on the PGD-AT or TRADES models where DeMem is applied.","fun_headline_variants_meta":{"raw":{"variants":["DeMem trims privacy leaks without hurting adversarial robustness","Variance penalty on losses cuts membership leaks in robust models","Targeting high-risk samples eases privacy-robustness tension","DeMem: selective de-memorization keeps robust models private","Loss variance regularizer slashes MIA success on CIFAR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000586,"raw_usage":{"total_tokens":2753,"prompt_tokens":942,"completion_tokens":1811,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":1726}},"tokens_in":558,"tokens_out":1811,"duration_ms":11320,"temperature":1.0,"reasoning_tokens":1726,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:23:12.179398+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute leave-one-out memorization scores on a PGD-AT or TRADES model for CIFAR-10 and correlate them with per-sample losses; if the Spearman correlation is near zero or negative, the loss proxy DeMem relies on is absent. Alternatively, measure per-sample memorization before and after DeMem: if high-loss samples retain their memorization while aggregate privacy improves, the proposed mechanism is not what is doing the work.","supporting_citations":[{"cited_title":"What neural networks memorize and why: Discovering the long tail via influence es- timation,","cited_arxiv_id":null,"evidence_quote":"LiRA is the membership-inference attack used to measure privacy leakage at low false-positive rates."},{"cited_title":"Language-driven anchors for zero-shot adver- sarial robustness,","cited_arxiv_id":null,"evidence_quote":"defines the memorization score that the paper uses to attribute privacy risk to individual samples."},{"cited_title":"Privacy risks of securing machine learning models against adversarial exam- ples,","cited_arxiv_id":null,"evidence_quote":"PGD-AT is the primary adversarial training method DeMem is added to and compared against."},{"cited_title":"On the privacy effect of data enhancement via the lens of memoriza- tion,","cited_arxiv_id":null,"evidence_quote":"TRADES is the second adversarial training method used to demonstrate DeMem's generality."},{"cited_title":"On the privacy risks of algorithmic fairness,","cited_arxiv_id":null,"evidence_quote":"provides prior evidence that differential privacy degrades robust generalization, which the paper analyzes at sample level."},{"cited_title":"We then analyze individual samples to explain why DP can fail, followed by a detailed presentation of our proposed approach","cited_arxiv_id":null,"evidence_quote":"establishes that adversarial training increases vulnerability to membership inference, the problem DeMem targets."}],"review_version":1}