Pith. sign in

REVIEW 5 major objections 5 minor 13 references

Existing unlearnable examples fail to transfer across tasks: on Taskonomy they barely degrade scene classification, depth, keypoints, or segmentation, and a new loss-landscape metric, Unlearnable Distance, quantifies how close poisons come

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 21:21 UTC pith:MWUL5W2P

load-bearing objection Empirical multi-task failure is real; the SAL/UD metric is a promising heuristic but the causal claim and benchmark need substantial hardening. the 5 major comments →

arxiv 2509.08058 v1 pith:MWUL5W2P submitted 2025-09-09 cs.LG cs.AI

How Far Are We from True Unlearnability?

Kai Ye , Liangcai Su , Chenxiong Qian This is my paper
classification cs.LG cs.AI
keywords unlearnable examplesavailability attacksdata poisoningloss landscapesharpness-aware minimizationmulti-task learningmachine learning security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that today's unlearnable examples—datasets perturbed so that models trained on them cannot generalize—are not truly unlearnable once the training task is not fixed in advance. On the Taskonomy multi-task benchmark, poisons from EM, OPS, and AR left scene classification, keypoint, depth, and segmentation accuracy nearly unchanged, and cross-task transfer also resisted the perturbations. To explain why, the authors look at the training process rather than final accuracy: they argue poisons work by flattening the loss landscape around model parameters, lowering what they call Sharpness-Aware Learnability (SAL). From SAL they build Unlearnable Distance (UD), a thresholded ratio of 'learnable' parameters in poisoned versus clean models, and benchmark six poison methods across datasets, architectures, and defenses. The message is that unlearnability is task- and model-dependent, and the community should stop measuring it by test accuracy alone.

Core claim

The paper concludes that existing unlearnable examples are not truly unlearnable: on Taskonomy, EM, OPS, and AR poisons barely degrade scene classification, keypoints, depth, or segmentation, and EM fails under cross-task transfer. The proposed explanation focuses on training dynamics: poisoned models update parameters more slowly and along flatter regions of the loss landscape. Sharpness-Aware Learnability (SAL) measures the largest training-loss change under a small L2 perturbation of one layer; Unlearnable Distance (UD) is the ratio of learnable parameters in poisoned versus clean models, using a K-means threshold from clean training. Across six methods and three datasets UD ranges from 0

What carries the argument

Sharpness-Aware Learnability (SAL, Eq. 3) is the paper's core diagnostic: for one layer at a time, with other layers frozen, it computes the maximum change in training loss when that layer's weights are perturbed within an L2 ball of radius eps=0.05. Low SAL means the loss surface is flat, so parameter updates barely change the loss—the signature of an unlearnable parameter. Learnable Threshold (LT, Eq. 4) separates learnable from unlearnable parameters by K-means clustering SAL values during clean training, and Unlearnable Distance (UD, Eq. 5) is the ratio of the average number of learnable parameters in the poisoned model to that in the clean model. This ratio carries the argument: it turn

Load-bearing premise

The load-bearing premise is that a parameter is unlearnable exactly when the training loss is flat under a small fixed perturbation of that layer alone—flatness is treated as the cause of the accuracy drop, not just a correlated side effect; if that causal link fails, UD measures a proxy and the benchmark's answer to 'how far' collapses.

What would settle it

Retrain both clean and poisoned models from independent random seeds (not from the same initialization, as Algorithm 1 requires) and recompute UD; if a poison such as OPS keeps low test accuracy while its UD rises near 1, the metric is an artifact of the shared initialization rather than a measure of unlearnability.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If SAL and UD are right, unlearnability is not a property of the dataset alone; it depends on model architecture, initialization, task, and defense, so 'unlearnable' status from one benchmark cannot be assumed to hold elsewhere.
  • The benchmark places TAP outside the UE family: adversarial poisons keep parameters learnable and drive accuracy down by teaching wrong features, so defenses that raise UD for true UEs will not necessarily help against availability attacks.
  • Existing UEs should be evaluated in multi-task and cross-task settings; a poison that weakens classification may leave segmentation and depth intact, so data owners cannot rely on current methods to protect leaked data used for diverse training purposes.
  • Defenses such as JPEG compression, adversarial training, UEraser, MixUp, and CutOut increase UD (except for TAP), offering a way to screen whether a defense actually restores learnability rather than merely changing accuracy.
  • Stronger models such as ViT have higher UD, implying that as model capacity grows, current poisons become less effective at blocking learning; developing UEs for large models requires new mechanisms, not stronger versions of existing perturbations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to apply UD to other modalities, such as text or audio, by measuring flatness of parameter updates on poisoned data; if SAL correlates with model scale, the epsilon and threshold settings would likely need recalibration.
  • Algorithm 1 shares the same initialization between clean and poisoned models, which is not how models are normally retrained; an honest evaluation would draw clean and poisoned models from independent random seeds and compare UD distributions, which might shrink or widen the reported gaps.
  • If flatness is causal, then perturbations optimized to minimize SAL directly—rather than minimizing loss as EM does—should produce stronger cross-task UEs; the paper's explanation implies this construction but does not test it.
  • The TAP outlier suggests a two-axis view of unlearnability: 'flatness' (SAL) and 'wrong-feature learning' are distinct failure modes, and a complete metric might need to combine both rather than classifying methods into a binary UE-versus-attack dichotomy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies whether existing unlearnable examples (UEs) achieve multi-task unlearnability, reports that they do not, and attempts to explain this through the loss landscape. It proposes Sharpness-Aware Learnability (SAL, Eq. 3), defined as the maximal training-loss increase under a small ℓ_p perturbation of one layer, and Unlearnable Distance (UD, Eq. 5), a ratio of the number of parameters whose SAL exceeds a K-means-derived threshold on the clean model. The authors use SAL/UD to benchmark EM, REM, DC, TAP, LSP, and OPS across CIFAR-10/100/ImageNet-100, multiple architectures, and several defenses, concluding that most UEs have low UD but TAP is not a true UE. The central empirical claim is that existing UEs fail to degrade multi-task performance on Taskonomy.

Significance. If the multi-task finding is correct, it is a useful contribution: it challenges the common single-task evaluation of UEs and opens a concrete new direction. The paper also ships code and benchmarks several methods across datasets, architectures, and defenses, which is a practical asset for the community. However, the proposed metrics are not yet established as intrinsic measures. SAL is motivated by a causal claim about loss flatness that is asserted rather than tested, and UD is a fitted, initialization-dependent quantity with no sensitivity analysis or external validation. The current benchmark therefore does not yet answer the title question; the metric part needs substantial strengthening. With those caveats, the empirical multi-task observation alone is a defensible contribution.

major comments (5)
  1. [§3.2, Definition 1 (Eq. 3)] The central premise that low SAL is the causal mechanism of unlearnability is not demonstrated. The paper asserts that flat regions imply the model cannot learn, but flatness is also associated with good generalization (e.g., SAM). The text in §3.2 says 'we believe' and 'inspired by', not derived or tested. Since UD is built on SAL, this gap is load-bearing. An ablation is needed, e.g., show that parameters with low SAL are neither necessary nor sufficient for the test-accuracy drop, ideally across different architectures and random seeds.
  2. [§4, Definition 2 and Algorithm 1 (Eqs. 4–5)] UD is a fitted descriptor rather than a validated intrinsic metric. The threshold β is obtained by K-means on the clean model's SAL with K=2, and then the same clean model and the poisoned model are initialized identically (Algorithm 1, line 1) and evaluated on the same training runs. This makes the agreement with test accuracy in Table 1 partly circular. The paper provides no sensitivity tests for ϵ=0.05, the 10 inner steps, K=2, or the epoch ranges T_c/T_p, and no error bars over random seeds. As a result, UD's numerical values and rankings are not robustly established.
  3. [§5.2, Tables 1–3] The claimed consistency between UD and test accuracy is not supported by the reported numbers. For example, in Table 1 on CIFAR-100, REM has test accuracy 20.32 with UD 1.000, while DC has test accuracy 54.66 with UD 0.996; on ImageNet-100, LSP has test accuracy 4.40 with UD 1.796, higher than REM's UD 1.280 despite REM having similar test accuracy. In Table 3, ViT gives EM UD 1.573 and OPS UD 0.818, while the text says ViT is harder to poison. Without variance estimates or a quantitative correlation analysis, the benchmark conclusions are not convincing.
  4. [§3.4, Figures 6–8] The multi-task failure claim rests on single runs without error bars or statistical tests. Figure 6 shows ratios of change to vanilla metrics that are close to random-noise levels, but no confidence intervals are reported. Figures 7 and 8 also lack variance information and task-specific sample sizes. The claim that UEs 'still perform well' on segmentation and other tasks needs an effect-size analysis; otherwise it may be a null result caused by high multi-task training variance.
  5. [§3.3, Figure 3, and §5.2, Table 1] The treatment of TAP makes SAL unfalsifiable as a measure of unlearnability. TAP strongly reduces test accuracy but has high SAL, so the paper declares it 'not a true UE' and an adversarial example. This is a definitional exclusion, not a test of the metric. If the criterion for being a true UE is low SAL, then the finding that true UEs have low SAL is true by construction. The paper needs an independent characterization of UEs (e.g., perturbation linearity, shortcut mechanism) and should treat TAP as a potential counterexample rather than excluding it post hoc.
minor comments (5)
  1. [Figure 2 caption] The caption refers to 'ResNet-46', which appears to be a typo for ResNet-18 (the model used in Section 3.3).
  2. [Appendix A.5] The text uses 'naive classification' inconsistently; the main text uses 'Toy Classification'. Please unify the terminology.
  3. [References] Reference 'Zhang, 2017' and 'DeVries, 2017' should include full author lists and venue information; the current formatting is incomplete.
  4. [Table 1] The table row for EM on ImageNet-100 reports test accuracy 1.20 and UD 0.011, while the surrounding text says 'except for EM, all methods have relatively large UD (>1)'. This sentence appears to describe the ImageNet-100 column, but the '<1' values for EM are not mentioned there; please clarify.
  5. [Algorithm 1] The pseudocode omits the actual optimization steps for θ_c and θ_p; it only shows parameter assignments. Adding the SGD update would make the algorithm self-contained.

Circularity Check

2 steps flagged

Multi-task finding is independent, but the SAL/UD 'explanation' is definitional: unlearnability is equated with low training-loss sharpness, and UD restates that equation.

specific steps
  1. self definitional [Section 3.3, after Definition 1 / Eq. 3]
    "We observe that models trained on unlearnable datasets have a lower SAL compared to those trained on vanilla datasets, and the SAL of these parameters further decreases with training proceeding. Although this generally corresponds to the pattern of their demonstrated test accuracy, where methods that exhibit lower test dataset accuracy have fewer unlearnable parameters, there are also exceptions: TAP ... use adversarial examples for availability attack and get a significant test accuracy degradation. This helps us to distinguish which methods truly demonstrate unlearnability and which are simp"

    SAL is defined in Eq. 3 as the maximum training-loss change under a small layer-wise perturbation, i.e., a flatness measure. Naming it 'Sharpness-Aware Learnability' and then using the observation that UEs have lower SAL as an 'explanation' of unlearnability makes the explanation rest on the definition: a parameter is unlearnable exactly when its local loss surface is flat. No causal mechanism linking flatness to the test-accuracy drop is derived. When test accuracy contradicts the metric (TAP), the paper reclassifies TAP as 'not a true UE' rather than treating the contradiction as evidence against SAL, making the classification unfalsifiable by construction.

  2. self definitional [Section 4, before Definition 2]
    "we discovered that an unlearnable dataset leads to training failure by reducing the SAL of model parameters. In other words, the proportion of parameters with high SAL can directly indicate the unlearnability of a dataset."

    UD is then built on this: Eq. 5 is the ratio of parameters whose SAL exceeds a K-means threshold fit to the clean model. Since SAL was already defined as 'learnability,' the statement that fewer high-SAL parameters indicates unlearnability is true by construction, not by empirical derivation. The threshold is fit to the clean model, so UD measures the relative flatness of the poisoned model's loss surface under the clean model's threshold; it is a restatement of the SAL definition rather than an independent measure validated against held-out behavior.

full rationale

The paper's headline empirical finding—that EM, OPS, and AR perturbations on Taskonomy do not degrade multi-task performance—is independent and not circular; it is based on task metrics on a held-out multi-task benchmark. However, the paper's more ambitious claim to explain unlearnability via loss landscape is circular in a definitional sense. SAL (Eq. 3) is defined as a sharpness/flatness measure of the training loss, and then the paper equates low SAL with unlearnability, using this equivalence to 'explain' why UEs work. The explanation reduces to the definition. UD (Eq. 5) inherits this circularity: it counts parameters above a threshold derived from the clean model's SAL and calls the ratio 'unlearnable distance,' so the benchmark is a restatement of the SAL definition rather than an independent test. The paper does validate SAL against test accuracy for most methods, which gives some empirical content, but the exception handling for TAP shows the metric is used to override the external criterion. There is no load-bearing self-citation or imported uniqueness theorem; the circularity is internal to the metric construction. Overall, the qualitative multi-task result is a genuine contribution, so the score is moderate rather than extreme.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central metric rests on several hand-chosen hyperparameters (epsilon, K, inner steps, epoch averaging) and two behavioral assumptions (flatness prevents learning; layer-wise frozen SAL approximates joint training). The paper introduces SAL and UD as the main quantities but provides no independent falsifiable evidence for either; they are validated only by consistency with test accuracy on the same datasets used to construct the threshold.

free parameters (5)
  • epsilon (weight perturbation radius for SAL) = 0.05 (l2 norm)
    Chosen in Section 3.3 and used in Eq. 3 and Algorithm 1; every SAL value and the resulting threshold and UD depend on it. No sensitivity analysis is reported.
  • number of inner optimization steps for v = 10
    Section 3.3: 'the iterative step for optimizing v is 10'. It affects how tightly the max in Eq. 3 is approximated.
  • K-means cluster count = 2
    Definition 2 (Eq. 4) uses K-Means to split SAL values into two groups; the threshold is the average of the two cluster centers. The choice of K is not justified or varied.
  • threshold averaging over epochs = average over T epochs of the mean of two cluster centers
    Eq. 4 defines beta(T) with a uniform average; no weighting or alternative thresholding is tested.
  • training length settings T_c / T_p = 100 epochs in main text; some plots show 20
    UD in Eq. 5 depends on the epoch ranges used to average learnable parameter counts. Figure 5 displays only 20 epochs while the text states 100, an inconsistency that affects reproducibility.
axioms (5)
  • domain assumption Flatness of the loss landscape in a small l2 neighborhood is causally responsible for a parameter's unlearnability.
    Section 3.2: 'when the parameters are in the flat area of the loss landscape, the change in loss after the parameter update is minimal, presenting a performance similar to the second condition.' This is the conceptual bridge from loss landscape to unlearnability; no theorem or controlled experiment establishes it.
  • domain assumption Layer-wise SAL with other layers frozen approximates the layer's true contribution to learning.
    Definition 1: 'the parameters of the remaining layers are temporarily frozen.' During real SGD all layers update jointly, so the frozen-layer approximation may misattribute flatness.
  • ad hoc to paper K-means on clean-model SAL gives a meaningful two-way split between learnable and unlearnable parameters.
    Definition 2 (Eq. 4). The justification is post hoc: the split is accepted because it makes poisoned models look unlearnable in Figure 5.
  • domain assumption Initialization is identical for clean and poisoned models in UD evaluation.
    Algorithm 1, line 1: 'theta_p <- theta_0, theta_c <- theta_0'. In practice models are retrained from random seeds; UD may be seed-dependent.
  • domain assumption PCA projection onto top-2 parameter directions captures the important optimization dynamics.
    Section 3.1 and A.4: PCA along the optimization path is used to justify the 'few key parameters' conclusion; the top components explain over 90% variance on toy models but this is not established for multi-task networks.
invented entities (2)
  • Sharpness-Aware Learnability (SAL) no independent evidence
    purpose: Quantifies the unlearnability of model parameters as the worst-case training-loss change under a small weight perturbation (Eq. 3).
    A computational construct defined by the authors; its only evidence is internal correlation with test accuracy and UD, not an external falsifiable prediction.
  • Unlearnable Distance (UD) no independent evidence
    purpose: Ratio of learnable parameter counts in poisoned vs clean models, used as the paper's benchmark metric for data unlearnability (Eq. 5).
    Built directly on SAL and the hand-chosen threshold; it has no independent benchmark outside this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 14418 in / 16902 out tokens · 174058 ms · 2026-08-04T21:21:41.180750+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of How Far Are We from True Unlearnability?." pith.science (2026). https://pith.science/paper/MWUL5W2P

@misc{pith2026250908058,
  author       = {Pith},
  title        = {Pith review of: How Far Are We from True Unlearnability?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWUL5W2P}},
  note         = {Machine review of arXiv:2509.08058}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

High-quality data plays an indispensable role in the era of large models, but the use of unauthorized data for model training greatly damages the interests of data owners. To overcome this threat, several unlearnable methods have been proposed, which generate unlearnable examples (UEs) by compromising the training availability of data. Clearly, due to unknown training purposes and the powerful representation learning capabilities of existing models, these data are expected to be unlearnable for models across multiple tasks, i.e., they will not help improve the model's performance. However, unexpectedly, we find that on the multi-task dataset Taskonomy, UEs still perform well in tasks such as semantic segmentation, failing to exhibit cross-task unlearnability. This phenomenon leads us to question: How far are we from attaining truly unlearnable examples? We attempt to answer this question from the perspective of model optimization. To this end, we observe the difference in the convergence process between clean and poisoned models using a simple model architecture. Subsequently, from the loss landscape we find that only a part of the critical parameter optimization paths show significant differences, implying a close relationship between the loss landscape and unlearnability. Consequently, we employ the loss landscape to explain the underlying reasons for UEs and propose Sharpness-Aware Learnability (SAL) to quantify the unlearnability of parameters based on this explanation. Furthermore, we propose an Unlearnable Distance (UD) to measure the unlearnability of data based on the SAL distribution of parameters in clean and poisoned models. Finally, we conduct benchmark tests on mainstream unlearnable methods using the proposed UD, aiming to promote community awareness of the capability boundaries of existing unlearnable methods.

Figures

Figures reproduced from arXiv: 2509.08058 by Chenxiong Qian, Kai Ye, Liangcai Su.

Figure 1
Figure 1. Figure 1: Optimization in loss landscape of training process on Toy Classification task. We use PCA [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Training loss landscape of ResNet-46. It is more difficult for the model to converge [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Most parameters in the model trained on UEs exhibit a low SAL (except for TAP, which is [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (a) Heat maps of the cosine similarity between vectors composed of SAL parameters for [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Finding SAL threshold β(T) to distinguish learnable and unlearnable parameters in model trained on the vanilla dataset. The learnable parameters of poisoned models are only a small fraction in the early stages of training and decrease rapidly (except for TAP, which is considered as adver￾sarial examples), which further corroborates our hypothesis regarding the inherent unlearnability of UEs, suggesting tha… view at source ↗
Figure 6
Figure 6. Figure 6: Current UEs fail in multi-task model training with ResNet backbone and Taskonomy [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The visualization of the performance of EM (Error-Minizing) noise under multi-task learn [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Difference in model performance between poisoned models (trained on UEs) and clean [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of the CDF of model parameters between clean training and poison training. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Optimization in loss landscape of training process on naive classification dataset and [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Heat map of SAL variation of model parameters with training epochs. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Finding SAL threshold β to distinguish learnable and unlearnable parameters on the vanilla dataset by K-means. The poisoned models have only a small number of learnable parameters in the early training stage, which quickly diminish as the training proceeds. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Proportion of learnable parameters under different UEs with SAL threshold [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 2 canonical work pages

  1. [6]

    Unlearnable examples: Making personal data unexploitable.arXiv preprint arXiv:2101.04898,

    Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unlearnable examples: Making personal data unexploitable.arXiv preprint arXiv:2101.04898,

    Pith/arXiv arXiv

  2. [7]

    Fantastic generalization measures and where to find them.arXiv preprint arXiv:1912.02178,

    Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them.arXiv preprint arXiv:1912.02178,

    Pith/arXiv arXiv 1912

  3. [8]

    On large-batch training for deep learning: Generalization gap and sharp minima.arXiv preprint arXiv:1609.04836,

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- ter Tang. On large-batch training for deep learning: Generalization gap and sharp minima.arXiv preprint arXiv:1609.04836,

    Pith/arXiv arXiv

  4. [9]

    Apbench: A unified benchmark for availability poisoning attacks and defenses.arXiv preprint arXiv:2308.03258, 2023a

    Tianrui Qin, Xitong Gao, Juanjuan Zhao, Kejiang Ye, and Cheng-Zhong Xu. Apbench: A unified benchmark for availability poisoning attacks and defenses.arXiv preprint arXiv:2308.03258, 2023a. Tianrui Qin, Xitong Gao, Juanjuan Zhao, Kejiang Ye, and Cheng-Zhong Xu. Learning the un- learnable: Adversarial augmentations suppress unlearnable example attacks.arXiv...

    Pith/arXiv arXiv

  5. [10]

    How does sharpness-aware minimization minimize sharpness?arXiv preprint arXiv:2211.05729,

    Kaiyue Wen, Tengyu Ma, and Zhiyuan Li. How does sharpness-aware minimization minimize sharpness?arXiv preprint arXiv:2211.05729,

    Pith/arXiv arXiv

  6. [2007]

    Entropy-sgd: Biasing gradient descent into wide valleys.Journal of Statistical Mechanics: Theory and Experiment, 2019(12): 124018,

    Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys.Journal of Statistical Mechanics: Theory and Experiment, 2019(12): 124018,

    work page 2019

  7. [2016]

    Improved regularization of convolutional neural networks with cutout.arXiv preprint arXiv:1708.04552,

    Terrance DeVries. Improved regularization of convolutional neural networks with cutout.arXiv preprint arXiv:1708.04552,

    Pith/arXiv arXiv

  8. [2017]

    Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data.arXiv preprint arXiv:1703.11008,

    Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data.arXiv preprint arXiv:1703.11008,

    Pith/arXiv arXiv

  9. [2018]

    mixup: Beyond empirical risk minimization.arXiv preprint arXiv:1710.09412,

    Hongyi Zhang. mixup: Beyond empirical risk minimization.arXiv preprint arXiv:1710.09412,

    Pith/arXiv arXiv

  10. [2019]

    Sharpness-aware minimiza- tion for efficiently improving generalization.arXiv preprint arXiv:2010.01412,

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimiza- tion for efficiently improving generalization.arXiv preprint arXiv:2010.01412,

    Pith/arXiv arXiv 2010

  11. [2021]

    Robust unlearnable examples: Protecting data against adversarial learning.arXiv preprint arXiv:2203.14533,

    Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, and Dacheng Tao. Robust unlearnable examples: Protecting data against adversarial learning.arXiv preprint arXiv:2203.14533,

    Pith/arXiv arXiv

  12. [2022]

    One-pixel shortcut: on the learning preference of deep neural networks.arXiv preprint arXiv:2205.12141,

    Shutong Wu, Sizhe Chen, Cihang Xie, and Xiaolin Huang. One-pixel shortcut: on the learning preference of deep neural networks.arXiv preprint arXiv:2205.12141,

    Pith/arXiv arXiv

  13. [2024]

    12 Published as a conference paper A APPENDIX A.1 VISUALIZATION OFUES UNDERMULTI-TASKSCENARIOS Please refer to Figure 7 for more details. class_object class_scene depth_euclidean edge_occlusion edge_texture keypoints2d keypoints3d normal principal_curvature reshading segment_unsup2d segment_unsup25d T ask Metrics 30 20 10 0 10 20 30 Ratio of Change to Van...

    work page arXiv 2018