Pith. sign in

REVIEW 5 major objections 6 minor 88 references

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

T0 review · 5 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InstructMixup claims that mixup-style augmentation works best when it stays inside one image, editing its salient patches with a generative model rather than blending two samples.

desk verdict A credible, broad empirical augmentation study with a genuinely new generative-editing component; the label-preservation story hinges on an unnamed verifier and the theory has a mismatch with the implementation, but the core empirical claim is worth taking seriously. read the letter →

arxiv 2607.19324 v1 pith:T7WO36PF submitted 2026-07-21 cs.CV

Khawar Islam , Arif Mahmood , Xin Jin , Naveed Akhtar This is my paper
classification cs.CV MSC 68T4568T0768U10
keywords dataaugmentationmixupsaliency-guidedgenerativeimageeditinglabel-preservingfractalmixingvicinalriskminimizationrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the dominant mixup recipe—interpolating between two images—is neither necessary nor optimal. InstructMixup instead builds each augmented sample from a single image: spectral-residual saliency locates discriminative patches at multiple scales, an instruction-guided generative model refines them offline, a frozen verifier confirms the label survives, fractal texture is blended into the salient regions, and the edited patches are rotated, blurred, and recomposed into the original frame. The original hard label is kept throughout, and because the edits are computed once and cached, the method adds only a saliency map and a cache lookup to training. The paper derives a second-order approximation of the vicinal risk showing that this pipeline enforces invariance to the generative edit and penalizes loss curvature along salient directions, and it reports consistent accuracy gains over nine competing augmentation methods across seven benchmarks. A sympathetic reader would care because the result suggests a cheap, label-safe augmentation that is both empirically strong and mechanistically explained.

What carries the argument

The load-bearing machinery is the single-frame augmentation operator A_ξ(x) = T_ξ(x) + Δ_ξ(x), split into a structured saliency-guided transform and a localized zero-mean fractal perturbation. The structured transform T_ξ includes the cached instruction-guided edit E_ξ, rotation of the salient stream, blurring of the non-salient stream, and weighted recomposition; the perturbation Δ_ξ = (1−γ)β(B_ξ ⊙ Ẑ) injects fractal texture only inside accepted salient masks. The paper's key identity is the second-order vicinal-risk expansion VRM(h_θ) ≈ E[g(T_ξ(x))] + β²/6 · E[tr(H_g(T_ξ(x)) Σ_loc(x))], where Σ_loc is the mask-dependent fractal covariance. That expansion converts the heuristic recipe into

What would settle it

Measure the verifier's per-class error on a fine-grained benchmark such as CUB-200 or FGVC-Aircraft; if a non-negligible fraction of cached edits are mislabeled, train with the verifier disabled and observe whether the reported fine-grained margins vanish or invert. A second check: compute the curvature penalty term tr(H_g Σ_loc) on a held-out set and test whether setting the fractal blend β = 0 eliminates the robustness improvements predicted by Eq. 27.

Watch

Extended reading notes

Core claim

The central claim is that label-consistent augmentation can be generated wholly within a single visual sample by editing its salient regions with an instruction-guided generative model, blending self-similar fractal structure into those same regions, and recomposing the result, all while retaining the original hard label. The paper's second-order vicinal-risk analysis decomposes the augmented objective into an invariance term, which drives the model to be robust to the structured generative edit, and a saliency-local stability penalty β²/6 · tr(H_g Σ_loc), which penalizes loss curvature only along the perturbed salient directions. The paper argues this explains why the method generalizes bet

Load-bearing premise

The load-bearing premise is that every cached generative edit that survives the frozen verifier genuinely preserves the source class, so that keeping the original hard label is justified—and the paper itself concedes that disabling the verifier admits edits that drift toward a neighboring class, with the worst damage on fine-grained datasets.

Editorial extensions

If this is right

  • If InstructMixup's central claim holds, mixup-style augmentation no longer needs a second image: all variation is produced inside the source sample, sidestepping the semantic breakage that cross-image blending causes.
  • The original hard label stays valid throughout, so the method drops into standard cross-entropy training without soft-label bookkeeping, making it easy to integrate into existing pipelines.
  • Because generative edits are cached offline, the per-epoch training cost is nearly the same as no augmentation, which contradicts the common assumption that generative augmentation is too expensive for large-scale training.
  • The derived second-order risk predicts both observed effects—invariance to the generative edit and curvature suppression along salient directions—so the empirical gains come with a mechanism that could guide future augmentation design.
  • The reported margins over the strongest baseline are consistent across coarse and fine-grained recognition, corruption and occlusion robustness, calibration, transfer learning, and self-supervised pre-training, suggesting the method is not tuned to one benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The verifier's per-class error rate is never measured; a natural extension is to report accuracy as a function of verifier confidence, since fine-grained classes are exactly where the label-preservation premise is thinnest.
  • The offline cache makes the generative editor a plug-in: swapping in a stronger or cheaper editor is a drop-in change, which could turn the method into a platform for studying how edit diversity translates to robustness.
  • The saliency-localized curvature penalty suggests a general design rule—concentrate high-frequency perturbation where class evidence lives and keep background clean—so other perturbation families beyond fractals could exploit the same identity.
  • The paper's own conclusion notes that mixing modes are selected uniformly at random; learned gating or instance-adaptive mode selection is the obvious next step, trading a little training time for potentially larger gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes InstructMixup, a per-sample data augmentation method that extracts multi-scale salient patches, edits them offline with an instruction-guided generative model under a label-consistency filter, blends self-similar fractal structure into the salient regions, applies rotation/blur to salient/non-salient streams, and recomposes the result with the original image, optionally mixing with Mixup/CutMix/ResizeMix. The authors claim state-of-the-art results over nine competing augmentation methods across seven benchmarks, and derive a second-order vicinal-risk approximation that they interpret as enforcing invariance to the generative edit while penalizing curvature along salient directions. An extensive experiment suite covers CNNs, ViTs, fine-grained recognition, corruption robustness, calibration, transfer, self-supervised, few-shot, and detection.

Significance. If the claims hold, InstructMixup would be a practically valuable augmentation method: the offline cache makes it cheap, the within-sample design avoids cross-image label corruption, and the theoretical analysis is an attempt to explain why it works. The manuscript's empirical scope is impressive and includes useful ablations (Table IV, Table VI), efficiency accounting (Panel D of Table VI), and a parameter-free Taylor-expansion analysis (Eqs. 17-27). The core predictions - invariance to the structured transform and a saliency-local curvature penalty - are falsifiable in principle. However, the theoretical derivation is currently not faithful to the implemented algorithm, and the label-preservation assumption rests on an unverified verifier; these issues must be addressed before the central claims can be accepted.

major comments (5)
  1. [§IV-B, Eq. (14)-(16) vs. Algorithm 1, Eqs. (7)-(8)] The theory decomposes the augmentation as A_xi = T_xi + Delta_xi, with the fractal perturbation Delta_xi = (1-gamma) beta (B_xi . Z_tilde) added after the structured transform. In the algorithm, however, the fractal z is blended into q^e_j in Eq. (7) before the dual-stream rotation/blur in Eq. (8). Thus the fractal enters the rotated salient stream and the blurred non-salient stream, whereas Eq. (15) applies R and Psi_sigma only to the generative edit (and to x in the background). The expansion in Eq. (27) is therefore an analysis of a different augmentation operator from the one implemented. Either change the algorithm so fractal is added post-transform, or redo the derivation with the fractal inside T_xi. As written, the central theoretical claim does not match the method.
  2. [§IV-C, Eq. (24)] B_xi is defined as a {0,1}^{1 x H x W} mask, but Eq. (24) writes E_xi[delta delta^T] = (beta^2/3) E_xi[B_xi Sigma_Z B_xi^T]. With B_xi a row vector, B_xi Sigma_Z B_xi^T is a scalar, while delta delta^T is a matrix in the input space. If B_xi is intended as a diagonal matrix (diag of the mask), it should be written diag(B_xi) Sigma_Z diag(B_xi). The trace in Eq. (26) only makes sense with the diagonal interpretation. This is a technical but load-bearing flaw in the derivation's main formula.
  3. [§III-B, Eq. (6) and §VII 'Effect of Label-Consistency Verification'] The label-preservation guarantee is entirely delegated to the frozen verifier V(.) in Eq. (6). The paper never names the architecture of V, reports no rejection-rate statistics, and gives no measurement of V's error on the actual distribution of edited patches. Sec. VII concedes that disabling the verifier admits edits that drift toward neighboring classes, especially on fine-grained datasets - where the reported gains are largest. Without a concrete verifier specification and per-dataset false-accept rates, the assumption that T_xi leaves the label unchanged (used to justify Eq. (11) and the Sec. IV-C expansion) is unsupported. Please add these measurements or substantially weaken the label-consistency claim.
  4. [§VI (Table I Panel B, Table II)] For fine-grained classification and downstream transfer, the text states: 'we report baseline results directly from that work' (AdAutoMix). The abstract and contributions claim 'reproducing 27 competing augmentation strategies,' but these baselines are not reproduced under the same pipeline. Borrowed numbers may use different schedules, pre-training, or hyperparameters, so the reported margins (e.g., +1.16-1.68% over AdAutoMix) are not controlled comparisons. The authors should reproduce the baselines in their setup or explicitly restrict the SOTA claim to settings where all baselines were run in-house.
  5. [§VI Transfer Learning, Table II] The text says 'ResNet-50 and ViT-B models are first pretrained on ImageNet-1K,' but Table II's caption and column headers say 'CLIP ResNet-50 and DINOv2 ViT-S/14.' These are materially different models. The transfer-learning claims cannot be evaluated until this inconsistency is resolved.
minor comments (6)
  1. [§III-C and Fig. 5] The fractal-blend strength is denoted beta in Eq. (7) and lambda in Fig. 5 and the surrounding text; unify the notation.
  2. [§VII, Table IV] The symbol M_f appears in the table body ('M_f + M_m + M_c + M_r') but is only defined in the prose; define it in the caption as well.
  3. [§VII, 'Motivation behind High-level Mixing'] The phrase 'as in [85]' is used but RandomMix is not introduced in Related Work; provide context or remove the citation.
  4. [Abstract/Contributions] The paper claims to outperform 'nine competing augmentation methods' but Tables II and III include more than nine methods (e.g., GridMix, FracMix, PixMix). Clarify the counting of compared methods and the '27 competing augmentation strategies' claim.
  5. [Throughout] Notation for the conference predecessor is inconsistent: 'S2-FracMix', 'S^2-FracMix', and 'S-hat-2-FracMix' are used; standardize.
  6. [Fig. 5 caption] The caption says 'saliency threshold t' and 'fractal-blend strength lambda', but Eq. (3) uses tau with lower bound t and Eq. (7) uses beta; align the caption with the equations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the VRM analysis is a parameter-free Taylor expansion; the verifier limitation is an explicit, non-circular assumption.

full rationale

The claimed derivation chain is not circular. Section IV defines the augmented sample x' = A_xi(x), splits A into T_xi + Delta_xi, and constructs Delta_xi to be zero-mean (Eqs. 14-16). The second-order expansion (Eqs. 17-27) is a parameter-free Taylor identity: the first-order term vanishes because E[delta]=0 by construction, and the curvature term is the second moment of that same zero-mean perturbation. No fitted constant is inserted into the derivation; beta is set by a validation sweep and enters only as an explicit coefficient. The only substantive premise is the label-preservation assumption, which the paper states transparently in Sec. IV-C ('We additionally require that T_xi leave the semantic label unchanged') and ties to the offline verifier of Eq. 6. Sec. VII then explicitly concedes the limitation that disabling the verifier admits label drift, most damaging on fine-grained data. That is an empirical risk/assumption about the verifier's accuracy, not a circular reduction: agreement with y_i is a filtering rule, not a fitted parameter renamed as a prediction. The paper's self-citations ([35], [37], [55], [60]) are used as baselines, an explicit prior-work extension, or a source of fractal images; none is invoked as an external uniqueness theorem or as the justification for the central second-order result. External benchmarks and ablations are reported. Hence no prediction reduces to its inputs by construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The method introduces no new physical or model-level entities: the fractal library, saliency detector, and generative editor are imported from existing work. The non-trivial commitments are the four validation-tuned hyperparameters (β, t, |Z|, rotation range) and the six domain/math assumptions above, especially the verifier's reliability and the theory's simplified operator decomposition.

free parameters (4)
  • β (fractal-blend strength) = 0.20
    Chosen by sweeping candidate values on CIFAR-100 withheld validation (Sec. III-C, Fig. 5b) and fixed in all experiments; also denoted λ in Sec. VII.
  • t (saliency threshold lower bound) = 0.5
    Lower bound of the random acceptance threshold τ in Eq. 3; tuned on CIFAR-100 validation (Fig. 5a).
  • |Z| (fractal library size) = 500
    Tuned on CIFAR-100 validation (Fig. 5c); accuracy saturates at 500.
  • rotation range ±φ of salient stream = ±30°
    Tuned on CIFAR-100 validation (Fig. 5d); ±60° drops accuracy to 82.15%.
assumptions (6)
  • domain assumption Spectral-residual saliency map Φ(x) (Eq. 1) localizes class-discriminative content.
    The whole pipeline (patch acceptance, edit locus, fractal placement) is steered by this map; the paper provides no evaluation that the map aligns with ground-truth discriminative evidence.
  • domain assumption The frozen verifier V (Eq. 6) is a reliable guard of label preservation for generative edits.
    The label-preservation license in Sec. IV-C ('every cached edit must pass the offline verification of Eq. 6') and the hard-label training objective (Eq. 11) ride on V's accuracy; V is never specified, and the paper concedes edits drift toward neighboring classes when V is disabled (Sec. VII).
  • domain assumption Self-similar fractal statistics improve robustness when localized to salient regions.
    Borrowed from fractal augmentation literature [47], [36], [21]; the paper does not independently test why fractals help, only where to place them.
  • domain assumption Rotation of the salient stream (Eq. 8) induces label-preserving viewpoint invariance.
    ±30° rotation of the object patch is assumed not to change the class; Fig. 5d shows accuracy falls at ±60°, consistent with the assumption only for mild angles.
  • standard math Taylor expansion of the loss to second order is valid for the perturbation magnitude β=0.2.
    Eq. 19 requires twice-differentiable g and small δ; no check of higher-order terms is given, and the fractal perturbation is scaled by β=0.2 (Eq. 16).
  • ad hoc to paper The augmentation decomposes exactly as A_ξ = T_ξ + Δ_ξ with independent zero-mean fractal noise added post-transform.
    Eqs. 14-16 place the fractal in Δ_ξ after rotation/blur, but Algorithm 1/Eq. 7 blends the fractal into the patch before rotation and blur (Eq. 8). This modeling simplification is not disclosed in Sec. IV; it is needed to obtain E[δ]=0 and the clean β²/6 curvature term.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation." pith.science (2026). https://pith.science/paper/T7WO36PF

@misc{pith2026260719324,
  author       = {Pith},
  title        = {Pith review of: InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T7WO36PF}},
  note         = {Machine review of arXiv:2607.19324}
}
read the original abstract

In image and video technologies, data augmentation is widely used to improve the generalization of deep visual models, and mixup-based strategies that interpolate between samples have become the dominant approach. However, computing informative mixing regions adds substantial overhead, and blending content across different images frequently disrupts the semantic integrity of the resulting sample. We propose \our{}, a data augmentation method that constructs challenging yet label-consistent training samples entirely within a single visual sample. \our{} first extracts multi-scale salient patches from the sample using a lightweight saliency detector, refines each patch with an instruction-guided generative model, and blends the edited patch back into the non-salient regions of the same sample; because the generative edits are computed once and cached offline, this step adds negligible training cost. To further diversify the learned representation, \our{} injects self-similar fractal structure into the same salient regions at an adaptive ratio, so each training sample carries both fractal and non-fractal structure. We derive a second-order approximation of the resulting vicinal risk, showing that the method simultaneously enforces invariance to the generative edit and suppresses curvature along the perturbed salient directions, and we verify both predictions empirically. We evaluate on small to large backbones for instance Convolutional Neural Networks (CNNs), Vision Transformers (ViTs) and Vision-Language Foundational Models (VLMs) across seven benchmarks covering coarse- and fine-grained classification, robustness to corruption and occlusion, calibration, and transfer and self-supervised learning, InstructMixup outperforms nine competing augmentation methods, surpassing the strongest baseline across all benchmarks.

Figures

Figures reproduced from arXiv: 2607.19324 by the authors.

Figure 1
Figure 1. Representative augmentation samples created by different mixup methods. Each competing method constructs its sample from a source and a target [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. InstructMixup overview. In the InstructMixup mode, the spectral-residual detector computes a saliency map and selects salient patches at two scales. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Calibration plots of InstructMixup on CIFAR100 using [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative visualization of the InstructMixup pipeline on samples from CUB-Birds [ [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Hyper-parameter ablation of InstructMixup on CIFAR-100 with [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Grad-CAM [86] visualization on un-augmented and augmented images [87], [62], [28]. InstructMixup effectively identifies target objects even under severe occlusion or mixing scenarios. considered: Saliency (A+B), which mixes a target image B under guidance from the sali…
Figure 7
Figure 7. Figure 7: 2-D t-SNE embeddings of CIFAR-10 features under each augmenta [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

88 extracted references · 13 linked inside Pith

  1. [1]

    Openmix+: Revisiting data augmentation for open set recognition,

    G. Jiang, P. Zhu, Y . Wang, and Q. Hu, “Openmix+: Revisiting data augmentation for open set recognition,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 11, pp. 6777–6787, 2023

    2023

  2. [2]

    mixup: Beyond empirical risk minimization,

    H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” inInternational Conference on Learning Representations (ICLR), 2018

    2018

  3. [3]

    A survey of mix-based data augmentation: Taxonomy, methods, applications, and explainability,

    C. Cao, F. Zhou, Y . Dai, J. Wang, and K. Zhang, “A survey of mix-based data augmentation: Taxonomy, methods, applications, and explainability,”ACM Computing Surveys, vol. 57, no. 2, pp. 1–38, 2024

    2024

  4. [4]

    On mixup regular- ization,

    L. Carratino, M. Ciss ´e, R. Jenatton, and J.-P. Vert, “On mixup regular- ization,”Journal of Machine Learning Research, vol. 23, no. 325, pp. 1–31, 2022

    2022

  5. [5]

    Dynamic-aware enhanced data augmentation for light field super-resolution,

    Z.-Y . Mi and Y .-B. Yang, “Dynamic-aware enhanced data augmentation for light field super-resolution,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 36, no. 6, pp. 8431–8448, 2026

    2026

  6. [6]

    Guidedmixup: an efficient mixup strategy guided by saliency maps,

    M. Kang and S. Kim, “Guidedmixup: an efficient mixup strategy guided by saliency maps,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 1, 2023, pp. 1096–1104

    2023

  7. [7]

    Puzzle mix: Exploiting saliency and local statistics for optimal mixup,

    J.-H. Kim, W. Choo, and H. O. Song, “Puzzle mix: Exploiting saliency and local statistics for optimal mixup,” inInternational Conference on Machine Learning. PMLR, 2020, pp. 5275–5285

    2020

  8. [8]

    Co-mixup: Saliency-guided joint mixup with supermodular diversity,

    J.-H. Kim, J. Park, and S. Hwang, “Co-mixup: Saliency-guided joint mixup with supermodular diversity,” inAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

Show all 88 references
  1. [9]

    Data augmentation using random image cropping and patching for deep CNNs,

    R. Takahashi, T. Matsubara, and K. Uehara, “Data augmentation using random image cropping and patching for deep CNNs,”IEEE Transac- tions on Circuits and Systems for Video Technology, vol. 30, no. 9, pp. 2917–2931, 2020

    2020

  2. [10]

    Regularizing deep networks with semantic data augmentation,

    Y . Wang, G. Huang, S. Song, X. Pan, Y . Xia, and C. Wu, “Regularizing deep networks with semantic data augmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3733– 3748, 2022

    2022

  3. [11]

    Token-label alignment for vision transformers,

    H. Xiao, W. Zheng, Z. Zhu, J. Zhou, and J. Lu, “Token-label alignment for vision transformers,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 5495–5504

    2023

  4. [12]

    Enhance image classification via inter-class image mixup with diffusion model,

    Z. Wang, L. Wei, T. Wang, H. Chen, Y . Hao, X. Wang, X. He, and Q. Tian, “Enhance image classification via inter-class image mixup with diffusion model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17 223–17 233

    2024

  5. [13]

    Deep long-tailed learning: A survey,

    Y . Zhang, B. Kang, B. Hooi, S. Yan, and J. Feng, “Deep long-tailed learning: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 9, pp. 10 795–10 816, 2023. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 13

    2023

  6. [14]

    Using mixup as a regularizer can surprisingly improve accuracy & out-of-distribution ro- bustness,

    F. Pinto, H. Yang, S. N. Lim, P. Torr, and P. Dokania, “Using mixup as a regularizer can surprisingly improve accuracy & out-of-distribution ro- bustness,”Advances in Neural Information Processing Systems, vol. 35, pp. 14 608–14 622, 2022

    2022

  7. [15]

    A survey on mixup augmentations and beyond,

    X. Jin, H. Zhu, S. Li, Z. Wang, Z. Liu, C. Yu, H. Qin, and S. Z. Li, “A survey on mixup augmentations and beyond,”arXiv preprint arXiv:2409.05202, 2024

    2024 arXiv

  8. [16]

    Domain general- ization: A survey,

    K. Zhou, Z. Liu, Y . Qiao, T. Xiang, and C. C. Loy, “Domain general- ization: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4396–4415, 2023

    2023

  9. [17]

    A comprehensive survey on source-free domain adaptation,

    Z. Yu, J. Li, Z. Du, L. Zhu, and H. T. Shen, “A comprehensive survey on source-free domain adaptation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 8, 2024

    2024

  10. [18]

    Fine- grained recognition with learnable semantic data augmentation,

    Y . Pu, Y . Han, Y . Wang, J. Feng, C. Deng, and G. Huang, “Fine- grained recognition with learnable semantic data augmentation,”IEEE Transactions on Image Processing, vol. 33, 2024

    2024

  11. [19]

    Lgcoamix: Local and global context-and- object-part-aware superpixel-based data augmentation for deep visual recognition,

    F. Dornaika and D. Sun, “Lgcoamix: Local and global context-and- object-part-aware superpixel-based data augmentation for deep visual recognition,”IEEE Transactions on Image Processing, 2023

    2023

  12. [20]

    PatchMix augmentation to identify causal features in few-shot learning,

    C. Xu, C. Liu, X. Sun, S. Yang, Y . Wang, C. Wang, and Y . Fu, “PatchMix augmentation to identify causal features in few-shot learning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 6, 2023

    2023

  13. [21]

    Ipmix: Label-preserving data augmentation method for training robust classifiers,

    Z. Huang, X. Bao, N. Zhang, Q. Zhang, X. Tu, B. Wu, and X. Yang, “Ipmix: Label-preserving data augmentation method for training robust classifiers,”Advances in Neural Information Processing Systems, vol. 36, pp. 63 660–63 673, 2023

    2023

  14. [22]

    Cropmix: Sampling a rich input distribution via multi-scale cropping,

    J. Han, L. Petersson, H. Li, and I. Reid, “Cropmix: Sampling a rich input distribution via multi-scale cropping,”arXiv preprint arXiv:2205.15955, 2022

    2022 arXiv

  15. [23]

    Ddb: Diffusion driven balancing to address spurious correlations,

    A. Y . Parast, B. Azam, and N. Akhtar, “Ddb: Diffusion driven balancing to address spurious correlations,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 17 526–17 535

    2025

  16. [24]

    Ghost: Hallucination-inducing image genera- tion for multimodal llms,

    A. Y . Parast, P. Hosseini, H. Asadollahzadeh, A. S. Moakhar, B. Azam, S. Feizi, and N. Akhtar, “Ghost: Hallucination-inducing image genera- tion for multimodal llms,”arXiv preprint arXiv:2509.25178, 2025

    2025

  17. [25]

    Augmix: A simple data processing method to improve robustness and uncertainty,

    D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Lak- shminarayanan, “Augmix: A simple data processing method to improve robustness and uncertainty,” inInternational Conference on Learning Representations (ICLR), 2020

    2020

  18. [26]

    Manifold mixup: Better representations by interpolat- ing hidden states,

    V . Verma, A. Lamb, C. Beckham, A. Najafi, I. Mitliagkas, D. Lopez-Paz, and Y . Bengio, “Manifold mixup: Better representations by interpolat- ing hidden states,” inInternational Conference on Machine Learning. PMLR, 2019, pp. 6438–6447

    2019

  19. [27]

    Mergemix: A unified augmentation paradigm for visual and multi-modal understanding,

    X. Jin, S. Li, S. Jian, K. Yu, and H. Wang, “Mergemix: A unified augmentation paradigm for visual and multi-modal understanding,” arXiv preprint arXiv:2510.23479, 2025

    2025

  20. [28]

    Cutmix: Reg- ularization strategy to train strong classifiers with localizable features,

    S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y . Yoo, “Cutmix: Reg- ularization strategy to train strong classifiers with localizable features,” inInternational Conference on Computer Vision (ICCV), 2019

    2019

  21. [29]

    Alignmixup: Improving representations by interpolating aligned features,

    S. Venkataramanan, E. Kijak, L. Amsaleg, and Y . Avrithis, “Alignmixup: Improving representations by interpolating aligned features,” inPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 19 174–19 183

    2022

  22. [30]

    Resizemix: Mixing data with preserved object information and true labels,

    J. Qin, J. Fang, Q. Zhang, W. Liu, X. Wang, and X. Wang, “Resizemix: Mixing data with preserved object information and true labels,”arXiv preprint arXiv:2012.11101, 2020

    2012 arXiv

  23. [31]

    Saliencymix: A saliency guided data augmentation strategy for better regularization,

    A. S. Uddin, M. S. Monira, W. Shin, T. Chung, and S.-H. Bae, “Saliencymix: A saliency guided data augmentation strategy for better regularization,” inInternational Conference on Learning Representa- tions, 2020

    2020

  24. [32]

    Automix: Unveiling the power of mixup for stronger classifiers,

    Z. Liu, S. Li, D. Wu, Z. Liu, Z. Chen, L. Wu, and S. Z. Li, “Automix: Unveiling the power of mixup for stronger classifiers,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 441–458

    2022

  25. [33]

    Adversarial automixup,

    H. Qin, X. Jin, Y . Jiang, M. El-Yacoubi, and X. Gao, “Adversarial automixup,” inThe Twelfth International Conference on Learning Rep- resentations, 2024, pp. OpenReview, Spotlight

    2024

  26. [34]

    A survey on vision transformer,

    K. Han, Y . Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y . Tang, A. Xiao, C. Xu, Y . Xu, Z. Yang, Y . Zhang, and D. Tao, “A survey on vision transformer,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 87–110, 2023

    2023

  27. [35]

    Diffusemix: Label-preserving data augmentation with diffusion models,

    K. Islam, M. Z. Zaheer, A. Mahmood, and K. Nandakumar, “Diffusemix: Label-preserving data augmentation with diffusion models,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 621–27 630

    2024

  28. [36]

    Pixmix: Dreamlike pictures comprehensively improve safety measures,

    D. Hendrycks, A. Zou, M. Mazeika, L. Tang, B. Li, D. Song, and J. Steinhardt, “Pixmix: Dreamlike pictures comprehensively improve safety measures,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 783–16 792

    2022

  29. [37]

    ˆS2-fracmix: Label-preserving self-saliency mixup augmentation,

    K. Islam, A. Mahmood, X. Jin, and N. Akhtar, “ ˆS2-fracmix: Label-preserving self-saliency mixup augmentation,”arXiv preprint arXiv:2606.25784, 2026

    2026 arXiv

  30. [38]

    Smoothmix: a simple yet effective data augmentation to train robust classifiers,

    J.-H. Lee, M. Z. Zaheer, M. Astrid, and S.-I. Lee, “Smoothmix: a simple yet effective data augmentation to train robust classifiers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 756–757

    2020

  31. [39]

    Recursivemix: Mixed learning with history,

    L. Yang, X. Li, B. Zhao, R. Song, and J. Yang, “Recursivemix: Mixed learning with history,”Advances in Neural Information Processing Systems, vol. 35, pp. 8427–8440, 2022

    2022

  32. [40]

    Stylemix: Separating content and style for enhanced data augmentation,

    M. Hong, J. Choi, and G. Kim, “Stylemix: Separating content and style for enhanced data augmentation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14 862–14 870

    2021

  33. [41]

    Fmix: Enhancing mixed sample data augmentation,

    E. Harris, A. Marcu, M. Painter, M. Niranjan, A. Pr ¨ugel-Bennett, and J. Hare, “Fmix: Enhancing mixed sample data augmentation,”arXiv preprint arXiv:2002.12047, 2020

    2002 arXiv

  34. [42]

    Gridmix: Strong regularization through local context mapping,

    K. Baek, D. Bang, and H. Shim, “Gridmix: Strong regularization through local context mapping,”Pattern Recognition, vol. 109, p. 107594, 2021

    2021

  35. [43]

    Snapmix: Semantically proportional mixing for augmenting fine-grained data,

    S. Huang, X. Wang, and D. Tao, “Snapmix: Semantically proportional mixing for augmenting fine-grained data,” inProceedings of the AAAI Conference on Artificial Intelligence, 2021

    2021

  36. [44]

    Tokenmix: Rethinking image mixing for data augmentation in vision transformers,

    J. Liu, B. Liu, H. Zhou, H. Li, and Y . Liu, “Tokenmix: Rethinking image mixing for data augmentation in vision transformers,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 455–471

    2022

  37. [45]

    Mixpro: Data augmentation with maskmix and progressive attention labeling for vision transformer,

    Q. Zhao, Y . Huang, W. Hu, F. Zhang, and J. Liu, “Mixpro: Data augmentation with maskmix and progressive attention labeling for vision transformer,” inInternational Conference on Learning Representations, 2023

    2023

  38. [46]

    Harnessing hard mixed samples with decoupled regularizer,

    Z. Liu, S. Li, G. Wang, L. Wu, C. Tan, and S. Z. Li, “Harnessing hard mixed samples with decoupled regularizer,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  39. [47]

    Fractal features for texture analysıs,

    S. Vidivelli, S. Sathiya Devi, and G. Parthasarathy, “Fractal features for texture analysıs,” inInternational Conference on Data Science and Communication. Springer, 2023, pp. 247–257

    2023

  40. [48]

    Advmask: A sparse adversarial attack-based data augmentation method for image classifica- tion,

    S. Yang, J. Li, T. Zhang, J. Zhao, and F. Shen, “Advmask: A sparse adversarial attack-based data augmentation method for image classifica- tion,”Pattern Recognition, p. 109847, 2023

    2023

  41. [49]

    Attentive cutmix: An enhanced data augmentation approach for deep learning based image classification,

    D. Walawalkar, Z. Shen, Z. Liu, and M. Savvides, “Attentive cutmix: An enhanced data augmentation approach for deep learning based image classification,” inICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 3642–3646

    2020

  42. [50]

    Boosting dis- criminative visual representation learning with scenario-agnostic mixup,

    S. Li, Z. Liu, Z. Wang, D. Wu, Z. Liu, and S. Z. Li, “Boosting dis- criminative visual representation learning with scenario-agnostic mixup,” arXiv preprint arXiv:2111.15454, 2021

    2021 arXiv

  43. [51]

    Su- permix: Supervising the mixing data augmentation,

    A. Dabouei, S. Soleymani, F. Taherkhani, and N. M. Nasrabadi, “Su- permix: Supervising the mixing data augmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 13 794–13 803

    2021

  44. [52]

    Saliency grafting: Innocuous attribution-guided mixup with calibrated label mixing,

    J. Park, J. Y . Yang, J. Shin, S. J. Hwang, and E. Yang, “Saliency grafting: Innocuous attribution-guided mixup with calibrated label mixing,”arXiv preprint arXiv:2112.08796, 2021

    2021 arXiv

  45. [53]

    Gradsalmix: Gradient saliency-based mix for image data augmentation,

    T. Hong, Y . Wang, X. Sun, F. Lian, Z. Kang, and J. Ma, “Gradsalmix: Gradient saliency-based mix for image data augmentation,” in2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2023, pp. 1799–1804

    2023

  46. [54]

    Salfmix: a novel single image- based data augmentation technique using a saliency map,

    J. Choi, C. Lee, D. Lee, and H. Jung, “Salfmix: a novel single image- based data augmentation technique using a saliency map,”Sensors, vol. 21, no. 24, p. 8444, 2021

    2021

  47. [55]

    Context-guided responsible data augmentation with diffusion models,

    K. Islam and N. Akhtar, “Context-guided responsible data augmentation with diffusion models,” inICLR Workshop on Navigating and Address- ing Data Problems for Foundation Models, 2025

    2025

  48. [56]

    Effective data augmentation with diffusion models,

    B. Trabucco, K. Doherty, M. Gurinas, and R. Salakhutdinov, “Effective data augmentation with diffusion models,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  49. [57]

    Improving diffusion-based data augmentation with inversion spherical interpolation,

    Y . Wang and L. Chen, “Improving diffusion-based data augmentation with inversion spherical interpolation,”arXiv preprint arXiv:2408.16266, 2024

    2024 arXiv

  50. [58]

    Data augmentation for object detection via controllable diffusion models,

    H. Fang, B. Han, S. Zhang, S. Zhou, C. Hu, and W.-M. Ye, “Data augmentation for object detection via controllable diffusion models,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 1257–1266. IEEE TRANSACTIONS ON CIRCUITS AND SYST...

    2024

  51. [59]

    Diffaug: Enhance unsupervised contrastive learning with domain- knowledge-free diffusion-based data augmentation,

    Z. Zang, H. Luo, K. Wang, P. Zhang, F. Wang, S. Z. Li, and Y . You, “Diffaug: Enhance unsupervised contrastive learning with domain- knowledge-free diffusion-based data augmentation,” inForty-first Inter- national Conference on Machine Learning, 2024

    2024

  52. [60]

    Genmix: Effective data augmentation with generative diffusion model image editing,

    K. Islam, M. Z. Zaheer, A. Mahmood, K. Nandakumar, and N. Akhtar, “Genmix: Effective data augmentation with generative diffusion model image editing,”arXiv preprint arXiv:2412.02366, 2024

    2024

  53. [61]

    Saliency detection: A spectral residual approach,

    X. Hou and L. Zhang, “Saliency detection: A spectral residual approach,” in2007 IEEE Conference on computer vision and pattern recognition. Ieee, 2007, pp. 1–8

    2007

  54. [62]

    Mixup as locally linear out-of-manifold regularization,

    H. Guo, Y . Mao, and R. Zhang, “Mixup as locally linear out-of-manifold regularization,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 3714–3722

    2019

  55. [63]

    Vicinal risk mini- mization,

    O. Chapelle, J. Weston, L. Bottou, and V . Vapnik, “Vicinal risk mini- mization,”Advances in neural information processing systems, vol. 13, 2000

    2000

  56. [64]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, “Learning multiple layers of features from tiny images,” Master’s thesis, University of Tront, 2009

    2009

  57. [65]

    A downsampled variant of imagenet as an alternative to the cifar datasets,

    P. Chrabaszcz, I. Loshchilov, and F. Hutter, “A downsampled variant of imagenet as an alternative to the cifar datasets,”arXiv preprint arXiv:1707.08819, 2017

    2017 arXiv

  58. [66]

    Imagenet: A large- scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L. Li, K. Li, and F. Li, “Imagenet: A large- scale hierarchical image database,” inIEEE Conference on Computer Vision and Pattern Recognition, 2009

    2009

  59. [67]

    The Caltech-UCSD Birds-200-2011 Dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The Caltech-UCSD Birds-200-2011 Dataset,” California Institute of Tech- nology, Tech. Rep. CNS-TR-2011-001, 2011

    2011

  60. [68]

    Fine- grained visual classification of aircraft,

    S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi, “Fine- grained visual classification of aircraft,”arXiv preprint arXiv:1306.5151, 2013

    2013 arXiv

  61. [69]

    3d object representations for fine-grained categorization,

    J. Krause, M. Stark, J. Deng, and L. Fei-Fei, “3d object representations for fine-grained categorization,” in4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013

    2013

  62. [70]

    Automated flower classification over a large number of classes,

    M.-E. Nilsback and A. Zisserman, “Automated flower classification over a large number of classes,” in2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing. IEEE, 2008, pp. 722–729

    2008

  63. [71]

    The pascal visual object classes (voc) challenge,

    M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisser- man, “The pascal visual object classes (voc) challenge,”International journal of computer vision, vol. 88, pp. 303–338, 2010

    2010

  64. [72]

    Benchmarking neural network ro- bustness to common corruptions and perturbations,

    D. Hendrycks and T. Dietterich, “Benchmarking neural network ro- bustness to common corruptions and perturbations,”Proceedings of the International Conference on Learning Representations, 2019

    2019

  65. [73]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  66. [74]

    Aggregated residual transformations for deep neural networks,

    S. Xie, R. Girshick, P. Doll ´ar, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1492– 1500

    2017

  67. [75]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  68. [76]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11 976–11 986

    2022

  69. [77]

    Improved baselines with mo- mentum contrastive learning,

    X. Chen, H. Fan, R. Girshick, and K. He, “Improved baselines with mo- mentum contrastive learning,”arXiv preprint arXiv:2003.04297, 2020

    2003 arXiv

  70. [78]

    Exploring simple siamese representation learning,

    X. Chen and K. He, “Exploring simple siamese representation learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 15 750–15 758

    2021

  71. [79]

    Openmixup: A comprehensive mixup benchmark for visual classification,

    S. Li, Z. Wang, Z. Liu, D. Wu, C. Tan, W. Jin, and S. Z. Li, “Openmixup: A comprehensive mixup benchmark for visual classification,”arXiv preprint arXiv:2209.04851, 2022

    2022 arXiv

  72. [80]

    You only cut once: Boosting data augmentation with a single cut,

    J. Han, P. Fang, W. Li, J. Hong, M. A. Armin, I. Reid, L. Petersson, and H. Li, “You only cut once: Boosting data augmentation with a single cut,” inInternational Conference on Machine Learning. PMLR, 2022, pp. 8196–8212

    2022

  73. [81]

    Ssd: Single shot multibox detector,

    W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y . Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” inComputer Vision– ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, 2016, pp. 21–37

    2016

  74. [82]

    Fast r-cnn,

    R. Girshick, “Fast r-cnn,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 1440–1448

    2015

  75. [83]

    Wide residual networks,

    S. Zagoruyko and N. Komodakis, “Wide residual networks,” inBritish Machine Vision Conference 2016. British Machine Vision Association, 2016

    2016

  76. [84]

    Identity mappings in deep residual networks,

    K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” inEuropean conference on computer vision. Springer, 2016, pp. 630–645

    2016

  77. [85]

    Randomix: A mixed sample data augmentation method with multiple mixed modes,

    X. Liu, F. Shen, J. Zhao, and C. Nie, “Randomix: A mixed sample data augmentation method with multiple mixed modes,”Multimedia Tools and Applications, vol. 84, no. 8, pp. 4343–4359, 2025

    2025

  78. [86]

    Grad-cam: visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: visual explanations from deep networks via gradient-based localization,”International journal of computer vision, vol. 128, pp. 336–359, 2020

    2020

  79. [87]

    Improved regularization of convolutional neural networks with cutout,

    T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,”arXiv preprint arXiv:1708.04552, 2017

    2017 arXiv

  80. [88]

    Visualizing data using t-sne

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.”Journal of machine learning research, vol. 9, no. 11, 2008

    2008

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.