Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

T0 review · 2 major / 6 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Decoupled dataset distillation fails on fine-grained data until synthetic samples are pulled to class prototypes and forced to attend different local parts.

desk verdict Solid plug-in for fine-grained decoupled distillation: real gains at low IPC, honest ablations, soft spot is CAL-transfer fidelity not math. read the letter →

arxiv 2603.25144 v2 pith:PPMQZ4CF submitted 2026-03-26 cs.CV cs.AI

Hongxu Ma , Guang Li , Shijie Wang , Dongzhan Zhou , Baoli Sun , Takahiro Ogawa , Miki Haseyama , Zhihui Wang This is my paper
classification cs.CVcs.AI
keywords datasetdistillationfine-grainedrecognitiondecoupledcounterfactualattentionclassprototypesdiversitysynthetictrainingsets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard decoupled dataset distillation compresses a full training set into a tiny synthetic set using only coarse class labels, and it works on ordinary image benchmarks. On fine-grained tasks—where classes differ by subtle local parts—the same pipeline leaves two problems in the synthetic set: large within-class scatter with tiny between-class gaps, and nearly identical attended regions across same-class samples. This paper claims those defects are the reason student models trained on the synthetic set struggle at fine-grained recognition. FD² fixes them by pretraining a teacher that builds class prototypes from counterfactual attention, then adding two plug-in losses while synthesizing images: one that pulls each sample toward its own prototype and away from others, and one that diversifies attention maps inside each class group. Integrated into existing decoupled pipelines, the method raises post-evaluation accuracy on fine-grained and general image sets in most settings, especially when only one synthetic image per class is allowed.

What carries the argument

The fine-grained characteristic constraint and the similarity constraint (applied with dual-branch class supervision on Backbone+CAL features). The first aligns each distilled sample’s representation to its class prototype while repelling other prototypes via a normalized Euclidean loss; the second maximizes attention-map distance to previously synthesized same-class samples so different local discriminative regions are covered.

What would settle it

Train the same student on synthetic sets produced with the two constraints ablated versus enabled at fixed IPC on CUB-200-2011, FGVC-Aircraft, and Stanford Cars; if Top-1 accuracy and measured intra-class compactness / inter-class center distance do not improve when the constraints are on, the central claim fails.

Watch

Extended reading notes

Core claim

The paper establishes that the quality bottleneck for decoupled distillation on fine-grained data is not storage budget alone but the absence of fine-grained structure in the synthetic set, and that this structure can be restored by supervising synthesis with CAL-derived class prototypes and attention maps: a fine-grained characteristic constraint that improves intra-class compactness and inter-class separability, plus a similarity constraint that diversifies same-class attention, yielding stronger student recognition without changing the three-stage decoupled workflow.

Load-bearing premise

That the class prototypes and attention maps built by the dual-branch teacher during pretraining remain faithful supervisors for sample synthesis, so optimizing those two normalized losses actually helps a plain student recognize fine-grained classes rather than only reshaping geometry in teacher space.

Editorial extensions

If this is right

  • Existing decoupled distillers can gain fine-grained accuracy by adding the two losses without redesigning pretraining, synthesis, or soft-label stages.
  • When the budget is one image per class, explicit prototype alignment and attention diversity supply the missing discriminative cues that coarse class labels do not.
  • Distilled sets that are more compact within class and more diverse in attended parts transfer better across student architectures in most reported cases.
  • The same plug-in structure can be applied on general subsets such as ImageNette and ImageWoof, with larger gains for higher-capacity students.
  • Cropping real local patches alone is not a reliable substitute for learned localization when building compact fine-grained training sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If prototypes are the load-bearing signal, any cheaper part localizer that still yields stable class centers might replace counterfactual attention without changing the two constraints.
  • The same diversity-plus-prototype idea could be tested on other part-sensitive domains (medical subtypes, material defects) where class labels are coarse relative to local evidence.
  • When gains shrink for small students on easy data, capacity may limit use of the extra local cues, suggesting a student-aware weighting of the constraints rather than a fixed λ.
  • Group size saturation at larger NS implies a practical upper bound on how many diverse local views one class can usefully pack under a fixed IPC.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes FD², a plug-in framework for fine-grained dataset distillation under the decoupled paradigm (pretraining, sample synthesis, soft-label generation). It augments a Backbone+CAL teacher so that counterfactual attention learning maintains class prototypes and attention maps; during sample-wise synthesis it adds a fine-grained characteristic constraint (Eq. 9) that pulls each distilled sample toward its class prototype and away from others, and a similarity constraint (Eq. 10) that diversifies attention maps within same-class groups of size NS. Soft labels are taken from the backbone branch only. Empirically, integrating FD² into SRe2L++ and FADRM+ improves post-evaluation Top-1 accuracy in most settings on CUB-200-2011, FGVC-Aircraft, Stanford Cars, ImageNette, and ImageWoof, with large gains at IPC=1 on Cars, plus cross-architecture checks, constraint ablations, t-SNE/attention visualizations, and an appendix theoretical sketch linking the losses to compactness/separability and attention diversity.

Significance. If the empirical gains hold under broader scrutiny, FD² is a practical contribution: it targets a real failure mode of decoupled DD on fine-grained data (coarse class supervision and within-class homogenization) without redesigning the host pipeline, and reports consistent improvements across three fine-grained benchmarks and two hosts, with especially large IPC=1 gains on Stanford Cars. Strengths include multi-dataset tables, constraint ablations (Tab. 7), cross-architecture transfer (Tab. 3), efficiency comparison in the appendix, and promised code. The work is incremental relative to SRe2L++/FADRM+ and CAL, but the plug-in design and fine-grained focus make it useful for the DD community.

major comments (2)
  1. §3.3–3.4 and Eqs. (8)–(10): the central premise is that dual-branch CAL prototypes and attention maps remain faithful supervisors for sample synthesis, so that minimizing the normalized ℓ2 losses improves student recognition rather than only reshaping teacher-space geometry. Support is mostly indirect (post-eval accuracy, t-SNE, heatmaps). A load-bearing check is missing: e.g., freeze the host objective and ablate whether student gains track measured changes in prototype margin / attention diversity, or report student performance when prototypes/attention come from a non-CAL teacher. Without this, the mechanism claim is under-supported even if the accuracy tables stand.
  2. Tables 1–2: gains are uneven—large on Stanford Cars and many SRe2L++ cells, but small or negative on several FADRM+/CUB and ImageNette ResNet18 entries (e.g., ImageNette IPC=1 ResNet18 ↓0.6). The abstract/conclusion claim of improvement “in most settings” and “strong transferability” is directionally true but needs a clearer failure analysis: when does FD² help (IPC, host strength, student capacity, fine-grained difficulty), and when does it not? A short discussion or stratified summary would make the central empirical claim more precise and less overstated.
minor comments (6)
  1. Notation: the abstract and title use FD² / FD$^2$, while the body often writes FD2; unify. Also idIPC vs. IPC and NS vs. N_S should be consistent.
  2. Fig. 1a caption and text: “fine-grained characteristics” is used both for the unfavorable original-data property and for the proposed constraint; a short definition early in §1 would reduce ambiguity.
  3. Eq. (9)–(10): the symmetrically normalized ℓ2 metric is reasonable but not compared to cosine or unnormalized Euclidean; a one-line ablation or justification would help.
  4. Appendix theoretical analysis is a useful post-hoc sketch (Propositions 1–2, Corollaries) but is not needed for the empirical claim; if retained, state clearly that it is interpretive rather than a formal guarantee of student accuracy.
  5. Implementation: Tab. 3 (appendix) shows CAL hyperparameters vary by architecture/dataset; the main text should point readers to this table so the dual-branch pretraining is reproducible without hunting the appendix.
  6. Minor typos: “Standford Cars” in appendix figure captions; “SRe2++” vs. SRe2L++ in a few places; “Code will be released” vs. abstract GitHub link—align.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical plug-in losses with post-hoc rearrangements, not predictions forced by definition or self-citation.

full rationale

FD² is an empirical method paper. Its load-bearing claim is that adding two plug-in losses (fine-grained characteristic constraint LF and similarity constraint LS) to decoupled hosts (SRe2L++, FADRM+) improves student Top-1 in most settings. That claim is supported by multi-dataset tables, ablations, and visualizations, not by a derivation that reduces to fitted constants or self-referential identities. Hyperparameters (α, β, λ, NS) are selected by ordinary ablation on held-out accuracy. The appendix propositions/corollaries merely rearrange the designed losses (e.g., LF ≡ constant − M(β), so min LF ⇔ max M) under standard discriminative-margin assumptions; this is post-hoc justification of loss design, not a circular prediction. CAL is cited from Rao et al. (external). No uniqueness theorem, no fitted input renamed as prediction, and no self-citation chain that forces the central empirical result. Score 0 is appropriate.

Assumptions & free parameters 6 free parameters · 4 assumptions · 3 invented entities

Central claims rest on standard deep-learning practice plus several hand-chosen hyperparameters and the modeling assumption that CAL prototypes/attention are good fine-grained supervisors. No new physical entities; the “invented” pieces are algorithmic constructs.

free parameters (6)
  • CAL ratio α (pretrain & distill) = 0.3–0.5 (dataset/model dependent)
    Balances backbone CE vs CAL loss; selected by ablation (Fig. 4, Tab. 4), typically 0.3–0.5.
  • prototype alignment weight β = 0.5
    Trades same-class pull vs other-class push in LF; ablated, best at 0.5 (Tab. 5).
  • constraint mix λ = 0.8
    Relative weight of LF vs LS; ablated, best at 0.8 (Appendix Tab. 4).
  • group size NS = 4
    Number of same-class samples over which attention diversity is enforced; ablated, best at 4 (Tab. 6).
  • number of attention maps M = 8–32
    CAL hyperparameter chosen per backbone/dataset (Appendix Tab. 3), e.g. 8 for ResNet18 on CUB.
  • prototype momentum µ and center weight η
    Online prototype update and center regularizer strength inside CAL pretraining (Eqs. 5–6); values inherited/tuned but not exhaustively reported as free of data.
assumptions (4)
  • domain assumption Models trained on a small synthetic set D can approach the risk of models trained on the full set T when the distillation objective is sufficiently matched (standard DD premise, Eqs. 1–2).
    Foundational assumption of the entire DD literature; invoked in §3.1.
  • domain assumption CAL factual–counterfactual attention yields attention maps and class prototypes that localize the discriminative parts needed for fine-grained recognition.
    Imported from Rao et al. 2021 and used as the teacher signal throughout §3.3–3.4.
  • ad hoc to paper Normalized Euclidean distances in feature/attention space (ℓ2) are adequate surrogates for intra-class compactness, inter-class separability, and attention diversity.
    Defines LF and LS (Eqs. 9–10); justified only by the appendix propositions and empirical gains.
  • ad hoc to paper Soft labels should be taken from the backbone branch only, to avoid architectural mismatch with the plain student used at evaluation.
    Design choice stated in §3.2–3.3; not independently validated beyond the reported protocol.
invented entities (3)
  • Fine-grained characteristic constraint LF
    purpose: Pull distilled sample features toward the target-class CAL prototype and push them from other-class prototypes.
    New loss term (Eq. 9) introduced by the paper; independent evidence is only the ablation accuracy lift.
  • Similarity constraint LS
    purpose: Maximize attention-map distance among sequentially generated same-class distilled samples.
    New loss term (Eq. 10); evidence is attention heatmaps and modest accuracy gains when enabled.
  • FD² group-wise synthesis schedule with size NS
    purpose: Apply diversity only inside small same-class groups so later samples do not collapse to weak regions.
    Procedural construct in Algorithm 1 and Appendix; no external falsifiable prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation." pith.science (2026). https://pith.science/paper/PPMQZ4CF

@misc{pith2026260325144,
  author       = {Pith},
  title        = {Pith review of: FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PPMQZ4CF}},
  note         = {Machine review of arXiv:2603.25144}
}
abstract

Dataset distillation (DD) compresses a large training set into a small synthetic set, reducing storage and training cost, and has shown strong results on general benchmarks. Decoupled DD further improves efficiency by splitting the pipeline into pretraining, sample distillation, and soft-label generation. However, existing decoupled methods largely rely on coarse class-label supervision and optimize samples within each class in a nearly identical manner. On fine-grained datasets, this often yields distilled samples that (i) retain large intra-class variation with subtle inter-class differences and (ii) become overly similar within the same class, limiting localized discriminative cues and hurting recognition. To solve the above-mentioned problems, we propose FD$^{2}$, a dedicated framework for Fine-grained Dataset Distillation. FD$^{2}$ localizes discriminative regions and constructs fine-grained representations for distillation. During pretraining, counterfactual attention learning aggregates discriminative representations to update class prototypes. During distillation, a fine-grained characteristic constraint aligns each sample with its class prototype while repelling others, and a similarity constraint diversifies attention across same-class samples. Experiments on multiple fine-grained and general datasets show that FD$^{2}$ integrates seamlessly with decoupled DD and improves performance in most settings, indicating strong transferability. Code is available at https://github.com/Guang000/FD2.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Models

    cs.CV 2026-05 conditional novelty 7.0 of 10

    CLP-DD distills small synthetic datasets for linear probing on pre-trained models via closed-form inner solver and discriminative outer loss, matching or exceeding LGM+DSA performance at much lower cost on ImageNet-10...

Reference graph

Works this paper leans on

65 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR)

    Cazenavette, G., Wang, T., Torralba, A., Efros, A.A., Zhu, J.Y.: Dataset distil- lation by matching training trajectories. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR). pp. 10718–10727 (2022) 2, 4

    2022

  2. [2]

    arXiv preprint arXiv:2312.08912 (2023) 4

    Chen, M., Huang, B., Lu, J., Li, B., Wang, Y., Cheng, M., Wang, W.: Dataset distillation via adversarial prediction matching. arXiv preprint arXiv:2312.08912 (2023) 4

    arXiv 2023

  3. [3]

    In: Pro- ceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) (2010) 22

    Chen, Y., Welling, M., Smola, A.: Super-samples from kernel herding. In: Pro- ceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) (2010) 22

    2010

  4. [4]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2025) 2, 4, 9

    Cui, J., Bi, X., Luo, Y., Zhao, X., Liu, J., Shen, Z.: FADRM: Fast and accurate data residual matching for dataset distillation. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2025) 2, 4, 9

    2025

  5. [5]

    arXiv preprint arXiv:2501.07575 (2025) 2, 3, 4, 8

    Cui, J., Li, Z., Ma, X., Bi, X., Luo, Y., Shen, Z.: Dataset distillation via committee voting. arXiv preprint arXiv:2501.07575 (2025) 2, 3, 4, 8

    arXiv 2025

  6. [6]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2022) 2

    Cui, J., Wang, R., Si, S., Hsieh, C.J.: DC-BENCH: Dataset condensation bench- mark. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2022) 2

    2022

  7. [7]

    In: Proceedings of the International Conference on Machine Learning (ICML)

    Cui, J., Wang, R., Si, S., Hsieh, C.J.: Scaling up dataset distillation to imagenet-1k with constant memory. In: Proceedings of the International Conference on Machine Learning (ICML). pp. 6565–6590 (2023) 2, 4

    2023

  8. [8]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 248–255 (2009) 1, 2, 8

    2009

Show all 65 references
  1. [9]

    In: Proceedings of the International Conference on Learning Representations (ICLR) (2021) 1

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: Proceedings of the Internatio...

    2021

  2. [10]

    In: Proceedings of the European Conference on Computer Vision (ECCV)

    Du, R., Chang, D., Bhunia, A.K., Xie, J., Ma, Z., Song, Y.Z., Guo, J.: Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 153– 168 (2020) 4

    2020

  3. [11]

    Annals of eugenics pp

    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of eugenics pp. 179–188 (1936) 19 16 H. Ma and G. Li et al

    1936

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Gidaris, S., Komodakis, N.: Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4367–4375 (2018) 22

    2018

  5. [13]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Gu, J., Vahidian, S., Kungurtsev, V., Wang, H., Jiang, W., You, Y., Chen, Y.: Effi- cient dataset distillation via minimax diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 15793– 15803 (2024) 2, 4, 22

    2024

  6. [14]

    In: Proceedings of the International Conference on Learning Representations (ICLR) (2024) 4, 22

    Guo, Z., Wang, K., Cazenavette, G., Li, H., Zhang, K., You, Y.: Towards lossless dataset distillation via difficulty-aligned trajectory matching. In: Proceedings of the International Conference on Learning Representations (ICLR) (2024) 4, 22

    2024

  7. [15]

    He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778 (2016) 1, 12, 22

    2016

  8. [16]

    ArXiv (2019) 20

    Hu, T., Qi, H.: See better before looking closer: Weakly supervised data augmen- tation network for fine-grained visual classification. ArXiv (2019) 20

    2019

  9. [17]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4700–4708 (2017) 12

    2017

  10. [18]

    In: Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW)

    Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine- grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW). pp. 554–561 (2013) 8

    2013

  11. [19]

    In: Proceedings of the International Conference on Machine Learn- ing (ICML)

    Lee, S., Chun, S., Jung, S., Yun, S., Yoon, S.: Dataset condensation with con- trastive signals. In: Proceedings of the International Conference on Machine Learn- ing (ICML). pp. 12352–12364 (2022) 4

    2022

  12. [20]

    In: Proceedings of the IEEE International Conference on Image Processing (ICIP)

    Li, G., Togo, R., Ogawa, T., Haseyama, M.: Soft-label anonymous gastric x-ray image distillation. In: Proceedings of the IEEE International Conference on Image Processing (ICIP). pp. 305–309 (2020) 2

    2020

  13. [21]

    Li, G., Togo, R., Ogawa, T., Haseyama, M.: Compressed gastric image generation basedonsoft-labeldatasetdistillationformedicaldatasharing.ComputerMethods and Programs in Biomedicine227, 107189 (2022) 2

    2022

  14. [22]

    Neural Networks172, 106154 (2024) 2

    Li, G., Togo, R., Ogawa, T., Haseyama, M.: Importance-aware adaptive dataset distillation. Neural Networks172, 106154 (2024) 2

    2024

  15. [23]

    https://github.com/Guang000/Awesome-Dataset-Distillation (2022) 2

    Li, G., Zhao, B., Wang, T.: Awesome dataset distillation. https://github.com/Guang000/Awesome-Dataset-Distillation (2022) 2

    2022

  16. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Workshop

    Li, L., Li, G., Togo, R., Maeda, K., Ogawa, T., Haseyama, M.: Generative Dataset Distillation: Balancing global structure and local details. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Workshop. pp. 7664–7671 (2024) 2

    2024

  17. [25]

    In: IEEE International Conference on Image Processing (ICIP) (2024) 4

    Li, M., Li, G., Mao, J., Ogawa, T., Haseyama, M.: Diversity-driven generative dataset distillation based on diffusion model with self-adaptive memory. In: IEEE International Conference on Image Processing (ICIP) (2024) 4

    2024

  18. [26]

    In: IEEE/CVF International Conference on Computer Vision (ICCV), Workshop (2025) 2

    Li, M., Li, G., Mao, J., Ye, L., Ogawa, T., Haseyama, M.: Task-specific generative dataset distillation with difficulty-guided sampling. In: IEEE/CVF International Conference on Computer Vision (ICCV), Workshop (2025) 2

    2025

  19. [27]

    arXiv preprint arXiv:2511.17890 (2025) 4

    Li, W., Li, G., Maeda, K., Ogawa, T., Haseyama, M.: Decoupled audio-visual dataset distillation. arXiv preprint arXiv:2511.17890 (2025) 4

    2025

  20. [28]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2025) 2 Fine-Grained Dataset Distillation 17

    Li, W., Li, G., Maeda, K., Ogawa, T., Haseyama, M.: Hyperbolic dataset distilla- tion. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2025) 2 Fine-Grained Dataset Distillation 17

    2025

  21. [29]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2022) 4

    Liu, S., Wang, K., Yang, X., Ye, J., Wang, X.: Dataset distillation via factoriza- tion. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2022) 4

    2022

  22. [30]

    In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV)

    Liu, Y., Gu, J., Wang, K., Zhu, Z., Jiang, W., You, Y.: DREAM: Efficient dataset distillation by representative matching. In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV). pp. 17314–17324 (2023) 4

    2023

  23. [31]

    In: Proceedings of the IEEE/CVF international conference on computer vision (CVPR)

    Luo, W., Yang, X., Mo, X., Lu, Y., Davis, L.S., Li, J., Yang, J., Lim, S.N.: Cross-x learning for fine-grained visual categorization. In: Proceedings of the IEEE/CVF international conference on computer vision (CVPR). pp. 8242–8251 (2019) 4

    2019

  24. [32]

    Journal of Machine Learning Research pp

    van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of Machine Learning Research pp. 2579–2605 (2008) 11

    2008

  25. [33]

    arXiv preprint arXiv:1306.5151 (2013) 8

    Maji, S., Rahtu, E., Kannala, J., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013) 8

    2013 arXiv

  26. [34]

    In: Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). p. 4195–4205 (2023) 22

    2023

  27. [35]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Rao, Y., Chen, G., Lu, J., Zhou, J.: Counterfactual attention learning for fine-grained visual categorization and re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 1025–1034 (2021) 4, 6

    2021

  28. [36]

    Sajedi, A., Khaki, S., Amjadian, E., Liu, L.Z., Lawryshyn, Y.A., Plataniotis, K.N.: DataDAM:Efficientdatasetdistillationwithattentionmatching.In:Proceedingsof the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 17097– 17107 (2023) 4

    2023

  29. [37]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: In- verted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4510–4520 (2018) 12

    2018

  30. [38]

    In: Proceedings of the International Conference on Learning Repre- sentations (ICLR) (2018) 22

    Sener, O., Savarese, S.: Active learning for convolutional neural networks: A core- set approach. In: Proceedings of the International Conference on Learning Repre- sentations (ICLR) (2018) 22

    2018

  31. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Shao, S., Yin, Z., Zhou, M., Zhang, X., Shen, Z.: Generalized large-scale data condensation via various backbone and statistical matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 16709–16718 (2024) 4

    2024

  32. [40]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2024) 4

    Shao, S., Zhou, Z., Chen, H., Shen, Z.: Elucidating the design space of dataset condensation. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2024) 4

    2024

  33. [41]

    In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR)

    Su, D., Hou, J., Gao, W., Tian, Y., Tang, B.: D4M: Dataset distillation via dis- entangled diffusion model. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR). pp. 5809–5818 (2024) 4

    2024

  34. [42]

    In: Proceedings of the European Conference on Computer Vision (ECCV), Workshop (2024) 2

    Su, D., Hou, J., Li, G., Togo, R., Song, R., Ogawa, T., Haseyama, M.: Generative dataset distillation based on diffusion model. In: Proceedings of the European Conference on Computer Vision (ECCV), Workshop (2024) 2

    2024

  35. [43]

    In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR)

    Sun,P.,Shi,B.,Yu,D.,Lin,T.:Onthediversityandrealismofdistilleddataset:An efficient dataset distillation paradigm. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR). pp. 9390–9399 (2024) 9

    2024

  36. [44]

    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: Caltech-ucsd birds- 200-2011. Tech. rep., California Institute of Technology (2011) 8 18 H. Ma and G. Li et al

    2011

  37. [45]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Wang, K., Zhao, B., Peng, X., Zhu, Z., Yang, S., Wang, S., Huang, G., Bilen, H., Wang, X., You, Y.: CAFE: Learning to condense dataset by aligning features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 12196–12205 (2022) 2, 4

    2022

  38. [46]

    In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)

    Wang, S., Li, H., Wang, Z., Ouyang, W.: Dynamic position-aware network for fine- grained image recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). pp. 2791–2799 (2021) 4

    2021

  39. [47]

    In: Proceedings of the ACM Interna- tional Conference on Multimedia (ACM MM)

    Wang, S., Wang, Z., Li, H., Ouyang, W.: Category-specific semantic coherency learning for fine-grained image recognition. In: Proceedings of the ACM Interna- tional Conference on Multimedia (ACM MM). pp. 174–183 (2020) 4

    2020

  40. [48]

    arXiv preprint arXiv:1811.10959 (2018) 2

    Wang, T., Zhu, J.Y., Torralba, A., Efros, A.A.: Dataset distillation. arXiv preprint arXiv:1811.10959 (2018) 2

    2018 arXiv

  41. [49]

    In: Proceedings of the European Conference on Computer Vision (ECCV)

    Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 499–515 (2016) 19

    2016

  42. [50]

    Transactions on Machine Learning Research (2025) 4

    Wu, H., Su, D., Hou, J., Li, G.: Dataset condensation with color compensation. Transactions on Machine Learning Research (2025) 4

    2025

  43. [51]

    Xiao, L., He, Y.: Are large-scale soft labels necessary for large-scale dataset distil- lation? In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2024) 4

    2024

  44. [52]

    In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2025) 4

    Xue, E., Li, Y., Liu, H., Wang, P., Shen, Y., Wang, H.: Towards adversarially robust dataset distillation by curvature regularization. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2025) 4

    2025

  45. [53]

    In: Advances in Neu- ral Information Processing Systems (NeurIPS), Workshop (2025) 4

    Ye, L., Hamidi, S.M., Li, G., Ogawa, T., Haseyama, M., Plataniotis, K.N.: Information-guided diffusion sampling for dataset distillation. In: Advances in Neu- ral Information Processing Systems (NeurIPS), Workshop (2025) 4

    2025

  46. [54]

    Transactions on Machine Learning Research (2024) 4

    Yin, Z., Shen, Z.: Dataset distillation in large data era. Transactions on Machine Learning Research (2024) 4

    2024

  47. [55]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2023) 2, 4, 8

    Yin, Z., Xing, E., Shen, Z.: Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2023) 2, 4, 8

    2023

  48. [56]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: An extremely efficient convolu- tional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6848–6856 (2018) 12

    2018

  49. [57]

    In:Proceedings ofthe International Conference onMachine Learning(ICML)

    Zhao, B., Bilen, H.: Dataset condensation with differentiable siamese augmenta- tion. In:Proceedings ofthe International Conference onMachine Learning(ICML). pp. 12674–12685 (2021) 2, 4, 22

    2021

  50. [58]

    In: Proceedings of the International Conference on Learning Representations (ICLR) (2021) 2, 4

    Zhao, B., Bilen, H.: Dataset condensation with gradient matching. In: Proceedings of the International Conference on Learning Representations (ICLR) (2021) 2, 4

    2021

  51. [59]

    In: Pro- ceedings of the Advances in Neural Information Processing Systems (NeurIPS), Workshop (2022) 2, 4

    Zhao, B., Bilen, H.: Synthesizing informative training samples with gan. In: Pro- ceedings of the Advances in Neural Information Processing Systems (NeurIPS), Workshop (2022) 2, 4

    2022

  52. [60]

    In: Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    Zhao, B., Bilen, H.: Dataset condensation with distribution matching. In: Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 6514–6523 (2023) 2, 4, 22

    2023

  53. [61]

    In: Proceedings of the Computer Vision and Pattern Recogni- tion Conference (CVPR)

    Zhong, X., Fang, H., Chen, B., Gu, X., Qiu, M., Qi, S., Xia, S.T.: Hierarchical features matter: A deep exploration of progressive parameterization method for dataset distillation. In: Proceedings of the Computer Vision and Pattern Recogni- tion Conference (CVPR). pp. 30462–30...

    2025

  54. [62]

    arXiv preprint arXiv:2402.13007 (2024) 4 Fine-Grained Dataset Distillation 19 Appendix Sec

    Zhou, B., Zhong, L., Chen, W.: Improve cross-architecture generalization on dataset distillation. arXiv preprint arXiv:2402.13007 (2024) 4 Fine-Grained Dataset Distillation 19 Appendix Sec. 1 analyzes the effectiveness of FD2; Sec. 2 evaluates fine-grained datasets with more d...

    2024 arXiv

  55. [63]

    Moreover, by Eq.(16), the diversity of the resulting representations is also limited

    (17) Hence, if same-class initialization images are close to each other, the resulting attention maps tend to focus on similar regions. Moreover, by Eq.(16), the diversity of the resulting representations is also limited. Proof.From Eq. (12), ∥Ay,i −A y,j ∥2 2 ≤L 2 y∥x(0) y,i ...

  56. [64]

    (18) into Eq

    (18) Substituting Eq. (18) into Eq. (14) gives Eq. (17). Then Eq. (16) shows that smallertr(Σ (y) A )also restrictstr(Σ (y) h ).□ To counteract this tendency, we use the similarity constraint LS(˜xy,i) = 1−E j<i[ℓ2(Ay,i, Ay,j)],1< i≤N S, (19) which serves as a penalty against ...

  57. [65]

    (23) and (24) yields tr Σ(y) A ≥ i−1 N2 S η 2 (Ej<i[ℓ2(Ay,i, Ay,j)])2

    (24) Combining Eqs. (23) and (24) yields tr Σ(y) A ≥ i−1 N2 S η 2 (Ej<i[ℓ2(Ay,i, Ay,j)])2 . (25) Thus, minimizingL S(˜xy,i)enlarges a lower bound on intra-class attention di- versity. By Eq. (16), this further promotes more diverse representations.□ 2 More Results and Discussi...

    2011

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.