Delta-SVD: Efficient Compression for Personalized Text-to-Image Models

Jianfei Cai; Qixiang Chen; Shangyu Chen; Tangyuan Zhang

REVIEW 4 major objections 5 minor 18 references

Delta-SVD: Efficient Compression for Personalized Text-to-Image Models

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Delta-SVD claims DreamBooth fine-tune updates are low-rank enough that SVD truncation compresses a personalized text-to-image model by roughly 270x while keeping generation quality near the full fine-tune.

desk verdict A simple, honest post-hoc SVD compression for DreamBooth deltas that works about as well as claimed, with a real but acknowledged limitation around the global energy threshold. read the letter →

arxiv 2508.16863 v1 pith:Y65FTXDE submitted 2025-08-23 cs.CV

Tangyuan Zhang , Shangyu Chen , Qixiang Chen , Jianfei Cai This is my paper

classification cs.CV

keywords modelcompressiondiffusionmodelsDreamBoothpersonalizedtext-to-imagegenerationlow-rankapproximationsingularvaluedecompositionweightdeltasStable

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that a DreamBooth-personalized Stable Diffusion model can be compressed after training, without retraining or access to the original data, by applying singular value decomposition to the difference between the fine-tuned weights and the base weights. The central claim is that personalization touches the UNet only lightly, so these delta weights are sparse and strongly low-rank. The method keeps a small number of dominant singular directions per layer, chosen by a cumulative energy threshold, and stores the two low-rank factors instead of the full delta. On 30 DreamBooth subjects, an energy threshold of 0.5 gives a CLIP score of 0.835 versus 0.833 for the fully fine-tuned model while shrinking the UNet to 353.8 MB, and a threshold of 0.06 gives 12.1 MB with reasonable quality. If correct, this makes storing and serving many subject-specific models practical, even for released checkpoints where no training data or scripts survive.

What carries the argument

The central object is the weight-delta matrix ΔW^(l) between each fine-tuned UNet layer and the corresponding pretrained layer. The mechanism is layer-wise SVD with an energy-based rank-selection rule: the cumulative energy function E(t) decides how many singular components each layer keeps, so layers with rich updates retain higher rank while nearly unchanged layers are aggressively compressed. Storing the two factors (A, B) per layer on top of the frozen base model makes the method plug-and-play: no architectural change, no retraining, and on-the-fly reconstruction during inference.

What would settle it

Take a DreamBooth checkpoint whose weight updates are large and spread out, such as a style or multi-concept fine-tune, and plot the cumulative singular-energy curve of its deltas. If at τ = 0.06 the compressed model's CLIP score drops by more than the subject-to-subject variance of the full model, or if no single τ works well for both identity-heavy and style-heavy subjects, the universal low-rank and global-threshold claim fails.

Watch

Extended reading notes

Core claim

The paper's core discovery is that the parameter change induced by DreamBooth tuning, ΔW = W_finetuned − W_base, is low-rank enough that a layer-wise truncated SVD can reconstruct the personalization effect almost exactly. Rather than compressing the full model, Delta-SVD factorizes each delta into UΣV^T, defines a cumulative energy function E(t) as the share of total singular-value energy captured by the top t components, and picks the smallest rank whose energy reaches a chosen threshold τ. It stores A = U_t Σ_t and B = V_t^T per layer, then at inference simply adds the product back onto the base weights. At τ = 0.5 the compressed model matches or slightly exceeds the fully fine-tuned CLIP

Load-bearing premise

A single manually chosen global energy threshold τ, applied identically to every layer, preserves generation quality across all subjects; if τ must be tuned per subject, the plug-and-play deployment claim breaks.

Editorial extensions

If this is right

Any released DreamBooth checkpoint can be compressed after the fact, even when training data or training scripts are unavailable.
At 0.5 energy retention, storage drops from 3,276.8 MB to 353.8 MB with CLIP score 0.835 versus 0.833 for the full model.
At 0.06 energy retention, storage drops to 12.1 MB, about 270x smaller, with reasonable generation quality that still beats LoRA and 8-bit quantization in the reported comparison.
Because the architecture is unchanged, the compressed deltas slot directly into existing Stable Diffusion inference pipelines without additional code or retraining.
Since only the UNet is compressed and the text encoder and VAE are left untouched, the savings target the main storage driver in personalized text-to-image models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same delta-SVD recipe likely applies to any fine-tuning that moves weights within a low-dimensional subspace, not just DreamBooth; a testable extension is compressing LoRA-merged or style-tuned checkpoints and checking whether their singular-energy spectra remain concentrated.
The strong results at a 0.06 energy threshold suggest that per-layer rank allocation could go further—for example, leaving mid-block attention layers nearly uncompressed while crushing convolutional layers—potentially pushing storage below 10 MB without visible loss.
Because reconstruction merely adds two low-rank factors back to a shared base model, Delta-SVD could be combined with quantization of A and B or with serving many subjects from one common base checkpoint, turning each subject's cost into just its factor pair.
The global manual threshold is the fragile point; an automatic per-subject or per-layer threshold selected by validation metrics would turn the method from a heuristic into a deployable system.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper does exactly what it says: layer-wise SVD on DreamBooth weight deltas, keep top singular components by cumulative energy, store low-rank factors. The key empirical claim—that DreamBooth deltas are strongly low-rank—is plausible and supported by the experiments: at 50% energy retention, CLIP and SSIM are indistinguishable from the full model, and even at 270x compression the quality beats LoRA and 8-bit quantization. That is a genuinely useful, though modest, result for anyone hosting many checkpoints.

What the paper gets right: it is honest about scope. The Limitations section admits the threshold may not generalize across subjects and notes there is no inference speedup. The method is a direct composition of known tools—SVD truncation of fine-tuning deltas is not new in other domains—but the post-hoc, training-free application to DreamBooth is not in the cited literature. That counts as a new result.

The soft spots: - The single global energy threshold tau is a free parameter. The paper reports only averaged metrics, never per-subject variance in the tradeoff. If tau must be tuned per subject, the "plug-and-play" claim shrinks. The authors know this, and it is in the Limitations, but a reviewer should demand per-subject degradation curves. - The SSIM jump at tau=0.06 (0.208 to 0.268) is unexplained. Higher SSIM against ground truth under more aggressive compression is odd; it could be a metric artifact or a collapse to generic prior. Needs an explanation, not just a row in the table. - No code or detailed evaluation scripts, and no comparison against plain fixed-rank SVD per layer. The energy threshold is a heuristic; the paper should show it beats a fixed-rank baseline by more than noise. - The baselines are not apples-to-apples: LoRA is a training-time method, quantization preserves the full architecture. That is fine, but the paper should be clearer about the different tradeoffs.

Verdict: this is a legitimate incremental paper. It deserves serious peer review because the claim is testable, the method is simple, and the practical use case is real. I would push for code release and per-subject analysis before acceptance. Who it is for: practitioners and researchers in efficient serving of personalized diffusion models, or anyone studying the intrinsic dimensionality of fine-tuning. I would cite it if I worked in that area, though I wouldn't build a research program on it.

Referee Report

4 major / 5 minor

Summary. The paper proposes Delta-SVD, a post-hoc, training-free compression method for DreamBooth-finetuned Stable Diffusion UNets. It computes per-layer delta weights ΔW = W_ft − W_pre, applies SVD to each delta, and truncates based on a global cumulative-energy threshold τ (Eq. 2–3). The low-rank factors A = U_tΣ_t and B = V_t^T are stored, and at inference the delta is reconstructed as AB and added to the base weights. Experiments on the DreamBooth dataset (30 subjects) report CLIP score, SSIM, and FID at energy thresholds τ = 0.8, 0.5, 0.2, 0.06. The authors claim up to a 270× storage reduction (12.1 MB, Ours@0.06) with quality close to the full model, and better than LoRA (rank 128) and 8-bit quantization.

Significance. The paper addresses a practical and increasingly relevant problem: compressing already-released DreamBooth checkpoints without access to training data or retraining. The method is simple, architecture-preserving, and the low-rank structure of delta weights is a plausible empirical observation. If the generality of the global energy threshold were established and the storage arithmetic clarified, this would be a useful contribution to efficient personalized text-to-image deployment. However, the current evidence is insufficient to support the headline plug-and-play claim, and one reported metric anomaly (SSIM at extreme compression) needs explanation. The paper would benefit from additional ablations and per-subject robustness analysis.

major comments (4)

[§5 Limitations; §3.2, Algorithm 1; Table 1] The central deployment claim is that Delta-SVD is fully plug-and-play and can compress any released DreamBooth checkpoint without per-subject tuning. This rests on a single global energy threshold τ applied to all layers and all subjects. Table 1 reports only dataset-averaged metrics at τ = 0.06, 0.2, 0.5, 0.8; no per-subject breakdown is given. The Limitations section explicitly states: 'Delta-SVD relies on a manually selected compression threshold, which may not generalize across subjects.' This admission directly undermines the plug-and-play claim. If τ must be tuned per subject, the method reverts to a manual workflow, and the 270×/12.1MB result (Ours@0.06) is not a general property of DreamBooth deltas. Please report per-subject metric degradation at each τ, or propose and validate an automatic/adaptive threshold.
[§4.3, Table 1] At Ours@0.06, SSIM (0.268 ± 0.110) is higher than the full DreamBooth model (0.208 ± 0.14) and higher than Ours@0.8 (0.208), while FID degrades from 157.7 to 181.3. This non-monotonic behavior is unexplained. It suggests that extreme truncation changes the output distribution in a way that inflates SSIM (e.g., smoothing or loss of high-frequency detail), and therefore the claim that Ours@0.06 has 'reasonable generation quality' is not supported by the reported metrics. The authors should explain this anomaly or provide per-subject qualitative and quantitative evidence.
[§4.3 and §3.2] The method's rank-selection component is the energy-based threshold in Eq. (3), motivated by the claim that fixed uniform truncation over-compresses layers with rich updates and under-compresses layers with negligible changes. However, no experiment compares Delta-SVD to a fixed-rank SVD truncation or fixed-proportion rank selection at equal total parameter budgets, nor is there a comparison to SVD applied directly to full fine-tuned weights rather than to the delta. Without these ablations, the specific contributions of the energy criterion and the delta formulation are not isolated; the reported gains could be attributable to SVD alone.
[§4.3, Table 1] The storage accounting is unclear. Ours@0.06 lists 8.01M parameters and a size of 12.1MB. At 16-bit precision, 8.01M parameters require about 16MB; at 32-bit, about 32MB. The reported 12.1MB implies additional quantization or a non-standard storage format that is not described. Because the 270× compression ratio is a headline result, the precision, the handling of the base model, and any metadata/overhead must be specified.

minor comments (5)

[Figure 1] The x-axis is not described and the figure appears to list layer names; please add axis labels and explain how cosine similarity was averaged over subjects.
[§4.5, Table 2] Reporting average remaining rank per layer group without standard deviations or per-subject ranges makes it hard to assess variability; also define the layer groups more precisely.
[§4.5, 'Compression performance on more complex style'] The text says 'outperforming the baseline as figure 4 shows,' but it is unclear what the baseline is. Please clarify the comparison.
[§4.1 and §4.3] Reproducibility details are missing: no code, no seeds, no number of evaluation images per subject, and no inference hyperparameters.
[General] Typos and formatting issues: 'as showed in Figure 2' should be 'as shown'; Table 1 cell '8.01 -99% 12.1' needs consistent formatting; 'anenergy-based' is missing a space; Eq. (1) should specify the full SVD dimensions when d and k are unequal.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Delta-SVD's compression quality is evaluated on independent metrics, and the manual energy threshold is an operating hyperparameter, not a fitted prediction.

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs. The key observation—that DreamBooth delta weights are low-rank—is an empirical claim supported by Figure 1 and by the measured ranks in Table 2, not a theorem derived from the method's own definitions. The compression method (Algorithm 1) computes delta weights, applies SVD, and truncates by an energy threshold τ; this procedure by construction controls retained energy, but the reported success metrics (CLIP score, SSIM, FID) are computed on generated images and are not defined in terms of the singular values or the energy retention. Therefore a good evaluation score cannot be enforced by the compression equation itself. The energy threshold τ is a manually chosen operating point, not a parameter fitted to the evaluation metrics and then renamed as a prediction; the paper presents results across multiple thresholds (0.8, 0.5, 0.2, 0.06) rather than a single fitted value. The Limitations section explicitly concedes that τ 'may not generalize across subjects,' which is a valid external-validity caveat but not a circular step. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation; all references are to external prior work. In short, the claimed compression quality is an empirical outcome, not a tautology, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical entities, particles, forces, or architectural components are introduced. Delta-SVD is a factorization of existing weights.

free parameters (1)

Energy threshold tau = 0.8, 0.5, 0.2, 0.06 in reported operating points
Chosen by hand; the paper's Limitations admit it 'relies on a manually selected compression threshold.' It determines the per-layer rank t in Eq. (3) and therefore the compression-quality trade-off. No principled rule is given for selecting tau.

assumptions (4)

domain assumption DreamBooth fine-tuning updates are sparse and low-rank.
Used to justify SVD compression. Evidence is qualitative: Figure 1 cosine similarity and observed singular-value decay, with no formal bound or statistical test.
ad hoc to paper Cumulative singular-value energy E(t) is a valid surrogate for generation quality.
Eq. (2) defines E(t) and Eq. (3) uses it to pick ranks. The paper does not establish a link between energy fraction and CLIP/SSIM/FID beyond the reported operating points.
ad hoc to paper A single global tau works across all layers and subjects.
Algorithm 1 applies the same threshold to every layer; the Limitations acknowledge the threshold 'may not generalize across subjects.'
domain assumption Base model weights W_pre are available to compute deltas.
Section 3.2 Eq. (1) defines delta as W_ft minus W_pre; if the base checkpoint is unavailable the method cannot be applied.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Delta-SVD: Efficient Compression for Personalized Text-to-Image Models." pith.science (2026). https://pith.science/paper/Y65FTXDE

@misc{pith2026250816863,
  author       = {Pith},
  title        = {Pith review of: Delta-SVD: Efficient Compression for Personalized Text-to-Image Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y65FTXDE}},
  note         = {Machine review of arXiv:2508.16863}
}

read the original abstract

Personalized text-to-image models such as DreamBooth require fine-tuning large-scale diffusion backbones, resulting in significant storage overhead when maintaining many subject-specific models. We present Delta-SVD, a post-hoc, training-free compression method that targets the parameter weights update induced by DreamBooth fine-tuning. Our key observation is that these delta weights exhibit strong low-rank structure due to the sparse and localized nature of personalization. Delta-SVD first applies Singular Value Decomposition (SVD) to factorize the weight deltas, followed by an energy-based rank truncation strategy to balance compression efficiency and reconstruction fidelity. The resulting compressed models are fully plug-and-play and can be re-constructed on-the-fly during inference. Notably, the proposed approach is simple, efficient, and preserves the original model architecture. Experiments on a multiple subject dataset demonstrate that Delta-SVD achieves substantial compression with negligible loss in generation quality measured by CLIP score, SSIM and FID. Our method enables scalable and efficient deployment of personalized diffusion models, making it a practical solution for real-world applications that require storing and deploying large-scale subject customizations.

Figures

Figures reproduced from arXiv: 2508.16863 by the authors.

**Figure 2.** Comparison of generation results across the fully fine-tuned model, LoRA [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Visualization of generated images under different compression ratios using [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Evaluation of image quality under different Delta-SVD compression [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 7 canonical work pages

[1]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Fang,G.,Ma,X.,Song,M.,Mi,M.B.,Wang,X.:Depgraph:Towardsanystructural pruning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16091–16101 (2023)

work page 2023
[2]

Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A.H., Chechik, G., Cohen-Or, D.: An image is worth one word: Personalizing text-to-image gener- ation using textual inversion (2022),https://arxiv.org/abs/2208.01618

arXiv 2022
[3]

org/abs/2106.09685

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models (2021),https://arxiv. org/abs/2106.09685

arXiv 2021
[4]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., Kalenichenko,D.:Quantizationandtrainingofneuralnetworksforefficientinteger- arithmetic-only inference. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2704–2713 (2018) 12 Anonymous

work page 2018
[5]

In: European Conference on Computer Vision

Kim, B.K., Song, H.K., Castells, T., Choi, S.: Bk-sdm: A lightweight, fast, and cheap version of stable diffusion. In: European Conference on Computer Vision. pp. 381–399. Springer (2024)

work page 2024
[6]

In: Proceedings of the 2016 conference on empirical methods in natural language processing

Kim, Y., Rush, A.M.: Sequence-level knowledge distillation. In: Proceedings of the 2016 conference on empirical methods in natural language processing. pp. 1317– 1327 (2016)

work page 2016
[7]

arXiv preprint arXiv:2006.07253 (2020)

Lin, T., Stich, S.U., Barba, L., Dmitriev, D., Jaggi, M.: Dynamic model pruning with feedback. arXiv preprint arXiv:2006.07253 (2020)

arXiv 2006
[8]

In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

Park, W., Kim, D., Lu, Y., Cho, M.: Relational knowledge distillation. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3967–3976 (2019)

2019

Show all 18 references

[9]

arXiv preprint arXiv:2307.01952 (2023)

Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023)

2023 arXiv
[10]

Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text- conditional image generation with clip latents (2022),https://arxiv.org/abs/ 2204.06125

2022 arXiv
[11]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)

2022
[12]

Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dream- booth:Finetuningtext-to-imagediffusionmodelsforsubject-drivengeneration.In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 22500–22510 (2023)

2023
[13]

Advances in neural information processing systems35, 36479–36494 (2022)

Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al.: Photorealistic text- to-image diffusion models with deep language understanding. Advances in neural information processing systems35,...

2022
[14]

Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models (2022),https: //arxiv.org/abs/2010.02502

2022 arXiv
[15]

arXiv preprint arXiv:1910.04732 (2019)

Wang, Z., Wohlwend, J., Lei, T.: Structured pruning of large language models. arXiv preprint arXiv:1910.04732 (2019)

1910 arXiv
[16]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Yang, J., Shen, X., Xing, J., Tian, X., Li, H., Deng, B., Huang, J., Hua, X.s.: Quantization networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 7308–7316 (2019)

2019
[17]

IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9), 5700–5714 (2021)

Young, S.I., Zhe, W., Taubman, D., Girod, B.: Transform quantization for cnn compression. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9), 5700–5714 (2021)

2021
[18]

In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition

Zhao, B., Cui, Q., Song, R., Qiu, Y., Liang, J.: Decoupled knowledge distillation. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition. pp. 11953–11962 (2022)

2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Fang,G.,Ma,X.,Song,M.,Mi,M.B.,Wang,X.:Depgraph:Towardsanystructural pruning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16091–16101 (2023)

work page 2023

[2] [2]

Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A.H., Chechik, G., Cohen-Or, D.: An image is worth one word: Personalizing text-to-image gener- ation using textual inversion (2022),https://arxiv.org/abs/2208.01618

arXiv 2022

[3] [3]

org/abs/2106.09685

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models (2021),https://arxiv. org/abs/2106.09685

arXiv 2021

[4] [4]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., Kalenichenko,D.:Quantizationandtrainingofneuralnetworksforefficientinteger- arithmetic-only inference. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2704–2713 (2018) 12 Anonymous

work page 2018

[5] [5]

In: European Conference on Computer Vision

Kim, B.K., Song, H.K., Castells, T., Choi, S.: Bk-sdm: A lightweight, fast, and cheap version of stable diffusion. In: European Conference on Computer Vision. pp. 381–399. Springer (2024)

work page 2024

[6] [6]

In: Proceedings of the 2016 conference on empirical methods in natural language processing

Kim, Y., Rush, A.M.: Sequence-level knowledge distillation. In: Proceedings of the 2016 conference on empirical methods in natural language processing. pp. 1317– 1327 (2016)

work page 2016

[7] [7]

arXiv preprint arXiv:2006.07253 (2020)

Lin, T., Stich, S.U., Barba, L., Dmitriev, D., Jaggi, M.: Dynamic model pruning with feedback. arXiv preprint arXiv:2006.07253 (2020)

arXiv 2006

[8] [8]

In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

Park, W., Kim, D., Lu, Y., Cho, M.: Relational knowledge distillation. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3967–3976 (2019)

2019