RECORD
Diffusion Model with Clustering-based Conditioning for Food Image Generation
Computer Vision and Pattern Recognition · Submitted 1 Sep 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim The proposed method is evaluated on the Food-101 dataset and shows improved performance when compared with existing image generation works.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Image-based dietary assessment serves as an efficient and accurate solution for recording and analyzing nutrition intake using eating occasion images as input. Deep learning-based techniques are commonly used to perform image analysis such as food classification, segmentation, and portion size estimation, which rely on large amounts of food images with annotations for training. However, such data dependency poses significant barriers to real-world applications, because acquiring a substantial, diverse, and balanced set of food images can be challenging. One potential solution is to use synthetic food images for data augmentation. Although existing work has explored the use of generative adversarial networks (GAN) based structures for generation, the quality of synthetic food images still remains subpar. In addition, while diffusion-based generative models have shown promising results for general image generation tasks, the generation of food images can be challenging due to the substantial intra-class variance. In this paper, we investigate the generation of synthetic food images based on the conditional diffusion model and propose an effective clustering-based training framework, named ClusDiff, for generating high-quality and representative food images. The proposed method is evaluated on the Food-101 dataset and shows improved performance when compared with existing image generation works. We also demonstrate that the synthetic food images generated by ClusDiff can help address the severe class imbalance issue in long-tailed food classification using the VFN-LT dataset.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Image-based dietary assessment serves as an efficient and accurate solution for recording and analyzing nutrition intake using eating occasion images as input. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Deep learning-based techniques are commonly used to perform image analysis such as food classification, segmentation, and portion size estimation, which rely on large amounts of food images with annotations for training. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | However, such data dependency poses significant barriers to real-world applications, because acquiring a substantial, diverse, and balanced set of food images can be challenging. | background 0.63 | 0.07 | 0.02 | asserted 0.99 |
| s3 | One potential solution is to use synthetic food images for data augmentation. | background 0.93 | 0.04 | 0.02 | hedged possibility 0.80 |
| s4 | Although existing work has explored the use of generative adversarial networks (GAN) based structures for generation, the quality of synthetic food images still remains subpar. | background 0.53 | 0.04 | 0.02 | asserted 1.00 |
| s5 | In addition, while diffusion-based generative models have shown promising results for general image generation tasks, the generation of food images can be challenging due to the substantial intra-class variance. | background 0.57 | 0.05 | 0.03 | asserted 0.91 |
| s6 | In this paper, we investigate the generation of synthetic food images based on the conditional diffusion model and propose an effective clustering-based training framework, named ClusDiff, for generating high-quality and representative food images. | contribution 0.95 | 0.94 | 0.25 | asserted 0.97 |
| s7 | The proposed method is evaluated on the Food-101 dataset and shows improved performance when compared with existing image generation works. | result 1.00 | 0.94 | 0.44 | asserted 1.00 |
| s8 | We also demonstrate that the synthetic food images generated by ClusDiff can help address the severe class imbalance issue in long-tailed food classification using the VFN-LT dataset. | result 0.96 | 0.96 | 0.24 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Image-based dietary assessment serves as an efficient and accurate solution for recording and analyzing nutrition intake using eating occasion images as input. Deep learning-based techniques are commonly used to perform image analysis such as food classification, segmentation, and portion size estimation, which rely on large amounts of food images with annotations for training. However, such data dependency poses significant barriers to real-world applications, because acquiring a substantial, diverse, and balanced set of food images can be challenging. One potential solution is to use synthetic food images for data augmentation. Although existing work has explored the use of generative adversarial networks (GAN) based structures for generation, the quality of synthetic food images still remains subpar. In addition, while diffusion-based generative models have shown promising results for general image generation tasks, the generation of food images can be challenging due to the substantial intra-class variance. In this paper, we investigate the generation of synthetic food images based on the conditional diffusion model and propose an effective clustering-based training framework, named ClusDiff, for generating high-quality and representative food images. The proposed method is evaluated on the Food-101 dataset and shows improved performance when compared with existing image generation works. We also demonstrate that the synthetic food images generated by ClusDiff can help address the severe class imbalance issue in long-tailed food classification using the VFN-LT dataset.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 5 sentences without a confident label
Cites as background 4 papers
-
MPG [ref] uses conditional StyleGAN2 [ref] to generate realistic pizza images with various ingredients.
MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs · Synthetic Image Generation · 0.90
-
While various food recognition systems have been developed to target real-world scenarios, such as continual learning [ref] , few-shot learning [ref] , and long-tailed classification [ref] , they usually necessitate more sophisticated training regimes.
Online Class-Incremental Learning For Real-World Food Image Classification · Image-based Dietary Assessment · 0.88
-
While various food recognition systems have been developed to target real-world scenarios, such as continual learning [ref] , few-shot learning [ref] , and long-tailed classification [ref] , they usually necessitate more sophisticated training regimes.
Single-Stage Heavy-Tailed Food Classification · Image-based Dietary Assessment · 0.88
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.