Pith. sign in

REVIEW 4 major objections 5 minor 65 references

SFOOD: A Multimodal Benchmark for Comprehensive Food Attribute Analysis Beyond RGB with Spectral Insights

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new benchmark with 2.35 million food samples argues that spectral bands, not RGB, are needed to read sweetness and identify herbs.

desk verdict The dataset work is real, but the two headline claims—food is uniquely hard, and three spectral bands beat RGB—rest on a biased test split and a synthetic RGB baseline. read the letter →

arxiv 2507.04412 v1 pith:CXYNN5RP submitted 2025-07-06 cs.CV

Zhenbo Xu , Jinghan Yang , Gong Huang , Jiqing Feng , Liu Liu , Ruihan Sun , Ajin Meng , Zhuo Zhang
show 1 more author
Zhaofeng He
This is my paper
classification cs.CV
keywords foodrecognitionhyperspectralimagingspectralbandselectionsweetnesspredictionChineseherbalmedicineclassificationvisualquestionansweringbenchmarkdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SFOOD, a benchmark suite with 3,266 fine-grained food categories, 2.35 million samples, 17 parent categories, and six tasks spanning classification, segmentation, counting, VQA, sweetness regression, and herbal-medicine identification. The authors' goal is to give the field a reusable testbed for food attributes beyond category labels, especially attributes they argue RGB cameras cannot capture. Their central empirical claim is that spectral measurements are crucial for fine-grained food analysis: with the same number of bands as RGB, selected spectral wavelengths reduce sweetness prediction error and sharply improve herbal classification. They also report that current large vision-language models and standard vision backbones perform far worse on food than on ImageNet or COCO, and worse than humans on food VQA, suggesting food is an under-served hard domain. If the benchmark holds up, it gives the community both a common evaluation standard and evidence that multi-band spectral sensing, not just better RGB models, is the way forward for food digitization.

What carries the argument

The load-bearing object is the SFOOD benchmark itself, especially the pairing of hyperspectral images with instrument-measured labels: Brix sweetness values from a sweetness meter, weights from a precision scale, and expert-labeled herbal categories. The argument for spectral necessity runs through controlled wavelength comparisons: three narrow RGB-like bands sampled from the hyperspectral cube (698.07, 544.14, and 436.53 nm) serve as the RGB baseline, while PCA-reduced 40-band images and three selected bands (944, 1013, and 1019 nm for sweetness; 822, 903, and 1007 nm for herbs) are fed to the same backbones. Spectral inputs consistently reduce sweetness MAE and raise herbal Top-1 accuracy, which is the mechanism carrying the paper's central claim.

What would settle it

Retrain the same backbones on standard RGB photographs of the same fruits and herbs, or reconstruct sRGB from the full hyperspectral cube using a measured camera response curve, and compare sweetness MAE and herbal Top-1 accuracy; if RGB reaches or approaches the 40-band spectral results (MAE near 1.27 Brix and Top-1 near 70.98%), the paper's central spectral claim fails.

Watch

Extended reading notes

Core claim

The central claim of the paper is that SFOOD is the largest and most comprehensive food benchmark to date, and that spectral information beyond visible light carries decisive signal for food attributes that are invisible or nearly invisible in RGB: fruit sweetness, measured as Brix by a sweetness meter, and the identity of visually similar Chinese herbal medicines. On the sweetness task, ResNet-50's MAE drops from 1.608 Brix on the three-band RGB baseline to 1.270 on 40 PCA bands, and SSFTT with just three selected bands (944, 1013, 1019 nm) reaches 1.482 versus 1.722 on RGB; on herbal classification, SSFTT's Top-1 accuracy rises from 32.90% on RGB to 70.98% on 40 bands and 45.30% on three selected bands. The paper also documents that large vision-language models such as GPT-4o and Gemini fail most food VQA sub-tasks, with refusal rates above 93% on weight estimation, and that standard segmentation and counting models lose substantial accuracy relative to COCO and FSC-147, supporting the claim that food is harder than mainstream vision subjects.

Load-bearing premise

The load-bearing premise is that the RGB baseline made from three narrow hyperspectral bands is a fair stand-in for real RGB cameras, which use broad overlapping filters and full spatial detail; if that baseline is unfair, the spectral advantage may shrink or disappear.

Editorial extensions

If this is right

  • The six-task SFOOD benchmark gives the community a single standard for measuring progress on food classification, segmentation, counting, VQA, sweetness, and herbal identification.
  • The spectral comparisons imply that three carefully chosen narrow bands can replace full hyperspectral cubes for sweetness and herbal tasks, lowering the hardware cost of practical food sensors.
  • The large LVLM failures on weight estimation and quality analysis indicate that current multimodal models lack basic food perception, not just food knowledge, creating a concrete target for future training data and model design.
  • The 3,266-category classification set exposes fine-grained confusions between visually similar dishes across cuisines, making it a stress test for fine-grained recognition beyond existing food sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the spectral advantage holds against a genuine RGB camera baseline, then a small set of near-infrared bands could be embedded in phones or kitchen devices to estimate sugar content, ripeness, or herb authenticity at scale.
  • Beyond the paper: because the hyperspectral images are only 35 by 35 pixels, the sweetness and herb results largely isolate chemical signatures from shape and texture; fusing full-resolution RGB with a few spectral bands is a natural extension the paper does not run.
  • Beyond the paper: SFOOD's test split is deliberately weighted toward hard samples, so the absolute accuracy numbers understate performance on easy foods; comparing against ImageNet under identical splitting would give a cleaner difficulty measure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SFOOD, a food benchmark that merges 11 existing food datasets with newly collected hyperspectral and instrument-measured labels, totaling 3,266 fine-grained categories and 2,351k data points across six tasks: fine-grained classification, instance segmentation, counting, visual question answering, spectral sweetness prediction, and spectral Chinese herbal medicine classification. The authors evaluate a range of CNN/transformer classifiers, counting models, and LVLMs, and they report two headline findings: (i) current models perform poorly on food relative to humans and relative to ImageNet/COCO, and (ii) spectral information is crucial for food attribute analysis, even with only three spectral bands. The paper also introduces a difficulty-weighted data split intended to place harder samples in the test set.

Significance. If the claims hold, SFOOD would be a valuable community resource: it unifies existing food data under a common taxonomy, adds instrument-measured sweetness values and a 60k-sample hyperspectral herbal dataset, and provides a multi-task evaluation suite that goes beyond classification. The empirical breadth—seven LVLMs, multiple CNN/transformer backbones, and four counting methods—is a strength, as is the stated intention to open-source the benchmark. The difficulty-weighted split is an interesting methodological idea, but as implemented it confounds the reported difficulty comparison, and the spectral-advantage conclusion rests on an RGB baseline that is not a realistic camera model. Both issues are testable with additional experiments, and the underlying data collection effort is substantial enough that the paper deserves a major revision rather than rejection.

major comments (4)
  1. [§3.3, Tables 2–3] The test split is constructed by training a model on the complete dataset and retaining the 20% of samples with the largest difficulty score (1.0 − S), yet no comparison is reported against a random split. The large gaps in Table 2 (e.g., Swin Base at 48.59% Top-1 on SFOOD test versus 83.5% on ImageNet) and Table 3 therefore quantify performance on a deliberately selected hard subset, not the intrinsic difficulty of food. Consequently, the abstract and Section 5 conclusions that food is 'one of the most difficult objects to study' and that large models are 'still poor at digitizing food' are not supported by the reported experiments. Please rerun the same protocol with a random split, or rebenchmark on the full data, and adjust the difficulty claims accordingly.
  2. [§4.5–4.6, Tables 6–7] The 'RGB' baseline is synthesized from three narrow hyperspectral bands at 698.07, 544.14, and 436.53 nm. Real RGB cameras use broad, overlapping spectral sensitivity functions and capture full spatial detail from the sensor, so this baseline does not represent traditional RGB perception. The comparison in Tables 6–7 therefore isolates band choice, not the camera modality named in the abstract ('Compared to traditional RGB cameras, spectral analysis is crucial'). The reported margins (MAE 1.270 versus 1.608 on sweetness; Top-1 70.98 versus 32.90 on herbs) could shrink or disappear under a realistic RGB baseline. Concretely, render RGB from the 127-band cubes using standard color-matching functions or capture real RGB images, and rerun Tables 6–7; alternatively, reframe the claim as 'selected narrow bands outperform fixed narrow bands'.
  3. [§3.2 vs §3.4] The dataset statistics are internally inconsistent. Section 3.2 reports 2,000 sweetness samples and hyperspectral images of size 35×35×180, while Section 3.4 reports 3k sweetness samples and size 35×35×127, and Table 8 lists 127 wavelengths. These discrepancies must be resolved because the benchmark's scale claims ('2,351 k data points', 'largest and most comprehensive food benchmark to date') and every spectral experiment depend on the correct numbers.
  4. [§3.3, Splitting] The difficulty scores are computed by a model trained on the complete dataset, which includes the samples that will later be placed in the test set. Even though the final models are trained only on the remaining 80%, the difficulty measure is an in-sample quantity and may reflect memorization rather than out-of-sample difficulty. The paper should state this clearly and, if possible, compute difficulty with a model trained only on the candidate training portion, or at least justify why the in-sample estimate is appropriate.
minor comments (5)
  1. [Table 4] Table 4 is very difficult to read because the columns are not aligned and the quality-analysis metrics appear repeated across rows; please reformat so that each metric is clearly associated with each model.
  2. [References] The reference list does not contain a separate entry for ChineseFoodNet or Uni-Food: in Section 3.2 ChineseFoodNet is cited as [18] (which is VegFru), and in Section 2 Uni-Food is cited as [23] (which is RODE). Please add the correct references.
  3. [Abstract] The phrase 'We spent a lot of manpower and equipment costs' is informal for a journal report; please replace it with a quantitative description of the collection effort, such as person-hours or number of acquisition sessions.
  4. [Tables 6–7] The spectral experiments report single runs without standard deviations or multiple seeds; given the small margins in some comparisons (e.g., 1.270 versus 1.287 for PCA(40) versus PCA(20) on ResNet-50 in Table 6), error bars or seed-averaged results are needed to judge robustness.
  5. [§4.4] The human evaluation is described only as 'three non-experts' with averaged results; please report the number of trials, the variance across evaluators, and whether the same images were shown to all evaluators.

Circularity Check

2 steps flagged · score 6.0 of 10

Two headline findings are partly built into the dataset construction: the 'RGB' baseline is defined as three narrow hyperspectral bands, so '3 selected bands beat RGB' is a band-selection comparison, and the test split is deliberately biased toward samples that a fitted model finds hardest, so 'food is hard' partly restates the split rule.

  1. self definitional [Section 4.5, Table 6; abstract finding (ii)]
    "We obtain RGB images by synthesizing images at three wavelengths: 698.07 nm, 544.14 nm, and 436.53 nm. ... Therefore, we arrive at two conclusions: (1) Spectral data is crucial for food property analysis; (2) Analysis based on three prominent spectral bands can significantly outperform RGB-based analysis."

    The baseline called 'RGB' is defined as three narrow bands extracted from the same hyperspectral cube. The headline 'even with only the same number (3) of spectral bands as RGB' therefore compares two three-band spectral representations, not a conventional RGB camera against spectral imaging. Any three discriminative bands chosen by BSNet can beat the fixed three wavelengths, so the claimed spectral-versus-RGB advantage reduces to the statement that optimized band selection beats a fixed band selection. Real RGB sensors integrate broad overlapping spectral responses and full spatial detail, which this definition discards, so the abstract's modality claim is not established by the table.

  2. fitted input called prediction [Section 3.3 (data partitioning) and Section 4.1 (fine-grained classification)]
    "we adopt a weight-adaptive data partitioning method that increases the likelihood of more challenging samples being concentrated in the test set. ... We record (1.0 − S) as the difficulty level of the sample. Ultimately, we select samples individually based on the weight of difficulty until 20% of the data is chosen as test data. ... which fully confirms the difficulty of fine-grained food classification."

    The test set is intentionally composed of the samples for which a model trained on the full data has the lowest confidence. Reporting low test accuracy on that set as evidence that food is inherently hard is reporting the selection rule back as a discovery. The comparison with ImageNet and Food-101 would be informative only if those test sets were constructed under the same difficulty-weighted rule, which they are not; the drop in Top-1 accuracy is therefore partly an artifact of the split rather than an independent property of the food domain.

full rationale

SFOOD is an empirical benchmark paper, so there is no equation-level derivation whose output equals its input. The circularity is at the level of construction. The first step is self-definitional: the RGB baseline is defined as three narrow hyperspectral wavelengths, making the headline '3 selected spectral bands beat RGB' a comparison between two three-band spectral representations, not a comparison of camera modalities; the claim is true by the chosen definition of RGB and does not validate the abstract's 'spectral analysis beyond RGB' framing. The second step is a fitted-input-called-prediction: a model is fit to the complete data to define per-sample difficulty, the test set is selected to maximize that difficulty, and the resulting low accuracy is then reported as evidence that food is hard, with external datasets serving as naive baselines that were not split by the same rule. These construction choices affect two of the paper's central claims, but the benchmark's concrete assets — 3,266 categories, 2,351k samples, six tasks, and the raw comparison of PCA and band-selection variants on the sweetness and herb tasks — retain independent empirical content, so the paper is not wholly circular. The score of 6 reflects that two headline findings reduce to construction choices while substantial independent benchmark work remains.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical findings rest on a small number of fitted analysis choices (selected bands, PCA dimensions, difficulty-weighted split) and on unverified measurement assumptions (Brix alignment, synthetic RGB fairness). No new theoretical entities are introduced.

free parameters (4)
  • BSNet-selected bands for sweetness = 944, 1013, 1019 nm
    Selected by BSNet from the hyperspectral data; the reported 3-band result depends on these data-chosen wavelengths and is not an out-of-sample fixed band set.
  • BSNet-selected bands for herbal classification = 822, 903, 1007 nm
    Same data-driven selection; no external validation on independent hyperspectral captures.
  • PCA reduced dimensions = 40 and 20 components
    PCA components computed from the same dataset and used as multispectral comparison baselines.
  • Test-set difficulty fraction = 20% hardest samples by model confidence
    The split deliberately concentrates difficult samples in the test set, shaping the 'models are poor' conclusion.
assumptions (4)
  • domain assumption Brix meter readings and weighing-instrument measurements are accurate and correspond to the exact items imaged.
    Section 3.2 describes juicing fruit pulp and reading a sweetness meter, and weighing to 0.01g; these labels are ground truth for the spectral and VQA tasks. Any alignment or calibration error propagates directly into reported MAE.
  • ad hoc to paper Synthetic RGB from three narrow bands is a fair baseline for RGB perception.
    Section 4.5 constructs RGB from 698.07, 544.14, and 436.53 nm; real RGB cameras use broad spectral response functions, so this assumption stacks the deck toward spectral input.
  • domain assumption The 11 merged public datasets have reliable labels that survive unification into 3,266 categories.
    Section 3.3 relies on DINOv2 similarity and manual annotation to merge categories; label noise from source datasets is inherited but not measured.
  • ad hoc to paper A single model's confidence (1 minus S) is a valid measure of sample difficulty for the entire test split.
    Section 3.3 uses model A's confidence to rank difficulty and selects the hardest 20% as test; this proxy shapes the benchmark difficulty conclusion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SFOOD: A Multimodal Benchmark for Comprehensive Food Attribute Analysis Beyond RGB with Spectral Insights." pith.science (2026). https://pith.science/paper/CXYNN5RP

@misc{pith2026250704412,
  author       = {Pith},
  title        = {Pith review of: SFOOD: A Multimodal Benchmark for Comprehensive Food Attribute Analysis Beyond RGB with Spectral Insights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CXYNN5RP}},
  note         = {Machine review of arXiv:2507.04412}
}
read the original abstract

With the rise and development of computer vision and LLMs, intelligence is everywhere, especially for people and cars. However, for tremendous food attributes (such as origin, quantity, weight, quality, sweetness, etc.), existing research still mainly focuses on the study of categories. The reason is the lack of a large and comprehensive benchmark for food. Besides, many food attributes (such as sweetness, weight, and fine-grained categories) are challenging to accurately percept solely through RGB cameras. To fulfill this gap and promote the development of intelligent food analysis, in this paper, we built the first large-scale spectral food (SFOOD) benchmark suite. We spent a lot of manpower and equipment costs to organize existing food datasets and collect hyperspectral images of hundreds of foods, and we used instruments to experimentally determine food attributes such as sweetness and weight. The resulting benchmark consists of 3,266 food categories and 2,351 k data points for 17 main food categories. Extensive evaluations find that: (i) Large-scale models are still poor at digitizing food. Compared to people and cars, food has gradually become one of the most difficult objects to study; (ii) Spectrum data are crucial for analyzing food properties (such as sweetness). Our benchmark will be open source and continuously iterated for different food analysis tasks.

Figures

Figures reproduced from arXiv: 2507.04412 by the authors.

Figure 1
Figure 1. SFOOD contains 6 different food analysis tasks, 17 main categories, 3,266 food subcategories, and 2.35 million data. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our currently adopted devices to build SFOOD. From [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Distributions of categories in fine-grained image classification (left), instance segmentation (middle), and object counting (right). The outer circle of the first figure plots the distribution of the number of images. SFOOD 1 Data collection and filtering Open-source Datasets Self-collected Datasets VLM Scorer Human Scorer 2 Labeling Manual Category Alignment Manual Interactive Labeling Manual Modification Physical… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Data processing procedures of our SFOOD. and Chinese herbal medicine. Detailed category definitions can be found in the Appendix. It is worth noting that we have not yet included natural plants and cultivated crops in SFOOD, although they are in some sense a form of fo…
Figure 5
Figure 5. Figure 5: Four example subcategories in each major category. From left to right: bakery, cake & pie, egg, seafood, meat, noodles & [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Fine-grained classification results by Swin Transformer on the test set. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Multi-food instance segmentation results by SOLOv2 on the test set. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 10
Figure 10. Figure 10: A 35x35 crop from [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 9
Figure 9. Figure 9: Tomatoes with different sweetness shown in RGB (left) [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 11
Figure 11. Figure 11: Responses from MiniCMP-V-2.6, Qwen2-VL-72B,GPT-4o in VQA tasks. From top to bottom, Image classification, Food [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Responses from MiniCMP-V-2.6, Qwen2-VL-72B,GPT-4o in VQA tasks, recipe generation and nutrition estimation. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 57 canonical work pages

  1. [1]

    Fruitq: a new dataset of multiple fruit images for freshness evaluation

    Olusola O Abayomi-Alli, Robertas Dama ˇseviˇcius, Sanjay Misra, and Adebayo Abayomi-Alli. Fruitq: a new dataset of multiple fruit images for freshness evaluation. Multime- dia Tools and Applications, 83(4):11433–11460, 2024. 2

    work page 2024

  2. [2]

    Live to eat and eat to live longer

    Amanda Adler and Rhea Saksena. Live to eat and eat to live longer. Nature Food, 4(12):1029–1030, 2023. 2

    work page 2023

  3. [3]

    Products-10k: A large-scale product recognition dataset

    Yalong Bai, Yuxiang Chen, Wei Yu, Linfang Wang, and Wei Zhang. Products-10k: A large-scale product recognition dataset. arXiv preprint arXiv:2008.10545, 2020. 2

    arXiv 2008

  4. [4]

    Recipenlg: A cooking recipes dataset for semi-structured text generation

    Michał Bie ´n, Michał Gilski, Martyna Maciejewska, Wojciech Taisner, Dawid Wisniewski, and Agnieszka Lawrynowicz. Recipenlg: A cooking recipes dataset for semi-structured text generation. In Proceedings of the 13th International Conference on Natural Language Generation, pages 22–28, 2020. 2, 3

    work page 2020

  5. [5]

    Food-101 – mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discriminative components with random forests. In European Conference on Computer Vision, 2014. 6

    work page 2014

  6. [6]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part VI 13 , pages 446–461. Springer, 2014. 2, 3, 4

    work page 2014

  7. [7]

    Bs-nets: An end- to-end framework for band selection of hyperspectral image

    Yaoming Cai, Xiaobo Liu, and Zhihua Cai. Bs-nets: An end- to-end framework for band selection of hyperspectral image. IEEE Transactions on Geoscience and Remote Sensing , 58 (3):1969–1984, 2019. 3

    work page 1969

  8. [8]

    Cascade r-cnn: High quality object detection and instance segmentation

    Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: High quality object detection and instance segmentation. IEEE transactions on pattern analysis and machine intelligence , 43(5):1483–1498, 2019. 7

    work page 2019

Show all 65 references
  1. [9]

    Deep-based ingredi- ent recognition for cooking recipe retrieval

    Jingjing Chen and Chong-Wah Ngo. Deep-based ingredi- ent recognition for cooking recipe retrieval. In Proceedings of the 24th ACM international conference on Multimedia , pages 32–41, 2016. 2, 4

    2016

  2. [10]

    Hybrid task cascade for instance seg- mentation

    Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaox- iao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, et al. Hybrid task cascade for instance seg- mentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4...

  3. [11]

    Beverage products packaging dataset for auto- matic shelf recognition and its application

    SIYUAN CHEN, YUMEI PU, XIAOXUAN LI, DANFEI LIU, et al. Beverage products packaging dataset for auto- matic shelf recognition and its application. 4

  4. [12]

    Fire: Food image to recipe generation

    Prateek Chhikara, Dhiraj Chaurasia, Yifan Jiang, Omkar Ma- sur, and Filip Ilievski. Fire: Food image to recipe generation. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision , pages 8184–8194, 2024. 3, 6

    2024

  5. [13]

    A low-shot object counting network with iterative prototype adaptation

    Nikola DJuki ´c, Alan Luke ˇziˇc, Vitjan Zavrtanik, and Matej Kristan. A low-shot object counting network with iterative prototype adaptation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 18872– 18881, 2023. 7

    2023

  6. [14]

    Retrieval and classi- fication of food images

    Giovanni Maria Farinella, Dario Allegra, Marco Moltisanti, Filippo Stanco, and Sebastiano Battiato. Retrieval and classi- fication of food images. Computers in biology and medicine, 77:23–39, 2016. 2

    2016

  7. [15]

    Metafood cvpr 2024 challenge on physically informed 3d food reconstruction: Methods and results

    Jiangpeng He, Yuhao Chen, Gautham Vinod, Talha Ibn Mahmud, Fengqing Zhu, Edward Delp, Alexander Wong, Pengcheng Xi, Ahmad AlMughrabi, Umair Haroon, et al. Metafood cvpr 2024 challenge on physically informed 3d food reconstruction: Methods and results. arXiv preprint arXiv:2407...

    2024 arXiv

  8. [16]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 7

    2017

  9. [17]

    Learning to count anything: Reference-less class-agnostic counting with weak supervision

    Michael Hobley and Victor Prisacariu. Learning to count anything: Reference-less class-agnostic counting with weak supervision. arXiv preprint arXiv:2205.10203, 2022. 7

    2022 arXiv

  10. [18]

    Vegfru: A domain-specific dataset for fine-grained visual categoriza- tion

    Saihui Hou, Yushan Feng, and Zilei Wang. Vegfru: A domain-specific dataset for fine-grained visual categoriza- tion. In Proceedings of the IEEE international conference on computer vision, pages 541–549, 2017. 2, 4

    2017

  11. [19]

    One-shot neu- ral band selection for spectral recovery

    Hai-Miao Hu, Zhenbo Xu, Wenshuai Xu, You Song, YiTao Zhang, Liu Liu, Zhilin Han, and Ajin Meng. One-shot neu- ral band selection for spectral recovery. In ICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023. 3

    2023

  12. [20]

    Mask scoring r-cnn

    Zhaojin Huang, Lichao Huang, Yongchao Gong, Chang Huang, and Xinggang Wang. Mask scoring r-cnn. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6409–6418, 2019. 7

    2019

  13. [21]

    Cwd30: A comprehensive and holistic dataset for crop weed recognition in precision agriculture

    Talha Ilyas, Dewa Made Sri Arsa, Khubaib Ahmad, Yong Chae Jeong, Okjae Won, Jong Hoon Lee, and Hyong- suk Kim. Cwd30: A comprehensive and holistic dataset for crop weed recognition in precision agriculture. arXiv preprint arXiv:2305.10084, 2023. 2

    2023 arXiv

  14. [22]

    Visible imaging to convolutionally discern and authenticate varieties of rice and their derived flours

    Manuel Izquierdo, Miguel Lastra-Mej ´ıas, Ester Gonz ´alez- Flores, Sandra Pradana-L ´opez, John C Cancilla, and Jos ´e S Torrecilla. Visible imaging to convolutionally discern and authenticate varieties of rice and their derived flours. Food Control, 110:106971, 2020. 2

    2020

  15. [23]

    Rode: Linear rectified mixture of diverse experts for food large multi-modal models

    Pengkun Jiao, Xinlan Wu, Bin Zhu, Jingjing Chen, Chong- Wah Ngo, and Yugang Jiang. Rode: Linear rectified mixture of diverse experts for food large multi-modal models. arXiv preprint arXiv:2407.12730, 2024. 3

    2024 arXiv

  16. [24]

    Foodx-251: a dataset for fine-grained food classification

    Parneet Kaur, Karan Sikka, Weijun Wang, Serge Belongie, and Ajay Divakaran. Foodx-251: a dataset for fine-grained food classification. arXiv preprint arXiv:1907.06167, 2019. 2, 4

    1907 arXiv

  17. [25]

    Automatic expansion of a food image dataset leveraging existing categories with domain adaptation

    Yoshiyuki Kawano and Keiji Yanai. Automatic expansion of a food image dataset leveraging existing categories with domain adaptation. In Computer Vision-ECCV 2014 Work- shops: Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part III 13, pages 3–17. Springer, 2015. 2, 3

    2014

  18. [26]

    A hierarchical grocery store image dataset with visual and se- mantic labels

    Marcus Klasson, Cheng Zhang, and Hedvig Kjellstr ¨om. A hierarchical grocery store image dataset with visual and se- mantic labels. In 2019 IEEE winter conference on applica- tions of computer vision, pages 491–500. IEEE, 2019. 2

    2019

  19. [27]

    Countr: Transformer-based generalised visual count- ing

    Chang Liu, Yujie Zhong, Andrew Zisserman, and Weidi Xie. Countr: Transformer-based generalised visual count- ing. arXiv preprint arXiv:2208.13721, 2022. 7

    2022 arXiv

  20. [28]

    Ingredient prediction via context learn- ing network with class-adaptive asymmetric loss

    Mengjiang Luo, Weiqing Min, Zhiling Wang, Jiajun Song, and Shuqiang Jiang. Ingredient prediction via context learn- ing network with class-adaptive asymmetric loss. IEEE Transactions on Image Processing, 2023. 3

    2023

  21. [29]

    Recipe1m+: A dataset for learning cross-modal embeddings for cooking recipes and food images

    Javier Marın, Aritro Biswas, Ferda Ofli, Nicholas Hynes, Amaia Salvador, Yusuf Aytar, Ingmar Weber, and Antonio Torralba. Recipe1m+: A dataset for learning cross-modal embeddings for cooking recipes and food images. IEEE Transactions on Pattern Analysis and Machine Intelligenc...

    2021

  22. [30]

    Fruitnet: Indian fruits im- age dataset with quality for machine learning applications

    Vishal Meshram and Kailas Patil. Fruitnet: Indian fruits im- age dataset with quality for machine learning applications. Data in Brief, 40:107686, 2022. 2

    2022

  23. [31]

    Isia food- 500: A dataset for large-scale food recognition via stacked global-local attention network

    Weiqing Min, Linhu Liu, Zhiling Wang, Zhengdong Luo, Xiaoming Wei, Xiaolin Wei, and Shuqiang Jiang. Isia food- 500: A dataset for large-scale food recognition via stacked global-local attention network. In Proceedings of the 28th ACM International Conference on Multimedia , pa...

    2020

  24. [32]

    Large scale visual food recognition

    Weiqing Min, Zhiling Wang, Yuxin Liu, Mengjiang Luo, Liping Kang, Xiaoming Wei, Xiaolin Wei, and Shuqiang Jiang. Large scale visual food recognition. IEEE Transac- tions on Pattern Analysis and Machine Intelligence , 45(8): 9932–9949, 2023. 2, 4

    2023

  25. [33]

    Fruits-262 dataset: A dataset containing a vast majority of the popular and known fruits, 2021

    M Minut. Fruits-262 dataset: A dataset containing a vast majority of the popular and known fruits, 2021. 2, 3, 4

    2021

  26. [34]

    Using deep learning for image-based plant disease detection

    Sharada P Mohanty, David P Hughes, and Marcel Salath ´e. Using deep learning for image-based plant disease detection. Frontiers in plant science, 7:1419, 2016. 2

    2016

  27. [35]

    Llava-chef: A multi- modal generative model for food recipes

    Fnu Mohbat and Mohammed J Zaki. Llava-chef: A multi- modal generative model for food recipes. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 1711–1721, 2024. 3

    2024

  28. [36]

    Omnicount: Multi-label object count- ing with semantic-geometric priors

    Anindya Mondal, Sauradip Nag, Xiatian Zhu, and An- jan Dutta. Omnicount: Multi-label object count- ing with semantic-geometric priors. arXiv preprint arXiv:2403.05435, 2024. 2, 3

    2024

  29. [37]

    Terrace-based food counting and segmentation

    Huu-Thanh Nguyen and Chong-Wah Ngo. Terrace-based food counting and segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 2364– 2372, 2021. 3

    2021

  30. [38]

    Sibnet: Food instance counting and segmentation

    Huu-Thanh Nguyen, Chong-Wah Ngo, and Wing-Kwong Chan. Sibnet: Food instance counting and segmentation. Pattern Recognition, 124:108470, 2022. 3

    2022

  31. [39]

    Honey dataset stan- dard using hyperspectral imaging for machine learning prob- lems

    Ary Noviyanto and Waleed H Abdullah. Honey dataset stan- dard using hyperspectral imaging for machine learning prob- lems. In 2017 25th European Signal Processing Conference (EUSIPCO), pages 473–477. IEEE, 2017. 2, 3

    2017

  32. [40]

    Uec-foodpix complete: A large-scale food image segmentation dataset

    Kaimu Okamoto and Keiji Yanai. Uec-foodpix complete: A large-scale food image segmentation dataset. In Pat- tern Recognition. ICPR International Workshops and Chal- lenges: Virtual Event, January 10–15, 2021, Proceedings, Part V, pages 647–659. Springer, 2021. 2, 3

    2021

  33. [41]

    Foodd: food detection dataset for calorie mea- surement using food images

    Parisa Pouladzadeh, Abdulsalam Yassine, and Shervin Shir- mohammadi. Foodd: food detection dataset for calorie mea- surement using food images. In New Trends in Image Anal- ysis and Processing–ICIAP 2015 Workshops: ICIAP 2015 International Workshops, BioFor, CTMR, RHEUMA, ISCA...

    2015

  34. [42]

    Learning to count everything

    Viresh Ranjan, Udbhav Sharma, Thu Nguyen, and Minh Hoai. Learning to count everything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3394–3403, 2021. 2, 3

    2021

  35. [43]

    Leveraging automatic personalised nutrition: food image recognition benchmark and dataset based on nutrition taxonomy

    Sergio Romero-Tapiador, Ruben Tolosana, Aythami Morales, Julian Fierrez, Ruben Vera-Rodriguez, Isabel Espinosa-Salinas, Gala Freixer, Enrique Carrillo de Santa Pau, Ana Ram ´ırez de Molina, and Javier Ortega- Garcia. Leveraging automatic personalised nutrition: food image reco...

  36. [44]

    Multi-task learn- ing for calorie prediction on a novel large-scale recipe dataset enriched with nutritional information

    Robin Ruede, Verena Heusser, Lukas Frank, Alina Roitberg, Monica Haurilet, and Rainer Stiefelhagen. Multi-task learn- ing for calorie prediction on a novel large-scale recipe dataset enriched with nutritional information. In 2020 25th Inter- national Conference on Pattern Reco...

    2020

  37. [45]

    Represent, compare, and learn: A similarity-aware framework for class-agnostic counting

    Min Shi, Hao Lu, Chen Feng, Chengxin Liu, and Zhiguo Cao. Represent, compare, and learn: A similarity-aware framework for class-agnostic counting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9529–9538, 2022. 7

    2022

  38. [46]

    Plantdoc: A dataset for visual plant disease detection

    Davinder Singh, Naman Jain, Pranjali Jain, Pratik Kayal, Sudhakar Kumawat, and Nipun Batra. Plantdoc: A dataset for visual plant disease detection. In Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pages 249–253. 2020. 2

    2020

  39. [47]

    The cropandweed dataset: A multi-modal learning approach for efficient crop and weed manipulation

    Daniel Steininger, Andreas Trondl, Gerardus Croonen, Julia Simon, and Verena Widhalm. The cropandweed dataset: A multi-modal learning approach for efficient crop and weed manipulation. In Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision , page...

    2023

  40. [48]

    Nutritionverse-3d: A 3d food model dataset for nutritional intake estimation

    Chi-en Amy Tai, Matthew Keller, Mattie Kerrigan, Yuhao Chen, Saeejith Nair, Pengcheng Xi, and Alexander Wong. Nutritionverse-3d: A 3d food model dataset for nutritional intake estimation. arXiv preprint arXiv:2304.05619, 2023. 3

    2023 arXiv

  41. [49]

    Classification of biscuit defect states and foreign objects using cnn-based features

    Yavuz Selim Taspinar. Classification of biscuit defect states and foreign objects using cnn-based features. In 13th Dort- mund International Research Conference 2022, 2022. 4

    2022

  42. [50]

    Nutrition5k: To- wards automatic nutritional understanding of generic food

    Quin Thames, Arjun Karpur, Wade Norris, Fangting Xia, Liviu Panait, Tobias Weyand, and Jack Sim. Nutrition5k: To- wards automatic nutritional understanding of generic food. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 8903–8911,...

    2021

  43. [51]

    Rice seedling detection in uav images using transfer learning and machine learning

    Hsin-Hung Tseng, Ming-Der Yang, R Saminathan, Yu-Chun Hsu, Chin-Ying Yang, and Dong-Hong Wu. Rice seedling detection in uav images using transfer learning and machine learning. Remote Sensing, 14(12):2837, 2022. 2

    2022

  44. [52]

    Solov2: Dynamic and fast instance segmenta- tion

    Xinlong Wang, Rufeng Zhang, Tao Kong, Lei Li, and Chun- hua Shen. Solov2: Dynamic and fast instance segmenta- tion. Advances in Neural information processing systems , 33:17721–17732, 2020. 7

    2020

  45. [53]

    Multi-state ingredient recognition via adaptive multi-centric network

    Min Wen, Jiajun Song, Weiqing Min, Weimin Xiao, Lin Han, and Shuqiang Jiang. Multi-state ingredient recognition via adaptive multi-centric network. IEEE Transactions on In- dustrial Informatics, 2023. 3

    2023

  46. [54]

    Automatic counting of in situ rice seedlings from uav images based on a deep fully convolutional neural network

    Jintao Wu, Guijun Yang, Xiaodong Yang, Bo Xu, Liang Han, and Yaohui Zhu. Automatic counting of in situ rice seedlings from uav images based on a deep fully convolutional neural network. Remote Sensing, 11(6):691, 2019. 2, 3

    2019

  47. [55]

    A large-scale benchmark for food im- age segmentation

    Xiongwei Wu, Xin Fu, Ying Liu, Ee-Peng Lim, Steven CH Hoi, and Qianru Sun. A large-scale benchmark for food im- age segmentation. In Proceedings of the 29th ACM interna- tional conference on multimedia , pages 506–515, 2021. 2, 3

    2021

  48. [56]

    Hsifoodingr-64: A dataset for hyperspectral food-related studies and a benchmark method on food ingredient retrieval

    Xiaojie Xia, Wei Liu, Liuan Wang, and Jun Sun. Hsifoodingr-64: A dataset for hyperspectral food-related studies and a benchmark method on food ingredient retrieval. IEEE Access, 11:13152–13162, 2023. 2, 3

    2023

  49. [57]

    Multiple attentional pyra- mid networks for chinese herbal recognition

    Yingxue Xu, Guihua Wen, Yang Hu, Mingnan Luo, Dan Dai, Yishan Zhuang, and Wendy Hall. Multiple attentional pyra- mid networks for chinese herbal recognition. Pattern Recog- nition, 110:107558, 2021. 2, 3, 4

    2021

  50. [58]

    Foodlmm: A versatile food assistant using large multi-modal model

    Yuehao Yin, Huiyan Qi, Bin Zhu, Jingjing Chen, Yu-Gang Jiang, and Chong-Wah Ngo. Foodlmm: A versatile food assistant using large multi-modal model. arXiv preprint arXiv:2312.14991, 2023. 3

    2023 arXiv

  51. [59]

    Fine-grained image classifi- cation by exploring bipartite-graph labels

    Feng Zhou and Yuanqing Lin. Fine-grained image classifi- cation by exploring bipartite-graph labels. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 1124–1133, 2016. 2

    2016

  52. [60]

    Foodsky: A food-oriented large language model that passes the chef and dietetic examination

    Pengfei Zhou, Weiqing Min, Chaoran Fu, Ying Jin, Mingyu Huang, Xiangyang Li, Shuhuan Mei, and Shuqiang Jiang. Foodsky: A food-oriented large language model that passes the chef and dietetic examination. arXiv preprint arXiv:2406.10261, 2024. 2, 3

    2024 arXiv

  53. [61]

    Learn more for food recognition via progressive self-distillation

    Yaohui Zhu, Linhu Liu, and Jiang Tian. Learn more for food recognition via progressive self-distillation. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 3879–3887, 2023. 3 SFOOD: A Multimodal Benchmark for Comprehensive Food Attribute Analysis Beyond ...

    2023

  54. [62]

    It is worth noting that these categories do not represent all foods

    The definition of food categories We have classified common foodstuffs, including raw in- gredients and processed dishes, into 17 major categories: bakery, cake & pie, egg, seafood, meat, noodles & steamed pasta, grain & cereal, soup, vegetable, fruit, bean product, dairy prod...

  55. [63]

    The definition of food instances Here, we discuss the annotation standards for food instance segmentation separately. After examining the labeling rules of current food datasets such as UEC Food-256, UEC-FoodPix, and Food- Seg103, we find that the instance labeling rules vary ...

  56. [64]

    2), we have obtained spectral data with wavelength between 400 nm to 1020 nm shown in Table 8

    Hyperspectral Data With the use of dual hyperspectral cameras(Fig. 2), we have obtained spectral data with wavelength between 400 nm to 1020 nm shown in Table 8

  57. [65]

    Experiments Setting and Analysis All experiments are conducted on a server with 8 NVIDIA GeForce RTX 3090. 9.1. Fine-grained image classification To fairly evaluate each model, we employ pre-trained mod- els from ImageNet and their training settings suggested by the Timm repos...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.