Pith. sign in

REVIEW 2 major objections 66 references

Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning

T0 review · 2 major / 0 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A food vision-language model trained with chain-of-thought tuning and group relative policy optimization outperforms baselines on calorie and nutrition tasks.

desk verdict New CalorieBench-80K benchmark with CoT annotations is the main addition, but the abstract supplies zero metrics or validation details to back any performance claims. read the letter →

arxiv 2606.04986 v1 pith:X2QZUK4F submitted 2026-06-03 cs.CV

Yu Zhu , Yongkang Li , Wenjie Zhu , Haoyi Jiang , Wenyu Liu , Wei Yang , Bin Li , Xinggang Wang This is my paper · ORCID
classification cs.CV
keywords foodvision-languagemodelcalorieestimationreinforcementlearningchain-of-thoughtmulti-tasknutritionalanalysisGRPOdietaryadvice
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents CalorieBench-80K as the first food image benchmark to include large-scale chain-of-thought annotations for calorie reasoning along with dietary advice. It then introduces Food-R1, a single multi-task vision-language model that first receives cold-start instruction tuning on these annotations and then undergoes reinforcement fine-tuning with Group Relative Policy Optimization. The central goal is to move beyond standard supervised fine-tuning so that the model develops stronger step-by-step reasoning and broader generalization across food-related vision-language jobs. If the approach succeeds, automated systems could produce more reliable calorie estimates and advice from ordinary food photographs.

What carries the argument

Group Relative Policy Optimization (GRPO) applied after CoT cold-start instruction tuning, which refines the policy to improve multi-step calorie reasoning and multi-task performance.

What would settle it

Ablating the GRPO reinforcement stage and re-evaluating Food-R1 on CalorieBench-80K yields no measurable improvement over the cold-start model alone.

Watch

Extended reading notes

Core claim

Food-R1 is a unified multi-task food vision-language model that first performs CoT-based cold-start instruction tuning on CalorieBench-80K and then applies reinforcement fine-tuning via Group Relative Policy Optimization; this two-stage process yields consistent gains over strong baselines on CalorieBench-80K and other representative food benchmarks.

Load-bearing premise

The curated chain-of-thought annotations are reliable ground truth and the reinforcement stage produces genuine reasoning gains rather than benchmark-specific fitting.

Editorial extensions

If this is right

  • A single model can address calorie estimation, dietary advice, and other food tasks without task-specific fine-tuning.
  • Chain-of-thought annotations allow the model to break calorie calculations into explicit reasoning steps from images.
  • Reinforcement fine-tuning after supervised tuning improves results compared with supervised tuning alone.
  • The released benchmark and model weights enable direct comparison and extension by other researchers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-stage CoT-plus-GRPO recipe could be tested on other narrow domains such as medical or agricultural image analysis where step-by-step reasoning matters.
  • Evaluating Food-R1 on everyday user photos taken under uncontrolled lighting and angles would test whether benchmark gains transfer to practical use.
  • If GRPO gains hold across different base vision-language models, the method could become a standard post-training step for specialized VLMs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The paper introduces CalorieBench-80K, a new large-scale food image benchmark containing curated calorie labels and Chain-of-Thought (CoT) dietary advice annotations, claimed to be the first such benchmark with CoT for calorie reasoning. It proposes Food-R1, a unified multi-task vision-language model that undergoes CoT-based cold-start instruction tuning followed by reinforcement fine-tuning via Group Relative Policy Optimization (GRPO). The central claim is that Food-R1 consistently outperforms strong baselines across food-related tasks on CalorieBench-80K and other representative benchmarks, with code, weights, and annotations released.

Significance. If the benchmark annotations prove reliable and the reported gains are attributable to the proposed training pipeline rather than annotation artifacts, the work could provide a useful demonstration of applying GRPO-style reinforcement learning to improve reasoning in domain-specific VLMs. The public release of the benchmark and model is a clear strength. However, the significance is limited by the absence of any quantitative validation of the new benchmark's ground truth, which undercuts attribution of performance improvements.

major comments (2)
  1. [Abstract and benchmark section] Abstract and § on benchmark construction: The CalorieBench-80K annotations are described only as 'curated' with no details on generation method (human experts vs. LLM-assisted), validation protocol, inter-annotator agreement, or measured error rates. This is load-bearing for the central claim because every reported result on CalorieBench-80K depends on these labels and CoT being reliable ground truth; without such evidence, measured gains cannot be confidently attributed to GRPO or improved reasoning.
  2. [Experiments] Experiments section: The abstract asserts consistent outperformance on CalorieBench-80K and other benchmarks, yet provides no metrics, baseline details, ablation studies, or error analysis. This prevents assessment of whether GRPO produces genuine generalization gains or merely fits the specific annotation distribution of the new benchmark.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We agree that the manuscript requires additional details on benchmark construction and experimental reporting to support the central claims, and we will revise accordingly.

read point-by-point responses

  1. Referee: [Abstract and benchmark section] Abstract and § on benchmark construction: The CalorieBench-80K annotations are described only as 'curated' with no details on generation method (human experts vs. LLM-assisted), validation protocol, inter-annotator agreement, or measured error rates. This is load-bearing for the central claim because every reported result on CalorieBench-80K depends on these labels and CoT being reliable ground truth; without such evidence, measured gains cannot be confidently attributed to GRPO or improved reasoning.

    Authors: We agree that the current description is insufficient for establishing ground-truth reliability. In the revised manuscript we will add a dedicated subsection detailing the annotation pipeline, including the mix of human expert and LLM-assisted generation, the validation protocol, inter-annotator agreement statistics, and measured error rates on a held-out subset. These additions will allow readers to assess whether performance gains can be attributed to the training pipeline. revision: yes

  2. Referee: [Experiments] Experiments section: The abstract asserts consistent outperformance on CalorieBench-80K and other benchmarks, yet provides no metrics, baseline details, ablation studies, or error analysis. This prevents assessment of whether GRPO produces genuine generalization gains or merely fits the specific annotation distribution of the new benchmark.

    Authors: We acknowledge the lack of quantitative detail in the current experiments section. The revision will expand this section with full per-task metrics, explicit baseline implementations and hyper-parameters, ablation studies that isolate the CoT cold-start and GRPO stages, and an error analysis comparing performance on in-distribution versus out-of-distribution food images to evaluate generalization. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; empirical claims rest on external benchmarks and released data

full rationale

The paper introduces CalorieBench-80K as a new benchmark with curated CoT annotations and reports Food-R1 performance after CoT cold-start + GRPO training. No mathematical derivation chain, self-definitional equations, or fitted parameters renamed as predictions appear in the provided text. Performance claims are evaluated on the new benchmark plus separate representative benchmarks, with code/model/weights released. This is standard empirical ML practice with no reduction of results to inputs by construction. Self-citations, if present, are not load-bearing for the central claim.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract supplies no information on free parameters, background axioms, or newly postulated entities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning." pith.science (2026). https://pith.science/paper/X2QZUK4F

@misc{pith2026260604986,
  author       = {Pith},
  title        = {Pith review of: Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X2QZUK4F}},
  note         = {Machine review of arXiv:2606.04986}
}
read the original abstract

Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.

Figures

Figures reproduced from arXiv: 2606.04986 by the authors.

Figure 1
Figure 1. Qualitative comparison before and after RFT. Compared with the SFT-only model, Food-R1 after post-training RFT produces more stable and accurate calorie estimates through a step-by-step reasoning process. Recent advances in Vision–Language Models have enabled new progress in food analysis. With broad world knowledge and multimodal reasoning, VLMs have been applied to food classification [33], ingredient recognition … view at source ↗
Figure 2
Figure 2. CalorieBench-80K construction and CoT generation pipeline. Left: Built upon MM-Food-100K, we apply data filtering, granularity alignment, and dietary advice annotation to obtain CalorieBench-80K. Right: We prompt ChatGPT to produce nutrition-focused CoT rationales for calorie estimation. fine-tuning; both yield notable gains. However, CoT-integrated VLM training for food analysis remains underexplored. In this work,… view at source ↗
Figure 3
Figure 3. Overview of Food-R1. Given an image and text prompt, Food-R1 per￾forms calorie estimation via CoT reasoning and outputs structured predictions, while also supporting food classification, ingredient recognition, recipe genera￾tion, dietary advice generation, and nutrition estimation. 3.2 Step 1: Data Filtering Although MM-Food-100K provides a solid foundation, we observe issues such as image–text mismatches and noisy… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Two-stage Training Paradigm. Stage 1 (SFT): The base model is fine￾tuned on mixed food datasets, incorporating CoT data for cold-start instruction tuning. Stage 2 (RFT): The SFT checkpoint serves as the reference model, while the policy model is optimized with GRPO usi…
Figure 5
Figure 5. Figure 5: Representative examples of Food-R1. The examples cover recipe generation, nutrition estimation, dietary advice, and multi-turn dialogue [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 9 canonical work pages

  1. [1]

    Multimedia Systems 29, 2049–2057 (2023)

    Battini S¨ onmez, E., Memi¸ s, S., Arslan, B., Batur, O.Z.: The segmented UEC food- 100 dataset with benchmark experiment on food detection. Multimedia Systems 29, 2049–2057 (2023)

  2. [2]

    Bola˜ nos, M., Radeva, P.: Simultaneous food localization and recognition (2017), arXiv:1604.07953

    work page Pith review arXiv 2017

  3. [3]

    In: Computer Vision – ECCV 2014

    Bossard, L., Guillaumin, M., Van Gool, L.: Food-101: Mining discriminative com- ponents with random forests. In: Computer Vision – ECCV 2014. Lecture Notes in Computer Science, vol. 8694, pp. 446–461 (2014) Food-R1 13

    2014

  4. [4]

    In: Proceedings of the 24th ACM International Conference on Multimedia

    Chen, J., Ngo, C.W.: Deep-based ingredient recognition for cooking recipe retrieval. In: Proceedings of the 24th ACM International Conference on Multimedia. pp. 32– 41 (2016)

    2016

  5. [5]

    Chen, L., Zhao, X., Zeng, Z., Huang, J., Zhong, Y., Ma, L.: Chart-R1: Chain- of-thought supervision and reinforcement for advanced chart reasoner (2025), arXiv:2507.15509

    work page arXiv 2025

  6. [6]

    Chhikara, P., Chaurasia, D., Jiang, Y., Masur, O., Ilievski, F.: FIRE: Food image to recipe generation (2024), arXiv preprint arXiv:2308.14391

    work page arXiv 2024

  7. [7]

    IEEE Journal of Biomedical and Health Informatics21, 588–598 (2017)

    Ciocca, G., Napoletano, P., Schettini, R.: Food recognition: A new dataset, ex- periments, and results. IEEE Journal of Biomedical and Health Informatics21, 588–598 (2017)

    2017

  8. [8]

    Dong, Y., Muraoka, Y., Shi, S., Zhang, Y.: MM-Food-100K: A 100,000- sample multimodal food intelligence dataset with verifiable provenance (2025), arXiv:2508.10429

    work page Pith review arXiv 2025

Show all 66 references
  1. [9]

    In: International Conference on Learning Representations (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022)

    2022

  2. [10]

    Huang, W., Jia, B., Zhai, Z., Cao, S., Ye, Z., Zhao, F., Xu, Z., Hu, Y., Lin, S.: Vision-R1: Incentivizing reasoning capability in multimodal large language models (2025), arXiv:2503.06749

    2025 arXiv

  3. [11]

    Jiang, B., Chen, S., Zhang, Q., Liu, W., Wang, X.: AlphaDrive: Unleashing the power of VLMs in autonomous driving via reinforcement learning and reasoning (2025), arXiv:2503.07608

    2025 arXiv

  4. [12]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2025)

    Jiang, J., Ma, C., Song, X., Zhang, H., Luo, J.: Corvid: Improving multimodal large language models towards chain-of-thought reasoning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2025)

    2025

  5. [13]

    Jiao, P., Wu, X., Zhu, B., Chen, J., Ngo, C.W., Jiang, Y.G.: RoDE: Linear rectified mixture of diverse experts for food large multi-modal models (2024), arXiv:2407.12730

    2024

  6. [14]

    In: International Conference on Learning Representations (2026)

    Li, Y., Xiong, K., Guo, X., Li, F., Yan, S., Xu, G., Zhou, L., Chen, L., Sun, H., Wang, B., Ma, K., Chen, G., Ye, H., Liu, W., Wang, X.: ReCogDrive: A rein- forced cognitive framework for end-to-end autonomous driving. In: International Conference on Learning Representations (2026)

    2026

  7. [15]

    In: Text Summarization Branches Out

    Lin, C.Y.: ROUGE: A package for automatic evaluation of summaries. In: Text Summarization Branches Out. pp. 74–81. Association for Computational Linguis- tics, Barcelona, Spain (2004)

    2004

  8. [16]

    IEEE Trans

    Liu, G., Jiao, Y., Chen, J., Zhu, B., Jiang, Y.G.: From canteen food to daily meals: Generalizing food recognition to more practical scenarios. IEEE Trans. Multimedia 27, 2724–2733 (2025)

    2025

  9. [17]

    In: Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision

    Liu, G., Yin, H., Zhu, B., Chen, J., Ngo, C.W., Jiang, Y.G.: Retrieval augmented recipe generation. In: Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision. pp. 2453–2463 (2025)

    2025

  10. [18]

    IEEE Trans

    Luo, M., Min, W., Wang, Z., Song, J., Jiang, S.: Ingredient prediction via context learning network with class-adaptive asymmetric loss. IEEE Trans. Image Process. 32, 5509–5523 (2023)

    2023

  11. [19]

    Cambridge University Press (2008)

    Manning, C.D., Raghavan, P., Sch¨ utze, H.: Introduction to Information Retrieval. Cambridge University Press (2008)

    2008

  12. [20]

    IEEE Trans

    Marin, J., Biswas, A., Ofli, F., Hynes, N., Salvador, A., Aytar, Y., Weber, I., Tor- ralba, A.: Recipe1M+: A dataset for learning cross-modal embeddings for cooking recipes and food images. IEEE Trans. Pattern Anal. Mach. Intell.43(1), 187–203 (2021) 14 Y. Zhu et al

    2021

  13. [21]

    IEEE Trans

    Min, W., Wang, Z., Liu, Y., Luo, M., Kang, L., Wei, X., Wei, X., Jiang, S.: Large scale visual food recognition. IEEE Trans. Pattern Anal. Mach. Intell.45, 9932– 9949 (2023)

    2023

  14. [22]

    In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

    Mohbat, F., Zaki, M.J.: LLaVA-chef: A multi-modal generative model for food recipes. In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. pp. 1711–1721 (2024)

    2024

  15. [23]

    In: Advances in Neural Information Processing Systems

    Mu, Y., Zhang, Q., Hu, M., Wang, W., Ding, M., Jin, J., Wang, B., Dai, J., Qiao, Y., Luo, P.: EmbodiedGPT: Vision-language pre-training via embodied chain of thought. In: Advances in Neural Information Processing Systems. vol. 36 (2023)

    2023

  16. [24]

    In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: A method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. pp. 311–318. Philadelphia, Penn- sylvania, USA (2002)

    2002

  17. [25]

    In: Proceedings of the Third Conference on Machine Translation: Research Papers

    Post, M.: A call for clarity in reporting BLEU scores. In: Proceedings of the Third Conference on Machine Translation: Research Papers. pp. 186–191. Association for Computational Linguistics, Brussels, Belgium (2018)

    2018

  18. [26]

    In: Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Salvador, A., Hynes, N., Aytar, Y., Marin, J., Ofli, F., Weber, I., Torralba, A.: Learning cross-modal embeddings for cooking recipes and food images. In: Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3068–3076 (2017)

    2017

  19. [27]

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y.K., Wu, Y., Guo, D.: DeepSeekMath: Pushing the limits of mathematical reasoning in open language models (2024), arXiv:2402.03300

    2024 arXiv

  20. [28]

    In: MultiMedia Model- ing

    Tanabe, H., Yanai, K.: CalorieVoL: Integrating volumetric context into multimodal large language models for image-based calorie estimation. In: MultiMedia Model- ing. Lecture Notes in Computer Science, vol. 15523, pp. 353–365 (2025)

    2025

  21. [29]

    Nutrients17(7), 1128 (2025)

    Tanabe, H., Yanai, K.: Reasoning-driven food energy estimation via multimodal large language models. Nutrients17(7), 1128 (2025)

    2025

  22. [30]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Thames, Q., Karpur, A., Norris, W., Xia, F., Panait, L., Weyand, T., Sim, J.: Nutri- tion5k: Towards automatic nutritional understanding of generic food. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8903–8911 (2021)

    2021

  23. [31]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., Yuan, L.: LLaVA-CoT: Let vision language models reason step-by-step. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 11464–11475 (2025)

    2025

  24. [32]

    Yao, D., Yao, K., Zhou, J., Zhang, Y.: CaLoRAify: Calorie estimation with visual- text pairing and LoRA-driven visual language models (2024), arXiv:2412.09936

    2024

  25. [33]

    IEEE Trans

    Yin, Y., Qi, H., Zhu, B., Chen, J., Jiang, Y.G., Ngo, C.W.: FoodLMM: A versatile food assistant using large multi-modal model. IEEE Trans. Multimedia27, 6949– 6961 (2025)

    2025

  26. [34]

    In: International Conference on Learning Representations (2025)

    Zhang, R., Wei, X., Jiang, D., Guo, Z., Zhang, Y., Tong, C., Liu, J., Zhou, A., Zhang, S., Gao, P., Li, H.: MAVIS: Mathematical visual instruction tuning with an automatic data engine. In: International Conference on Learning Representations (2025)

    2025

  27. [35]

    In: International Conference on Learning Rep- resentations (2020)

    Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: Evalu- ating text generation with BERT. In: International Conference on Learning Rep- resentations (2020)

    2020

  28. [36]

    yes” or “partial

    Zhao, Y., Huang, J., Hu, J., Wang, X., Mao, Y., Zhang, D., Zhang, H., Jiang, Z., Wu, Z., Ai, B., Wang, A., Zhou, W., Chen, Y.: SWIFT: A scalable lightweight infrastructure for fine-tuning. In: Proceedings of the AAAI Conference on Artificial Intelligence (2025) Food-R1 15 Appe...

    2025

  29. [37]

    4) One quantity item, whose name may be generic

    A list of ingredient names. 4) One quantity item, whose name may be generic. Your task: - Using both the image and the text, choose exactly one ingredient from the ingredient list that best matches this quantity item. - If the quantity name is generic, infer from the image whi...

  30. [38]

    Objective nutrition overview: Briefly describe the dish’s total energy and key nutrition features (e.g., protein, fat, carbohydrates)

  31. [39]

    high” or “low

    Practical suggestions: Give concrete, realistic tips to make this meal and the rest of the day more balanced and healthy. For example, you may suggest changing portion size, adding vegetables or whole grains, reducing sugary items, or balancing with lighter meals later in the ...

  32. [40]

    Remind the user to keep their overall diet diverse and balanced, instead of focusing on a single meal

    Encouragement and variety reminder: If there are no major concerns (not extremely high in energy, sugar, or sodium), encourage enjoying the dish in moderation. Remind the user to keep their overall diet diverse and balanced, instead of focusing on a single meal. Prompt for Die...

  33. [41]

    Ingredient identification: name the dish and its visible inferable components

  34. [42]

    Quantity estimation: assign realistic amounts with units; use label text if visible, otherwise provide plausible estimates

  35. [43]

    Calorie Estimation Fig

    Calorie estimation: state assumptions, show simple arithmetic for each item, and report the summed total. Calorie Estimation Fig. S4: Prompt templates for calorie estimation in the answer-only format and the format with CoT. # role definition You are a cooking and nutrition as...

  36. [44]

    Objective nutrition overview on total energy level and key nutrition features, such as fat, sugar, sodium, protein, whole grains, vegetables

  37. [45]

    Practical suggestions to make this meal and the rest of today’s diet more balanced, for example adjusting portion size, adding vegetables or whole grains, or balancing with lighter meals later in the day

  38. [46]

    Constraints: Do NOT give medical diagnoses, disease risk, or treatment advice

    Encouragement and variety reminder to encourage enjoying the dish in moderation and keeping the diet diverse and balanced instead of focusing on a single meal. Constraints: Do NOT give medical diagnoses, disease risk, or treatment advice. Dietary Advice Fig. S5: Prompt templat...

  39. [47]

    Visual reasoning: describe the key visual cues (shape, color, toppings, context) that help identify the dish

  40. [48]

    Disambiguation: briefly explain how you distinguish this dish from similar categories

  41. [49]

    Food Classification Fig

    Conclusion: state the final matching Food-101 category name. Food Classification Fig. S6: Prompt templates for food classification in the answer-only format and the format with CoT. Recipe Generation.We construct instruction-following data for recipe gen- eration using the sub...

  42. [50]

    Dish identification — infer the dish name from the image

  43. [51]

    Ingredient identification — list visible ingredients and only add common, clearly implied ingredients

  44. [52]

    Ingredient Recognition Fig

    Output alignment — ensure the list in <answer> matches what you reasoned and avoid unusual items. Ingredient Recognition Fig. S7: Prompt templates for ingredient recognition in the answer-only format and the format with CoT. # role definition You are a food and cooking assista...

  45. [53]

    <FullRecipe> should describe the main cooking steps in order as natural text

  46. [54]

    Base the recipe on the dish in the image and typical preparation methods

  47. [55]

    Recipe Generation Fig

    Follow a logical order from preparation to cooking to serving, and focus on the key steps instead of excessive detail. Recipe Generation Fig. S8: Prompt template for image-based recipe generation, where the model outputs an ordered sequence of cooking steps. C.2 Evaluation Det...

  48. [56]

    # reasoning Produce one block wrapped in <answer>...</answer>

    Ingredient Calories # role definition You are a cooking and nutrition assistant. # reasoning Produce one block wrapped in <answer>...</answer>. <answer>The ingredient is <IngredientName>. It weighs <Mass> g and has about <Kcal> kcal in total.</answer> Replace <IngredientName>,...

  49. [57]

    # reasoning Produce one block wrapped in <answer>...</answer>

    Ingredient Nutrition # role definition You are a cooking and nutrition assistant. # reasoning Produce one block wrapped in <answer>...</answer>. <answer>The ingredient is <IngredientName>. It weighs <Mass> g and provides about <Kcal> kcal in total, including <Fat> g of fat, <C...

  50. [58]

    # reasoning Produce one block wrapped in <answer>...</answer>

    Total Nutrition # role definition You are a cooking and nutrition assistant. # reasoning Produce one block wrapped in <answer>...</answer>. <answer>The dish weighs <Mass> g in total and provides about <Kcal> kcal, including <Fat> g of fat, <Carb> g of carbohydrate, and <Protei...

  51. [59]

    # reasoning Answer strictly in the following format: <think> …</think><answer>The ingredient is <IngredientName>

    Ingredient Calories # role definition You are a cooking and nutrition assistant who always responds with a reasoning process and a final answer. # reasoning Answer strictly in the following format: <think> …</think><answer>The ingredient is <IngredientName>. It weighs <Mass> g...

  52. [60]

    # reasoning Answer strictly in the following format: <think> …</think><answer>The ingredient is <IngredientName>

    Ingredient Nutrition # role definition You are a cooking and nutrition assistant who always responds with a reasoning process and a final answer. # reasoning Answer strictly in the following format: <think> …</think><answer>The ingredient is <IngredientName>. It weighs <Mass> ...

  53. [61]

    Total Nutrition # role definition You are a cooking and nutrition assistant who always responds with a reasoning process and a final answer. # reasoning Answer strictly in the following format: <think> …</think><answer>The dish weighs <Mass> g in total and provides about <Kcal...

  54. [62]

    Ingredient identification: state what’s in the dish and why it makes sense for this recipe

  55. [63]

    Quantity estimation: state a concrete amount and unit for each ingredient and justify briefly

  56. [64]

    End with the summed total

    Calorie estimation: For each item, firstly state a reasonable per-unit calorie density and then compute its total calories. End with the summed total. CoT Generation Fig. S11: Prompt template for the CoT-annotated subset of CalorieBench-80K. # role definition You are a cooking...

  57. [65]

    Replace <IngredientName>, <Mass>, <Kcal>, <Fat>, <Carb>, and <Protein> with the ingredient name, its mass in grams, its calories in kilocalories, and its three macronutrients in grams

  58. [66]

    Do not discuss the whole dish or other ingredients

    Mention only this single ingredient. Do not discuss the whole dish or other ingredients. Referring Nutrition Estimation Fig. S12: Prompt template forreferring nutrition estimationon Nutrition5k, where the model estimates the mass, calories, and macronutrients (fat, carbohy- dr...

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.