{"id":"9c86394c-e383-4693-9d05-2b04cc25680f","arxiv_id":"2501.09042","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CookingDiffusion generates step-by-step cooking images by feeding previous step texts and images as memory into Stable Diffusion, beating baselines on FID and a new CLIP-based consistency score.","lead":"This paper introduces cooking procedural image generation, the task of producing a consistent image for every step of a recipe, and presents CookingDiffusion, a Stable Diffusion model that conditions on previous step texts and images through three memory modules. The authors also build a YouCookII-based benchmark and a new CLIP-based consistency metric, and release no artifacts yet.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Avg-PCon does not measure cross-image procedural consistency: it is a weighted CLIP text-alignment score that omits self-step similarity and can be gamed by generic or repeated images.","rationale":"The paper's main contributions are the new task formulation and the three Memory Nets, and the FID improvements over fine-tuned Stable Diffusion (e.g., 24.336 vs. 29.585 in Scenario 1, Table 1) are plausible evidence of image quality. What is load-bearing is the consistency claim, because the paper explicitly argues that FID cannot measure consistency and therefore introduces Avg-PCon as the dedicated metric. Section 5.2.2 defines Avg-PCon as a weighted CLIP score between each generated image and the other steps' texts; it never uses image-image comparisons, it omits the current step's own text, and it downweights cross-step comparisons when the text descriptions are dissimilar. Since the model's memory nets are built from CLIP text and image encoders, the evaluation metric shares the same representation family as the conditioning mechanism, creating a self-referential evaluation. The oracle setting in scenarios 2 and 3 further limits the claim: those numbers are upper bounds generated with ground-truth historical images, not full-sequence generation. The concern is addressable with a control experiment or human evaluation, so the appropriate outcome remains the reader's CONDITIONAL verdict rather than ACCEPT or REJECT; hence I recommend no change to the reader's verdict.","tokens_in":17624,"tokens_out":7080,"duration_ms":76280,"concrete_test":"Recompute Avg-PCon under a control baseline that outputs the same fixed image for every step of every recipe, where the fixed image is, for example, the image corresponding to the mean CLIP embedding of the training set or a generic cooking scene. If this repeated-image control achieves Avg-PCon close to or above CookingDiffusion's reported values in Table 1, then Eqs. 7-8 cannot distinguish a procedurally consistent sequence from a static one, and the headline consistency claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CookingDiffusion generates procedurally consistent cooking images. The only dedicated evidence for this is Avg-PCon (Sec. 5.2.2, Eqs. 7-8), which is a weighted sum of CLIP similarities between each generated image and the texts of the other steps, with weights given by normalized text-text similarities. The metric never compares images to one another, so it cannot detect whether the same pan, ingredients, or background persist across steps. It also omits the self-term <I_gen_i, C_i>, so an image that fails to match its own step can still score high if it matches other steps. Its text-text weighting actively downweights the cross-step comparisons where visual persistence matters most: when two step texts describe different actions or ingredients, the cross-term is suppressed even though a consistent procedure should still show the same kitchen state. A model can therefore inflate Avg-PCon by producing images that are semantically generic or aligned with the recipe's text cluster, without any sequential visual progression. The self-referential character of the metric compounds this: the model's Text and Image Memory Nets operate on the same CLIP embedding family used by Avg-PCon, so the reported consistency advantage may reflect alignment with the metric rather than with human-perceived procedure consistency. Additionally, the scenario 2 and 3 results use ground-truth historical images as procedural prompts, which the paper acknowledges is an upper bound; those numbers cannot support a deployment claim about generating full sequences. The FID results are plausible evidence of image quality, but the consistency half of the central claim stands or falls with Avg-PCon.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'cooking procedural image generation,' in which a recipe's step texts are used to generate a sequence of images that are both faithful to each step and mutually consistent. The proposed CookingDiffusion extends Stable Diffusion with three Memory Nets: Text Memory Net (TMN), Image Memory Net (IMN), and Multi-modality Memory Net (MMN), which inject procedural representations from previous step texts, previous step images, or their mixture into the denoising U-Net. The authors preprocess YouCookII into 1185 training and 446 validation recipes and benchmark three scenarios: text-only procedural prompts, image-only procedural prompts, and mixed prompts with a fraction p of available images. They report FID and a new Average Procedure Consistency (Avg-PCon) metric, and claim that CookingDiffusion outperforms fine-tuned Stable Diffusion and adapted StackGAN, VQ Diffusion, and ControlNet baselines on both metrics, with additional qualitative ingredient and cooking-method manipulation results.","tokens_in":1362,"tokens_out":1627,"duration_ms":80941,"significance":"If the claim holds, this is a useful task formulation and a simple, modular contribution to conditional generation: the zero-initialized additive memory path preserves pre-trained Stable Diffusion behavior, the unified CLIP-based design lets TMN, IMN, and MMN share structure across modalities, and the preprocessed YouCookII benchmark plus the promised release would support future work. The FID improvements on the validation set are substantively large (e.g., 34.547 versus 40.394 for scenario 1 in Table 1), which gives independent evidence for image-quality gains. However, the paper's headline consistency evidence rests entirely on Avg-PCon, whose validity as a measure of procedural consistency is not established; the consistency claim therefore needs additional support before the contribution can be fully credited.","major_comments":[{"comment":"The Avg-PCon metric does not compare generated images with one another; it is a weighted average of CLIP text-image similarities between each generated image and the texts of the other steps. It therefore cannot detect persistence of the same pan, ingredients, or background, which is the sequential consistency the paper claims. Because the self-term <I_gen_i, C_i> is omitted, an image that is a generic match to the recipe's other steps can score highly even if it does not depict its own step, and the text-text weights suppress exactly the cross-step comparisons where a consistent kitchen state should remain visible despite different action texts. The central consistency claim should not rest on this metric unless it is validated against human judgments or an image-image consistency measure.","section":"Section 5.2.2, Eqs. (7)-(8)"},{"comment":"There is a circularity concern: Avg-PCon is computed with CLIP text and vision encoders, and the same CLIP embedding family is used by the Text, Image, and Multi-modality Memory Nets as well as by the keyframe selection in Section 5.1. A model trained to align with CLIP-based procedural representations may therefore score higher on Avg-PCon without being more consistent for human viewers. The paper should add an independent evaluation, e.g., a human study or a metric using features not involved in training, to support the headline consistency result.","section":"Sections 3.3-3.5 and 5.2.2"},{"comment":"In Scenarios 2 and 3 the historical image prompts are ground-truth keyframes, and the paper itself states in Section 3.2 that this is an upper bound on the improvement obtainable from image-based procedural prompts. The experiments therefore do not exercise the actual deployment setting in which previously generated images would be fed back as context, and the scenario-3 goal of 'generating the missing images for all the steps' is not evaluated end-to-end. Either an autoregressive evaluation with generated images should be reported, or the claims should be explicitly limited to conditioning on ground-truth images.","section":"Section 3.2 and Sections 5.3-5.4"},{"comment":"All quantitative results are single runs without error bars or significance tests, and several Avg-PCon differences in Table 1 are very small (e.g., 18.541 versus 18.470 on the validation set in Scenario 1). It is therefore unclear whether the reported consistency improvements are statistically meaningful. The authors should report means and standard deviations over multiple seeds and include significance tests; they should also avoid using FID computed on the training set as evidence of superiority, since it is not a generalization measure.","section":"Section 5.3, Table 1"}],"minor_comments":[{"comment":"The heading 'Multi-modalilty Memory Net' contains a typo, and the text later uses 'Avg-PCON' inconsistently; both should be corrected to 'Multi-modality Memory Net' and 'Avg-PCon'.","section":"Section 3.5 and Section 5.3"},{"comment":"The term 'normalized <C_i, C_j>' is never defined; the paper should specify the exact normalization (e.g., softmax over j) and any temperature, since this weighting determines how much the cross-step text-image terms are downweighted.","section":"Equation (7)"},{"comment":"The statement that the authors evaluate on both the validation and training sets 'due to a perceived lack of samples' is vague; the paper should report the number of procedures and explain what the training-set FID is intended to show.","section":"Section 5.1"},{"comment":"The adapted baselines differ in optimization scope (ControlNet freezes the Stable Diffusion backbone while CookingDiffusion fine-tunes it); a sentence acknowledging this asymmetry and its effect on the comparison would improve the fairness of the benchmark.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, it defines a genuinely new task—generating a sequence of cooking images, one per recipe step, with visual consistency across steps—and builds a benchmark from YouCookII. Second, its headline consistency metric, Avg-PCon, does not actually measure cross-image consistency. That mismatch is the paper's main problem.\n\nWhat's new and useful: the task framing is clean, and the three memory nets are a sensible way to condition Stable Diffusion on prior steps' text or images. The authors honestly note that scenarios 2 and 3 use ground-truth historical images, so those numbers are an upper bound. The FID gains over fine-tuned Stable Diffusion are plausible, and the manipulation examples (adding/replacing ingredients) are a nice qualitative bonus.\n\nThe soft spots are real. Avg-PCon (Eqs. 7–8) is a weighted sum of CLIP similarities between each generated image and the other steps' text, weighted by text-text similarities. It never compares images to each other, so it can't tell whether the same pan, ingredients, or background persist. It also omits the self-term, so a generic image that matches other steps' text can score high. Since the model itself uses CLIP encoders, the metric is partially self-referential. If the paper's central claim is \"consistent across sequential cooking steps,\" Avg-PCon doesn't support it.\n\nThere are also no error bars or significance tests anywhere; all tables report single runs. And the paper hasn't released code or data yet, so the benchmark can't be independently checked.\n\nThat said, these are fixable. The task and benchmark are worth having. The authors should replace Avg-PCon with an image-image consistency measure (e.g., pairwise LPIPS or DINO feature similarity with a temporal weighting), add a human evaluation, and run at least a few seeds for variance. For scenarios 2 and 3, they should show at least one autoregressive generation from a full text-only sequence to demonstrate the actual use case.\n\nWho should read this? Anyone in food computing or text-to-image generation who wants a concrete example of a procedural conditioning setup, and anyone who cares about why a metric like Avg-PCon can be misleading. It deserves a serious referee, but the consistency claim needs to be re-examined before it's accepted.\n\nMy recommendation: send to review, but flag the metric flaw prominently. If the authors fix the evaluation, this becomes a solid contribution.","headline":"Useful new task and benchmark, but the Avg-PCon metric measures text alignment rather than cross-image visual consistency, so the headline claim is not yet supported.","tokens_in":18488,"tokens_out":2867,"would_cite":false,"duration_ms":26348,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper presents CookingDiffusion, a Stable Diffusion-based model that generates a consistent image for every step of a cooking recipe by conditioning on the history of previous steps, and reports that it beats standard baselines on…","keywords":["cooking procedural image generation","procedural prompts","diffusion models","Stable Diffusion","Memory Net","text-to-image generation","recipe consistency","YouCookII benchmark"],"falsifier":"Take the same generated procedures and compute a direct image-to-image consistency measure, such as average CLIP similarity between generated images of the same recipe, or collect pairwise human judgments asking whether each image logically follows from the previous one; if the ranking of methods under Avg-PCon does not match the ranking under this direct measure, the paper's central consistency claim is not supported.","tokens_in":17413,"feed_emoji":"🍳","tokens_out":8168,"duration_ms":78829,"temperature":0.7,"pith_summary":"This paper introduces a new task: given a multi-step cooking recipe, generate one photo-realistic image per step, so the whole sequence reads as one continuous cooking procedure. To do this, it proposes CookingDiffusion, a Stable Diffusion-based model that adds a Memory Net to draw on earlier steps' text or images (or both) while generating the current step. The authors build a benchmark from the YouCookII cooking-video dataset and report that CookingDiffusion improves over fine-tuned Stable Diffusion, StackGAN, VQ Diffusion, and ControlNet baselines on Fréchet Inception Distance and on their proposed Average Procedure Consistency score. They also show the model can edit ingredients and cooking methods in the generated images. The claim is that procedural prompts, meaning the history of the recipe, are what make the difference.","feed_headline":"CookingDiffusion keeps generated recipe images consistent step by step","feed_subtitle":"Recipe apps could auto-illustrate instructions and fill in missing step photos with matching visuals.","key_machinery":"The operative mechanism is a procedural Memory Net: a thin module that builds a representation of all previous steps and injects it into the diffusion model's time embedding. For text history it uses a CLIP text encoder, for image history a CLIP vision encoder, and for mixed history both encoders are aligned and processed together. A masked self-attention block over the encoded sequence lets the model attend to earlier steps, and a zero-initialized linear layer preserves the pre-trained Stable Diffusion's behavior at the start of training. The paper also introduces Average Procedure Consistency, a CLIP-based score that weights each generated image's similarity to other steps' texts by the normalized similarity of the texts themselves, as the task-specific evaluation of procedural consistency.","core_discovery":"CookingDiffusion's central discovery is that feeding the diffusion model the history of a recipe, not just the current step's text, makes generated step images more consistent without sacrificing image quality. The history is encoded by three Memory Nets: a Text Memory Net for previous step descriptions, an Image Memory Net for previous step images, and a Multi-modality Memory Net for mixed text-image history. Each Memory Net encodes the history with CLIP-based text or vision encoders, applies masked self-attention over the sequence to form a procedural representation, and injects it into the time embedding of Stable Diffusion through a zero-initialized linear layer. The model is evaluated on a preprocessed YouCookII benchmark and, according to FID and Avg-PCon, outperforms the adapted baselines in both text-only and image-history scenarios. The paper further shows that editing the step text changes the generated content coherently, for example by removing, adding, or replacing ingredients and by changing cooking methods.","pith_inferences":["The same memory-injection design could be applied to other step-by-step visual domains, such as assembly instructions, lab protocols, or repair guides, where procedural consistency matters more than temporal continuity.","Because the image-history scenarios use ground-truth earlier images, a deployed system that re-feeds its own generated images would have to run autoregressively; measuring how much error accumulates in that loop is a direct test the paper does not perform.","A natural next test is whether a direct image-to-image consistency measure, for example pairwise similarity between generated step images, ranks methods the same way Avg-PCon does; that would clarify what part of the reported consistency gain is visual coherence.","The paper leaves temporal modeling out of scope; extending the same procedural-memory mechanism to step-wise video generation is an explicit future direction."],"forward_implications":["Text-only procedural history already improves consistency over the current-step-only baseline, so recipe sites can illustrate instructions step by step even when no step photos exist.","When ground-truth images of earlier steps are available, image-based memory gives the best reported FID and consistency scores, making visual history the strongest tested consistency signal.","The multi-modal Memory Net can generate images for steps that lack photos, so a partially illustrated recipe can be turned into a full visual guide.","Ingredient and cooking-method manipulations transfer to the generated images, so interactive recipe editing could update step visuals when a substitution or method change is made.","Adapting procedural context into StackGAN, VQ Diffusion, or ControlNet does not reproduce the same trade-off, indicating the benefit depends on how the memory is injected, not just on having context."],"supporting_citations":[{"why":"Supplies the pre-trained text-to-image generator and UNet backbone that CookingDiffusion modifies with its Memory Nets.","marker":"[35]"},{"why":"Provides the YouCookII cooking videos, step annotations, and timestamps that are preprocessed into the paper's benchmark.","marker":"[59]"},{"why":"Supplies the CLIP text and vision encoders used for Memory Net encodings, keyframe selection, and the Avg-PCon metric.","marker":"[31]"},{"why":"Defines the FID metric used to measure generated image quality against real step images.","marker":"[12]"},{"why":"Serves as a GAN-based baseline that is adapted with procedural representations for comparison.","marker":"[54]"},{"why":"Serves as a diffusion-based baseline that is adapted with the same TMN and IMN modules for comparison.","marker":"[10]"},{"why":"Serves as a conditioning-based baseline whose projection and downblock are modified to consume procedural prompts.","marker":"[56]"}],"fun_headline_variants":["CookingDiffusion turns recipe steps into consistent images","Memory nets keep AI cooking images in step","Step sequence becomes one coherent visual with CookingDiffusion","CookingDiffusion syncs generated images to recipe history","Recipe history drives consistent step images in CookingDiffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's whole consistency claim stands on the assumption that its Average Procedure Consistency score, which is computed by matching each generated image against the descriptions of the other steps and never compares images with images, actually measures whether a set of images forms one coherent cooking procedure.","fun_headline_variants_meta":{"raw":{"variants":["CookingDiffusion turns recipe steps into consistent images","Memory nets keep AI cooking images in step","Step sequence becomes one coherent visual with CookingDiffusion","CookingDiffusion syncs generated images to recipe history","Recipe history drives consistent step images in CookingDiffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000268,"raw_usage":{"total_tokens":1643,"prompt_tokens":995,"completion_tokens":648,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":573}},"tokens_in":611,"tokens_out":648,"duration_ms":7168,"temperature":1.0,"reasoning_tokens":573,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:20:57.358924+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same generated procedures and compute a direct image-to-image consistency measure, such as average CLIP similarity between generated images of the same recipe, or collect pairwise human judgments asking whether each image logically follows from the previous one; if the ranking of methods under Avg-PCon does not match the ranking under this direct measure, the paper's central consistency claim is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the YouCookII cooking videos, step annotations, and timestamps that are preprocessed into the paper's benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the FID metric used to measure generated image quality against real step images."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as a GAN-based baseline that is adapted with procedural representations for comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as a diffusion-based baseline that is adapted with the same TMN and IMN modules for comparison."}],"review_version":1}