{"id":"ef62017d-2601-48d1-aba1-94f3a56c7f1e","arxiv_id":"2608.06917","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A structured recipe-graph dataset and a supervised plus reinforcement fine-tuning framework for generating cooking-procedure graphs from food images, with a benchmark showing text quality does not imply procedural structure.","lead":"This paper introduces ReGraph, a dataset of 10,000 recipes converted into structured graphs of ingredients, actions, tools, and cooking relations, plus a two-stage training method that makes vision-language models output such graphs from food images. It matters because it offers a way to check whether recipe-generating AI models actually represent cooking steps, not just fluent text.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ReGraph's headline gains may reflect conformity to a single LLM-derived reference rather than robust procedural improvement; the evaluation protocol's single-reference exact-match scoring is the load-bearing assumption.","rationale":"The reader's weakest assumption is the single-reference, LLM-derived evaluation protocol. I agree that this is the most load-bearing issue, but I want to sharpen it: the problem is not only the single reference, but that the reward used to train RGL-RFT (RIR, Eq. 4) is computed with the same exact-match protocol against training annotations that are only about 89.5% relation-correct. Thus the model is explicitly optimized to imitate the annotation pipeline, and the test metric rewards the same imitation. This internal consistency makes the reported comparison internally valid, but it threatens the external interpretation that RGL improves 'procedural structure' in any absolute sense. The paper's Section 6 limitation statement is to its credit, but the limitation is directly on the central claim, so the conditional verdict is appropriate. I do not regard the lack of error bars or test-set hyperparameter selection as the primary concern: hyperparameter tuning on the test set could inflate small differences, but the main effects (SFT versus baselines, RFT versus SFT) are large enough that they likely survive a proper validation split. The reference-dependence issue, in contrast, could in principle reverse the conclusion about which graph is 'better.' The concrete test of independent multi-reference scoring would settle this. If RGL-RFT beats RGL-SFT across independent references, the central claim is robust; if not, the claim should be rephrased as 'improves alignment with one annotated workflow.'","tokens_in":25006,"tokens_out":5167,"duration_ms":48795,"concrete_test":"Sample 100 test images stratified by category. For each image, have three culinary annotators independently construct a ReGraph-schema graph from the image alone (without the original recipe text or existing annotation), following the same schema definition. Score Qwen3-VL-SFT, Qwen3-VL-RFT, and the best in-context baseline (e.g., Gemini 3.1 Pro 3-shot) against each annotator's graph using the paper's deterministic canonical matching. Report per-reference F1 and the fraction of references on which RFT > SFT and RFT > baseline. If RFT is not better than SFT on a majority of the three references, the single-reference protocol is load-bearing and the central claim needs re-scoping. If RFT wins on all three, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim—that RGL consistently improves generation of cooking entities and procedural relations—depends on the ReGraph reference graph being a valid measure of procedural structure. That reference is produced by Claude-Sonnet 4.5, normalized by GPT-4o, and human-corrected only on the 1,500-recipe test split; the training-split audit reports only 89.5% relation correctness (Table 4). The canonical vocabularies (Section 5.2) are derived from the same training annotations used to train RGL and to compute RIR rewards during RFT (Section 4.2.1), so the reward and evaluation metrics share the annotation pipeline's conventions. Section 6 concedes that 'a procedurally reasonable graph may still be penalized when it differs from this reference.' The supplementary semantic relaxation (Table 10) only expands lexical equivalence within the same reference structure; it does not probe whether alternative graph structures with different action granularity or ordering are equally valid. Consequently, the observed SFT-to-RFT gains and the gap versus baselines may reflect better imitation of the pipeline's idiosyncratic graph style rather than better procedural understanding. The paper's own error analysis (Section 5.7) shows that plausible simplifications (e.g., combining stir-mix-stir into combine-stir) are penalized, confirming the protocol rewards one specific decomposition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ReGraph, a dataset of 10,000 recipes from Recipe1M annotated with structured graphs over ingredients, actions, and tools, using targ, dest, and followed-by relations, entity attributes for ingredient states, and auxiliary Recipe Reasoning Chain-of-Thought traces. It then proposes Recipe Graph Learning (RGL), a two-stage framework: RGL-SFT supervises joint generation of RR-CoT and graph structure, and RGL-RFT applies GRPO with a Relative Improvement Reward that measures entity and relation F1 relative to the SFT model's own baseline, plus a lightweight format reward. Experiments on Qwen3-VL-8B and InternVL3-8B report consistent entity and relation F1 gains after RFT, and a comparison with existing recipe-generation methods shows that high text-generation scores do not translate into reference-aligned procedural structure. The paper includes ablations of the reward design, CoT supervision, scaling factor, and reward weights, along with qualitative and oracle analyses. Section 6 acknowledges the single-reference nature of the evaluation and the underdetermination of the task.","tokens_in":25219,"tokens_out":4230,"duration_ms":47482,"significance":"If the reported gains hold, the paper makes a useful contribution by shifting recipe evaluation from lexical text metrics to explicit procedural structure, and the released dataset and deterministic matching protocol would give the community a reproducible benchmark. The two-stage RGL design is sensible: the RR-CoT auxiliary supervision is a reasonable way to inject procedural decomposition, and the RIR is a principled baseline-relative reward that is not circular, since the baseline is the model's own training-set F1 and all models are scored against a fixed reference. The oracle analysis in Section 5.8 is informative, cleanly separating image-to-procedure inference from text-to-graph formalization. The main weakness is that the headline procedural claims are only as strong as the single-reference annotation protocol, and the hyperparameter choices in Section 5.5 appear to be selected on the test set; both issues need to be addressed before the central comparison can be fully trusted.","major_comments":[{"comment":"The scaling-factor and reward-weight ablations report test-set F1 for each configuration, and no validation split or cross-validation procedure is described anywhere in Section 5.1. Since s=2 and alpha=beta'=1 are then used in the main Table 8 results, the main RFT numbers are selected on the evaluation set, which inflates the reported gains and makes the comparison against baselines optimistic. Please introduce a held-out validation split for hyperparameter selection, or explicitly document that the reported test numbers are the result of test-set sweeping and provide corrected numbers under a valid selection procedure.","section":"Section 5.5, Tables 15 and 16"},{"comment":"The primary metric is exact-match alignment to a single reference graph produced by Claude-Sonnet 4.5, normalized by GPT-4o, and human-corrected only on the 1,500-recipe test split. Section 6 admits that 'a procedurally reasonable graph may still be penalized when it differs from this reference,' and Section 5.7 confirms that a plausible simplification such as combine-stir instead of stir-mix-stir is penalized. The semantic relaxation in Table 10 only widens lexical equivalence within the same reference structure; it does not allow alternative decompositions or orderings. Consequently, the absolute F1 numbers measure conformity to one annotation style rather than procedural correctness. The relative claim that RGL improves under this protocol is defensible, but the Abstract's statement that the graph representation assesses 'whether model outputs encode process-level knowledge' goes beyond what the protocol supports. A multi-reference evaluation, a procedural-equivalence metric, or a reframing of the claims as alignment with the ReGraph reference is needed.","section":"Section 6 and Section 5.2"},{"comment":"All experimental tables report point estimates from what appear to be single training runs, with no standard deviations, confidence intervals, or significance tests. Since RGL-RFT involves stochastic GRPO sampling and LoRA fine-tuning, the reported differences, particularly the relation F1 gap between RIR and raw F1 (8.62 vs. 7.23 in Table 13) and the small in-context differences in Table 8, may be within run-to-run noise. The paper's central claim of consistent improvement would be considerably more secure with variance estimates across multiple seeds or paired tests on the test instances.","section":"Section 5.3, Tables 8-13"},{"comment":"The RFT reward is computed against training annotations whose relation correctness is audited at only 89.5% (Table 4), and the canonical vocabularies in Section 5.2 are derived from the same training annotations. This creates a risk that RIR reinforces annotation-specific errors and that the evaluation vocabulary inherits the training pipeline's normalization choices. The paper does not analyze how training-label noise affects the RFT gains or whether the fixed frequency threshold (canonical labels below aggregate frequency 10 are removed) systematically penalizes rare but valid test ingredients. Please discuss these effects or provide an analysis quantifying their impact on the reported scores.","section":"Section 3.3, Table 4, and Section 4.2.1"}],"minor_comments":[{"comment":"The RGL rows report no SacreBLEU or ROUGE-L scores; since the text-versus-structure comparison is central, clarify whether RGL can produce free-form text at all or provide its text metrics for a fair comparison.","section":"Table 12"},{"comment":"The notation in Equation (5) uses beta' while the text refers to beta-prime; please define the symbol consistently in one place.","section":"Figure 3 and Section 4.2"},{"comment":"The figure uses sub-labels (a) and (b) while the text refers to Case (1) and Case (2); aligning the labels would make the error analysis easier to follow.","section":"Figure 5"},{"comment":"The revision-type percentages in Table 2 are described as non-mutually exclusive, but the table does not state this in its caption; adding that note would prevent misreading the column sum.","section":"Section 3.3"},{"comment":"The description of the semantic verifier is clear, but it would help to state explicitly that the verifier is asked only to accept residual name-level equivalence and cannot override ingredient-state distinctions; this is currently buried in the prompt-summary sentence.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems-and-benchmark contribution, but the test-set hyperparameter selection and the single-reference evaluation protocol are the two issues that most affect the credibility of the headline numbers. The authors seem aware of the second issue in Section 6, which is good, but they do not yet provide the multi-reference or equivalence-based evaluation needed to back the stronger procedural-knowledge claims. I would also ask the editor to verify that the dataset release includes the human-verified test annotations and frozen vocabularies, since those are central to reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the paper contributes a genuinely useful resource: a 10,000-recipe graph dataset with typed procedural relations (targ/dest/followed-by), explicit ingredient-state attributes, and CoT reasoning traces, plus a deterministic matching protocol for evaluation. Second, the headline result — RGL consistently improves entity and relation F1 over baselines on two 8B backbones — is plausible but the evaluation under-reports uncertainty: no validation split for the RFT hyperparameters (they pick s=2 and alpha=beta'=1 on the test set), no error bars or significance tests. Treat the headline numbers as upper bounds until that is fixed.\n\nWhat's actually new is the resource scale. Prior food graphs are smaller or less procedurally complete; ReGraph is at a different scale and makes ingredient-state evolution explicit. The framework design is a sensible recipe: SFT with CoT supervision, then GRPO with a relative-improvement reward. The comparison in Table 12 is the most convincing part — models with high BLEU/ROUGE produce almost no recoverable relational structure, which strongly supports the paper's motivation.\n\nOn the stress-test concern: I do not think the central comparison is circular. The reference is fixed, and all methods are scored the same way; RGL's gains are measured against that fixed reference. The single-reference protocol is a limitation, and the paper says so in Section 6. The error analysis shows that plausible simplifications are penalized, which makes the metric strict rather than invalid. The more pervasive worry is that both the reward and the evaluation come from the same annotation style, so RGL may be rewarded for imitating that style. But the task the paper defines is graph generation under that schema, so imitation is the task. It weakens any deep claim about procedural understanding, but the paper mostly avoids that claim.\n\nThe concrete fixes are straightforward: add a validation split, report means and variances over multiple runs, and hold out all test-set-selected hyperparameters. The relation F1 differences in the ablations are small (8.62 vs 7.23), so significance testing genuinely matters.\n\nWho it is for: anyone working on food computing, structured output from images, or procedural reasoning benchmarks. It deserves a serious referee. The dataset and benchmark alone justify the review effort.\n\nBottom line: accept with major revisions, not because the core is broken but because the evaluation reporting needs to be shored up before the numbers can be trusted.","headline":"Solid dataset and benchmark construction; the training gains are real but the evaluation reporting needs validation-split discipline and variance estimates before the effect sizes can be trusted.","tokens_in":25821,"tokens_out":3164,"would_cite":true,"duration_ms":32735,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fluent recipe text can hide an almost complete absence of recoverable cooking procedure; the paper's ReGraph dataset makes the gap measurable and its two-stage Recipe Graph Learning narrows it.","keywords":["recipe graph","food image","procedural structure","ingredient state","large multimodal models","chain-of-thought","reinforcement fine-tuning","recipe generation"],"falsifier":"A human study in which independent cooks convert the same food image into their own workflow graphs without seeing the ReGraph reference: if RGL's output matches cross-annotator consensus better than it matches the single reference, yet scores lower on the canonical protocol, the metric rewards one annotation style rather than procedural validity.","tokens_in":24747,"feed_emoji":"🍳","tokens_out":10300,"duration_ms":87110,"temperature":0.7,"pith_summary":"The paper tries to establish that generating a recipe from a food image is not the same as understanding the cooking procedure behind it: free-form recipe text can be fluent and lexically close to a gold recipe while encoding almost none of the procedural structure that a graph representation exposes. It introduces ReGraph, a 10,000-recipe dataset that turns cooking instructions into explicit graphs of ingredients, actions, and tools connected by typed relations, with ingredient states carried as entity attributes. On top of it, the paper proposes Recipe Graph Learning (RGL), a two-stage framework that supervises a multimodal model to emit a reasoning trace followed by a graph, then refines the graph with a reward for entity and relation recovery relative to the supervised baseline. Across two 8-billion-parameter backbones, RGL consistently improves entity and relation generation, and the analysis shows that capturing fine-grained ingredient states remains the most difficult dimension.","feed_headline":"Train directly on recipe graphs to expose hidden cooking structure","feed_subtitle":"A 10,000-recipe graph benchmark shows fluent recipe text can carry almost no recoverable procedure; direct graph training recovers more.","key_machinery":"The paper's central object is ReGraph, a graph schema that makes cooking procedure explicit: three entity types (ingredient, action, tool) with ingredient-state attributes, and three typed relations—targ linking a manipulated entity to the action applied to it, dest linking an action to its destination or output entity, and followed by encoding procedural ordering. The carrier of the argument is RGL, a two-stage training loop that first supervises generation of a Recipe Reasoning Chain-of-Thought trace together with the graph, then applies GRPO with a Relative Improvement Reward that scores each candidate's entity and relation F1 relative to the SFT baseline, plus a lightweight format reward. Evaluation uses canonical vocabularies derived from the 8,500 training recipes with exact entity and relation matching, so scores measure alignment with one annotated reference workflow rather than LLM judgment.","core_discovery":"The central discovery is that lexical quality and procedural content come apart in recipe generation: under ReGraph's deterministic canonical matching protocol, existing image-to-recipe models score high on text metrics, yet the procedural structure recoverable from their outputs is near zero, with the best in-context baseline reaching only 3.62% relation F1. Training on ReGraph changes this: RGL's supervised stage already overtakes all in-context baselines, and its reinforcement stage further lifts entity F1 to 30.93% and relation F1 to 8.62% on Qwen3-VL-8B, with consistent gains on InternVL3-8B. The paper also establishes that fine-grained ingredient-state capture is the hardest dimension: state-aware ingredient matching remains at 21.40% F1 after reinforcement, substantially below the 34.66% name-only matching, and the graph formalization itself—not just visual inference—is a major source of difficulty, since even feeding the full ground-truth recipe yields only 18.63% relation F1.","pith_inferences":["The single-reference protocol likely under-credits alternative but equally valid workflows; a multi-reference or procedural-equivalence evaluation would change absolute scores, though it would probably preserve the relative ordering of methods.","The same graph-supervision recipe may transfer to other procedural domains, such as assembling furniture, laboratory protocols, or repair steps, where free-form instructions hide structure in the same way.","Because the canonical vocabularies and the reference graphs are produced by LLMs and then human-normalized, the metric inherits their annotation granularity; re-deriving the vocabularies from different seeds or different LLMs would test how much of the measured gap is annotation-style dependent."],"forward_implications":["Text-based evaluation of recipe generation overstates model quality: high SacreBLEU and ROUGE-L scores can coexist with near-zero recoverable entity and relation structure.","Directly training models to output structured graphs, rather than converting free-form text afterwards, is the more effective route to reference-aligned procedure recovery.","Reinforcement fine-tuning with a baseline-relative reward improves relational structure, with the largest gains on the followed by ordering relations.","Ingredient-state tracking is the binding constraint for image-to-recipe systems; identifying ingredients is far easier than knowing what happens to them."],"supporting_citations":[{"why":"Recipe1M is the source corpus from which all 10,000 ReGraph recipes are sampled.","marker":"[28]"},{"why":"Inverse Cooking supplies the ingredient vocabulary and synonym mapping used in the canonical matching protocol.","marker":"[27]"},{"why":"Claude Sonnet 4.5 generates the initial RR-CoT traces and raw entity and relation extractions in the annotation pipeline.","marker":"[1]"},{"why":"GPT-4o performs schema normalization of the raw extractions into the final triple format.","marker":"[11]"},{"why":"The GRPO algorithm from this work is used in the reinforcement fine-tuning stage of RGL.","marker":"[29]"},{"why":"RecipeRAG is a state-of-the-art recipe generation baseline whose fluent text, when converted to graphs, yields only 3.30% relation F1.","marker":"[37]"},{"why":"SGRG is a baseline with semantic-level text evaluation whose recipes carry limited recoverable structure under the ReGraph schema.","marker":"[18]"},{"why":"SacreBLEU is the text metric used to show that lexical similarity does not imply procedural structure.","marker":"[24]"}],"fun_headline_variants":["Recipe text hides broken cooking logic; graph training fixes it","Fluent recipes, empty workflows: graph benchmark exposes gap","Train on recipe graphs to recover real cooking steps","Graph training recovers cooking structure that text misses","Even ground-truth recipes fail graph recovery, says ReGraph"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result rests on the premise that the single ReGraph reference annotation per recipe, together with the canonical matching protocol, is a faithful measure of procedural correctness rather than a measure of conformity to one LLM-driven annotation style.","fun_headline_variants_meta":{"raw":{"variants":["Recipe text hides broken cooking logic; graph training fixes it","Fluent recipes, empty workflows: graph benchmark exposes gap","Train on recipe graphs to recover real cooking steps","Graph training recovers cooking structure that text misses","Even ground-truth recipes fail graph recovery, says ReGraph"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000383,"raw_usage":{"total_tokens":2074,"prompt_tokens":1039,"completion_tokens":1035,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":655,"completion_tokens_details":{"reasoning_tokens":957}},"tokens_in":655,"tokens_out":1035,"duration_ms":9404,"temperature":1.0,"reasoning_tokens":957,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:24:25.073106+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A human study in which independent cooks convert the same food image into their own workflow graphs without seeing the ReGraph reference: if RGL's output matches cross-annotator consensus better than it matches the single reference, yet scores lower on the canonical protocol, the metric rewards one annotation style rather than procedural validity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Recipe1M is the source corpus from which all 10,000 ReGraph recipes are sampled."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Inverse Cooking supplies the ingredient vocabulary and synonym mapping used in the canonical matching protocol."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Claude Sonnet 4.5 generates the initial RR-CoT traces and raw entity and relation extractions in the annotation pipeline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GPT-4o performs schema normalization of the raw extractions into the final triple format."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RecipeRAG is a state-of-the-art recipe generation baseline whose fluent text, when converted to graphs, yields only 3.30% relation F1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SGRG is a baseline with semantic-level text evaluation whose recipes carry limited recoverable structure under the ReGraph schema."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SacreBLEU is the text metric used to show that lexical similarity does not imply procedural structure."}],"review_version":1}