{"id":"680ca738-dfcd-4345-9b24-66ee2dfa1351","arxiv_id":"2505.14629","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"KERL uses a food knowledge graph and three LoRA adapters on one LLM to recommend constrained recipes, generate cooking instructions, and produce micro-nutrition details.","lead":"This paper builds KERL, a food recommendation system that combines a food knowledge graph with a fine-tuned language model to answer constrained recipe questions, then generates cooking steps and nutrition facts. The authors also release template-generated benchmarks for recipe recommendation, recipe generation, and nutrition estimation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 5's headline gain is confounded: KERL-Recom receives retrieved KG context while baselines do not, and no deterministic KG-query baseline is reported; a context-matched control is needed before claiming KG-augmented LLM superiority.","rationale":"The reader's stated weakest assumption is that the benchmark is synthetic and template-based, limiting external validity. That is a real limitation, but the more load-bearing issue for the paper's central claim is the internal comparison in Table 5: KERL-Recom gets retrieved KG context while baselines do not, and no exact KG-query resolver is included. The reader's rationale does mention that 'the central comparison needs a same-context, no-LoRA baseline,' which is the same control I identify, but it is not the weakest_assumption field. I therefore mark agreement as partial. This concern is specific to the recommendation module; the recipe and nutrition evaluations (Tables 8 and 9) compare models on the same input modalities and are less confounded, though they still lack error bars. The paper deserves credit for releasing code and benchmark data, and for reporting per-tag results, but the headline recommendation result cannot be interpreted as evidence for KG-augmented LLM superiority until the context-matched baseline or deterministic baseline is added. Since the reader already reached CONDITIONAL, my read does not change the verdict; it sharpens the condition under which the central claim should be accepted.","tokens_in":21216,"tokens_out":6370,"duration_ms":59695,"concrete_test":"Re-run the KGQA evaluation in Table 5 with two additional rows on the same test set: (i) Phi-3-mini-128K zero-shot, given the exact same serialized subgraph context Cj that KERL-Recom receives plus the same question; and (ii) a deterministic baseline that applies the template's ingredient and nutrition constraints directly as SPARQL filters or as a post-filter on the retrieved subgraph's nutrient values. If either baseline reaches F1 comparable to KERL-Recom's 0.973, the reported improvement is attributable to retrieval/query resolution rather than to the LoRA-trained KG-augmented selection model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that KERL-Recom significantly outperforms existing approaches rests on Table 5, where KERL-Recom reaches F1=0.973 versus Phi-3-mini-128K's 0.410 and Llama-2's 0.713. The comparison is not apples-to-apples. In Section 3.1, KERL-Recom inference supplies the model with a serialized subgraph containing candidate recipe names, ingredients, and nutrient values, retrieved by template SPARQL queries from FoodKG. The pretrained baselines receive only the user question and must recall recipe titles from parametric memory. The benchmark is generated by the same template pipeline (Sections 4.1-4.2), so every ground-truth recipe is present in the retrieved candidate set and all nutrient values needed to filter are in the context. Under these conditions, a zero-shot LLM may be able to select correct recipes by reading the supplied nutrition and ingredient lists, meaning the LoRA adapter may contribute little. Conversely, because the ground-truth set R+(tj) is defined by the same constraint filters used to generate the benchmark, a deterministic SPARQL/post-filter baseline over FoodKG would be near-perfect by construction; the paper does not report such a baseline. The paper's own Limitations (Section 7) acknowledge failures when context is wrong or empty, but do not address this control. Therefore the reported 56- and 26-point F1 improvements, and the conclusion that the KG-augmented LLM significantly outperforms existing approaches, may reflect an asymmetric evaluation rather than a genuine advantage of the proposed method.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents KERL, a unified food system built on Phi-3-mini with separate LoRA adapters for recipe recommendation (KERL-Recom), recipe generation (KERL-Recipe), and nutrition generation (KERL-Nutri). Recommendation is framed as constrained question answering over FoodKG: template-based entity and constraint parsing generates SPARQL queries, retrieved recipe subgraphs are serialized into the LLM context, and the LoRA adapter selects recipe names satisfying the constraints. The authors also construct a 77,900-question KGQA benchmark from FoodKG tags, ingredients, and nutrient thresholds, and evaluate recipe-generation and nutrition benchmarks derived from Recipe1M. The headline result is that KERL-Recom reaches F1 0.973 on the KGQA test set, outperforming pretrained LLMs by 26-56 F1 points, and that the recipe and nutrition adapters improve over Phi-3 baselines.","tokens_in":21560,"tokens_out":5160,"duration_ms":45286,"significance":"If the recommendation comparison were clean, KERL would be a practical unified system and the KGQA benchmark would be a useful resource. Strengths include public code and benchmark datasets, a multi-LoRA design that shares one base model, and standard, clearly described recipe-generation and nutrition experiments. However, the central evidence for KG-augmented recommendation is currently confounded: the recommendation model is tested with retrieved KG context while the baselines are not, and no deterministic KG-query control is reported. The core scientific claim about the value of KG augmentation therefore requires additional experiments; with those controls the paper could make a solid contribution.","major_comments":[{"comment":"The comparison between KERL-Recom and pretrained LLMs is not apples-to-apples. Section 3.1 states that KERL-Recom receives, for each inference call, the user query plus a serialized subgraph containing candidate recipe names, ingredient lists, and nutrient values retrieved by SPARQL from FoodKG. The baselines in Table 5 are described only as pretrained LLMs, and the paper does not state that they are given the same subgraph context; the natural reading is that they receive the question alone and must recall recipe titles from parametric memory. Because the KGQA benchmark is generated by the same template/constraint pipeline (Sections 4.1-4.2), every ground-truth recipe name and all values needed for constraint filtering are present in KERL-Recom's context, so a zero-shot model with that context could be near-perfect without LoRA. The reported 56-point and 26-point F1 gains therefore do not yet establish that the KG-augmented LLM, rather than the retrieval step, drives the improvement. Please add (i) a context-matched zero-shot baseline: the same serialized subgraphs and query given to Phi-3-mini, and ideally to the other LLMs, without LoRA; and (ii) an ablation with the same context but no adapter.","section":"§5.2, Table 5"},{"comment":"No deterministic KG-query baseline is reported. Since R+(tj) is defined by the very ingredient-inclusion/exclusion and nutrient-threshold filters encoded in the SPARQL templates, a SPARQL query plus post-filter over FoodKG would be near-perfect by construction on this test set. Reporting this baseline is necessary to calibrate how much value the LLM selection adds over straightforward retrieval. As the paper stands, the conclusion that 'our proposed KG-augmented LLM significantly outperforms existing approaches' (Abstract) is not supported without this control.","section":"§5.2, Table 5; §4.1-4.2"},{"comment":"The benchmark's template-generated queries do not support the unqualified claim of a complete and coherent solution for personalized food recommendation. Every test question is generated by filling placeholders with exact FoodKG ingredient names and one of 15 closed-set tags, and the system parses entities from these same templates. Section 7 itself concedes that the system fails when context is empty or wrong. The claims about personalization would be strengthened by at least a small evaluation on paraphrased or free-form queries, synonym or misspelling variants, or tags outside the fixed list; as written, the system's behavior on realistic open-ended user input is untested.","section":"§4.1-4.2 and §7"}],"minor_comments":[{"comment":"The per-tag results for KERL-Recom are labeled 'KERL-Nutri' in the model column (e.g., the lactose row with F1=0.935), which is inconsistent with the surrounding text and should be corrected.","section":"Appendix D.1, Table 13"},{"comment":"The row label 'KERN-Recom' is a typo for KERL-Recom.","section":"Table 7"},{"comment":"The Phi-3 Xt+Xing row is malformed: the numeric columns are merged ('0.0380.2160.042') and the final perplexity entry is incomplete ('0. 11.99'), making that row unreadable.","section":"Table 8"},{"comment":"There are copyediting errors, including 'taks' for 'task' in the Introduction and 'The KERL-Recipe modules generates' in Section 3.1; these should be fixed.","section":"Introduction and Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The evaluation asymmetry in Table 5 is the main barrier to acceptance; the remaining experiments are standard and reproducible in principle. I do not see a novelty-disclosure issue, though the KGQA benchmark's provenance from FoodKG, which is prior work by one of the authors, should be made explicit in the dataset documentation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the recipe and nutrition pieces are solid and worth a look; the recommendation headline, as advertised, does not survive a careful read. The 56-point and 26-point F1 gains in Table 5 are very likely an artifact of asymmetric evaluation.\n\nWhat's new: a single Phi-3-mini base model with three LoRA adapters for recommendation, recipe generation, and nutrition; two template-generated benchmarks (77.9k KGQA pairs, ~500k nutrition samples). The integration is real, and the recipe-generation results (Table 8) beat LLaVA-Chef on most metrics with far fewer trainable parameters. The nutrition MAE numbers (Table 9) are consistently lower than the LLaVA-Chef baseline. Those parts are clearly described and reproducible in spirit.\n\nWhere it sags: the recommendation comparison. In Section 3.1, KERL-Recom gets a serialized subgraph containing candidate recipe names, ingredients, and nutrient values; the baselines get only the query. The KGQA benchmark is generated from the same template constraints used to define R+(tj), so the correct recipes are contained in the retrieved subset, and the nutrient values needed to filter are right there in the context. Without a no-LoRA baseline that also receives the same context, you cannot attribute the F1 improvement to the learned adapter. A deterministic SPARQL query with post-filtering would also be near-perfect by construction, and it is not included. The paper's Limitations section acknowledges sensitivity to wrong/empty context but does not address this control. Given that the abstract and conclusion make the strong claim that 'KG-augmented LLM significantly outperforms existing approaches,' the evaluation needs a matched-input control.\n\nMinor things: Table 8 contains a broken value for Phi-3 Xt+Xing perplexity; the pFoodReq table has a typo (KERN-Recom); the dairy-free tag has only 7 recipes, so per-tag F1 of 0.667 is not meaningful.\n\nBottom line: this is a useful applied-systems paper with two new benchmarks and a reasonable multi-LoRA architecture. The recipe and nutrition results stand on their own. The recommendation claim needs a serious rework of the comparison before the central message is credible. I'd send it to peer review, but I'd send it back with a required same-context no-LoRA baseline and a KG-query baseline.","headline":"Solid recipe and nutrition modules; the recommendation headline is confounded by an unfair context comparison and needs a matched baseline before the strong claims hold.","tokens_in":22045,"tokens_out":2713,"would_cite":false,"duration_ms":32857,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A compact LLM grounded in a food knowledge graph beats much larger models on recipe recommendation, generation, and nutrition.","keywords":["food recommendation","knowledge graph","large language models","LoRA","retrieval-augmented generation","recipe generation","nutrition estimation","question answering over knowledge graphs"],"falsifier":"Construct a test set of paraphrased and free-form versions of the same constraints—using synonyms for ingredients, alternative tag names, and misspellings—and measure how often the template SPARQL retrieval returns a correct candidate set before the LLM is asked to rank. If a large fraction of such inputs yield empty or wrong retrieval, the reported F1 on template-generated questions would not reflect real-world use, and the central claim that the system is a complete personalized recommender would be undermined.","tokens_in":21031,"feed_emoji":"🍲","tokens_out":12098,"duration_ms":98788,"temperature":0.7,"pith_summary":"The paper tries to establish that a single compact LLM—Phi-3-mini—combined with the FoodKG knowledge graph and three task-specific LoRA adapters can handle the full job of personalized meal assistance: recommending recipes under ingredient and nutrition constraints, writing cooking instructions, and reporting micronutrient values. On the authors' newly built 77,900-question benchmark, the recommendation adapter reaches F1 0.973, a 56-point gain over the 128K-context Phi-3-mini base and a 26-point gain over the much larger Llama-2-7B. The recipe and nutrition adapters also beat their baselines, including the multimodal LLaVA-Chef. Because the authors release their code and benchmark datasets, the claim is checkable rather than merely asserted.","feed_headline":"KG-augmented 3.8B LLM beats larger models on food tasks","feed_subtitle":"KERL reports F1 0.973 on constrained recipe recommendation, 26 points above Llama-2-7B, plus gains in recipe and nutrition generation.","key_machinery":"The load-bearing mechanism is retrieval-then-rank: a template-driven entity parser and SPARQL generator first restrict the universe to recipes carrying one of 15 health tags and matching ingredient filters, and the LLM's job is to select, from those retrieved subgraphs, the recipes that satisfy all constraints. Each of the three tasks—recommendation, recipe generation, and nutrition generation—is a separate LoRA (low-rank adaptation) adapter on the same Phi-3-mini-128K backbone, so one deployed base model can switch adapters per task and run efficiently on a single GPU. The knowledge graph supplies both the candidate set and the ground-truth answers, which is what lets a small fine-tuned model outrank much larger pretrained baselines.","core_discovery":"The central claim is that retrieving recipe subgraphs from FoodKG and feeding them to an LLM as context—rather than asking the model to answer from memory—is what makes constrained food question answering work. KERL-Recom parses a user query into a health tag, liked ingredients, disliked ingredients, and numeric nutrient limits, converts these into a template SPARQL query, retrieves the matching recipe subgraphs, and is trained with a LoRA adapter to output only recipes that satisfy every condition. The paper reports F1 0.973 on its KGQA test set versus 0.41 for Phi-3-mini-128K and 0.713 for Llama-2-7B, and on the older pFoodReq benchmark it reports F1 0.854 versus 0.637 for pFoodReq itself. For generation, a second adapter produces cooking steps with higher BLEU, ROUGE, METEOR, and CIDEr scores than LLaVA-Chef while training far fewer parameters, and a third adapter lowers mean average error across ten micronutrients, especially when given the title, ingredients, and instructions together.","pith_inferences":["Editorial inference: Because the benchmark questions are generated by filling the same templates the system parses, the reported F1 may overestimate performance on free-form user language; a human-written or paraphrased test set would show whether the KG grounding helps when entity parsing is harder.","Editorial inference: The multi-LoRA-plus-KG recipe could transfer to other constrained recommendation domains—such as travel, shopping, or medical meal planning—where a knowledge graph supplies candidates and each task gets its own adapter.","Editorial inference: The system's coverage is bounded by FoodKG's tag and ingredient vocabulary; enriching the graph or adding an ingredient-substitution mapping would extend the system to queries the current templates cannot parse.","Editorial inference: Because the recommendation module can only select within retrieved subgraphs, any SPARQL retrieval failure is unrecoverable; training a learned retriever on free-form questions could make the pipeline more robust than fixed templates."],"forward_implications":["A 3.8B-parameter model with three task-specific adapters can outperform much larger pretrained LLMs on constrained recipe recommendation, indicating that knowledge-graph grounding plus task-specific tuning can matter more than parameter count for this task.","The same base model with switchable adapters can serve recommendation, cooking-step writing, and nutrition analysis in one deployment, which is more practical than running separate specialized systems.","Retrieving subgraphs from FoodKG first means any recommended recipe already satisfies the stated ingredient and nutrient constraints whenever such a recipe exists in the graph, giving hard constraint satisfaction rather than free-form guesswork.","The new 77,900-question KGQA benchmark is over an order of magnitude larger than the previous pFoodReq benchmark, giving the food-KG community a substantially bigger testbed for constrained question answering.","On the independent pFoodReq benchmark, KERL-Recom improves F1 by 21.7 points over the pFoodReq method itself, suggesting the approach transfers beyond the authors' own template-generated test set."],"supporting_citations":[{"why":"Supplies FoodKG, the knowledge graph whose recipes, tags, ingredients, and nutrient triples are the source of subgraph retrieval and ground-truth answers.","marker":"Haussmann et al., 2019"},{"why":"Provides Recipe1M, which underlies FoodKG and supplies the recipe and nutrition benchmark splits used for training and evaluation.","marker":"Salvador et al., 2017"},{"why":"Defines pFoodReq, the prior constrained recipe QA benchmark and baseline that KERL-Recom outperforms by 21.7 F1 points.","marker":"Chen et al., 2021"},{"why":"Introduces LoRA, the low-rank adaptation method used to train all three task-specific adapters on the shared base model.","marker":"Hu et al., 2022"},{"why":"Provides Phi-3-mini, the base LLM whose 128K-context variant serves as the backbone and whose low zero-shot F1 highlights the need for KG grounding.","marker":"Abdin et al., 2024"},{"why":"Contributes LLaVA-Chef, the multimodal recipe-generation baseline that KERL-Recipe and KERL-Nutri are compared against, and supplies the test50k split and prompt templates.","marker":"Mohbat and Zaki, 2024"},{"why":"Provides GPT-4, which was used to generate the base question and prompt templates for the KGQA and nutrition benchmarks.","marker":"Achiam et al., 2023"}],"fun_headline_variants":["KG context lifts recipe LLM to 0.973 F1, beating 7B","Recipe recommender gets knowledge graph boost, tops larger LLMs","KERL: KG-augmented LLM nails personalized recipes and nutrition","From query to full meal: KERL's KG+LLM pipeline wins","Personalized recipe picking with KG: KERL beats larger models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system assumes every user question can be expressed through fixed templates using one of 15 health tags and the exact ingredient names stored in FoodKG; since the benchmark questions are generated by filling those same templates, the test never exposes the system to free-form wording, synonyms, misspellings, or ingredient substitutions, and a real question that deviates can produce an empty or wrong candidate set that the model cannot recover from.","fun_headline_variants_meta":{"raw":{"variants":["KG context lifts recipe LLM to 0.973 F1, beating 7B","Recipe recommender gets knowledge graph boost, tops larger LLMs","KERL: KG-augmented LLM nails personalized recipes and nutrition","From query to full meal: KERL's KG+LLM pipeline wins","Personalized recipe picking with KG: KERL beats larger models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000459,"raw_usage":{"total_tokens":2327,"prompt_tokens":998,"completion_tokens":1329,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":1229}},"tokens_in":614,"tokens_out":1329,"duration_ms":10314,"temperature":1.0,"reasoning_tokens":1229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:31:14.891054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a test set of paraphrased and free-form versions of the same constraints—using synonyms for ingredients, alternative tag names, and misspellings—and measure how often the template SPARQL retrieval returns a correct candidate set before the LLM is asked to rank. If a large fraction of such inputs yield empty or wrong retrieval, the reported F1 on template-generated questions would not reflect real-world use, and the central claim that the system is a complete personalized recommender would be undermined.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies FoodKG, the knowledge graph whose recipes, tags, ingredients, and nutrient triples are the source of subgraph retrieval and ground-truth answers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Recipe1M, which underlies FoodKG and supplies the recipe and nutrition benchmark splits used for training and evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes LLaVA-Chef, the multimodal recipe-generation baseline that KERL-Recipe and KERL-Nutri are compared against, and supplies the test50k split and prompt templates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides GPT-4, which was used to generate the base question and prompt templates for the KGQA and nutrition benchmarks."}],"review_version":1}