{"id":"41a2530f-fcee-4703-a7d7-8c03206c37c7","arxiv_id":"2412.10787","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"MAGUS augments any single-round recommender into a multi-round query-plus-item recommender using label propagation on a word graph, and reports gains on a simulated interactive benchmark over three e-commerce datasets.","lead":"This paper proposes MAGUS, a plug-and-play framework that runs any recommendation model in a multi-round loop, presenting artificial queries and items built from word combinations and updating scores via graph propagation. Its headline claim, that this speeds up users' discovery of preferred items, is evaluated with a simulated user whose yes/no answers follow the same word rules the system uses.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The simulated user in Definition D.1 answers Yes exactly when a query's words are a subset of a target item's word combination, which is the same containment relation that defines R+ edges in Section 3.1 and drives propagation in Eq.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the simulated user's answer rule is the very containment relation that MAGUS uses to build its graph and propagate feedback. I agree that this is the most serious threat to the central claim. The paper is internally coherent: the graph construction, label propagation, and feature propagation are clearly specified, and the empirical sweep across 12 recommenders and 3 datasets is consistent within the simulation. Credit is due for acknowledging the idealized user assumption and for including an ambiguous-feedback robustness study, even though that study preserves the subset relation as the affirmative criterion. However, these acknowledgments do not remove the circularity: if real users' answers to queries do not follow the subset relation, the reported SA@K and RA@K gains could disappear entirely. The proposed concrete test, a perturbation analysis of the user oracle, would directly measure how sensitive the results are to violations of this assumption. If the gains are robust to moderate oracle noise, the concern is mitigated; if they collapse, the paper's evidence does not establish the headline claim. Since the reader already judged the paper CONDITIONAL on exactly this basis, my stress-test does not change the verdict.","tokens_in":23104,"tokens_out":6963,"duration_ms":70309,"concrete_test":"Re-run the Table 1 evaluation with a perturbed user agent that, for each round, flips the containment-based Yes/No answer with probability epsilon in {0.1, 0.2, 0.3} (and, in a second arm, replaces the subset rule with relevance labels from real query logs or human raters if available). Plot SA@3 and SA@5 for X+MAGUS and for the strongest baseline. If SA@3 for X+MAGUS drops by more than 20% of its clean-oracle value while the baseline's SA@3 changes by less than 5%, the reported efficiency gains depend on perfect oracle compliance and the central claim about real-user efficiency is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MAGUS enhances the efficiency with which users identify preferred items in multiple-round interactions. The only evidence for this is the simulator in Appendix D.1: the user agent answers 'Yes' to a recommended query iff the query is an element within the word combination of a target item, and 'Yes' to an item iff the item is a target. This is exactly the containment relation used to build R+ edges (Section 3.1) and to propagate positive feedback in Eq. (9): when a query is affirmed, every item containing that query's words receives an increased score, capped at 1. Thus the evaluation oracle's notion of a 'useful query' is logically identical to the relation MAGUS exploits. Under this oracle, MAGUS performs a deterministic narrowing of the candidate set along the subset lattice, and the reported SA@3/SA@5 improvements over item-only and conversational baselines may reflect the structure of that lattice rather than any learned understanding of user preferences. The paper itself acknowledges in Appendix D.1 that 'users always have a clear picture of their interests... may not hold in practice,' and Appendix E.1 only introduces a 'Not Care' response, which still treats the subset relation as the ground truth for affirmative answers. The query set is also synthetic (Appendix C.2), enumerated from item attributes rather than real search logs. Therefore, the headline efficiency claim is not yet supported for real users; the experiments validate internal consistency of the graph-traversal mechanism, not transfer to human behavior.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MAGUS, a multiple-round recommendation framework that augments any offline-trained recommender by jointly scoring and recommending artificial queries and items. Queries and items are represented as nodes in a relational graph over words and word combinations; the base recommender initializes item scores, which are propagated over mutual-improvement/inhibition edges, and real-time feedback is incorporated via label propagation. The stated objective (Definition 2.1) is to minimize the number of rounds needed to surface an item in a target set. Experiments on three e-commerce datasets with 12 base recommenders report significant gains in round-wise accuracy (RA@3) and session-wise accuracy (SA@3, SA@5) over item-only and conversational baselines.","tokens_in":23292,"tokens_out":5570,"duration_ms":52672,"significance":"If the central efficiency claim were established, MAGUS would be a practically attractive plug-in: it is compatible with a wide range of base recommenders, has bounded online overhead, includes an offline weight-learning variant, and comes with a deployment-oriented architecture. The paper also provides a formal graph-based formulation, a complexity analysis, and a code link, which are strengths. However, the current evidence is generated by a simulator whose affirmative-response rule is identical to the containment relation encoded in the graph and used in score propagation; the reported gains therefore mostly validate internal consistency between the simulator's oracle and MAGUS's update rule. External validity for real user behavior remains to be demonstrated.","major_comments":[{"comment":"The simulated user in Appendix D.1 answers 'Yes' to a query exactly when the query is an element of the word combination of a target item. This is the same containment relation that defines the R+ edge class in Definition 3.1 and that Eq. (9) uses to raise the scores of all items containing an affirmed query when weights are set by Eq. (3). Consequently, the reported SA@3/SA@5 and RA@3 improvements are largely a test of how well MAGUS walks the subset lattice defined by the oracle, not of whether real users' yes/no answers can be exploited in this way. Appendix E.1's 'Not Care' response does not remove this problem, because affirmative answers still use the subset rule. A concrete remedy would be an evaluation with a user model whose yes/no response depends on item-level utility or on real logged query-item click data, with the containment relation used only by the recommender and not by the oracle; the authors should show that gains persist when the oracle and the propagation rule are not aligned.","section":"Appendix D.1, Definition D.1; Section 3.1; Eq. (9)"},{"comment":"The query set Q is synthesized by enumerating all word combinations over item feature words; it is not drawn from real user search logs, and the optional historical-query set H_q of Eq. (5) plays no role in the experiments. Thus the system is evaluated on artificial queries constructed from the same vocabulary used to build the graph. This makes it difficult to conclude that MAGUS improves efficiency for realistic user-issued queries; the authors should evaluate on a test collection containing real queries, or at minimum provide a sensitivity analysis over query-generation distributions.","section":"Appendix C.2; Eq. (5)"},{"comment":"The first finding in Section 4.2 compares SAC (single round, items only) with RA@3/SA@3/SA@5 (multiple rounds, queries and items). This comparison cannot be attributed to the addition of queries, because any multiple-round protocol with feedback has more chances to hit a target. The valid evidence for the contribution is the comparison with the RNN-augmented multiple-round baselines (X) and conversational baselines, but those comparisons inherit the same oracle-related concern as the first major comment.","section":"Section 4.2, paragraph 1; Table 1"},{"comment":"No error bars, standard deviations, or numbers of random sessions are reported for RA@3/SA@3/SA@5, and the * significance markers are not accompanied by a description of how the significance test was constructed or how many repeated runs were used. Since the simulated user is deterministic given the sampled session, the authors should report variance across session draws and across random seeds of the base recommenders.","section":"Table 1; Appendix C.5"}],"minor_comments":[{"comment":"In the paragraph defining the relational graph, 'MAGUS commerces by extracting' should read 'MAGUS commences by extracting'.","section":"Section 3.1"},{"comment":"'efficiency, with which users can identify' should be 'efficiency with which users can identify'.","section":"Abstract"},{"comment":"'We also derive into a complexity study' should read 'We also report a complexity study'.","section":"Section 4.4"},{"comment":"The axis labels in Figures 3 and 4 are not legible in the submitted version; please provide clean versions.","section":"Figures 3 and 4"},{"comment":"The phrase 'in the masonry layout' appears to be an artifact and should be clarified or removed.","section":"Appendix C.4, Figure 5 caption"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper describes a coherent algorithmic framework, and the experiments are internally consistent, but I regard the simulator's oracle as the main obstacle to acceptance. The claim that MAGUS significantly enhances efficiency is supported only under an oracle whose yes/no answers are defined by the exact word-containment relation the method propagates along. I recommend major revision rather than rejection because this can in principle be addressed with a realistic user model or a log-based evaluation, and because the framework's plug-in design and scope are otherwise suitable for the venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Top line: the paper has a clean, useful idea—wrap any offline recommender in a multi-round loop that recommends both word-combination queries and items, and propagate feedback over a word-containment graph. The task is genuinely new, and the sweep across 12 base recommenders and 3 datasets is thorough. But the headline claim about real users is not supported by the evidence given.\n\nThe soft spot is the evaluation oracle. In Appendix D.1 the simulated user answers 'yes' to a query exactly when the query's words are a subset of a target item's words. That is precisely the relation MAGUS propagates along in Eq. (9): after a 'yes,' every item containing those words gets a boost. So the reported RA@K and SA@K gains quantify how well the system searches a word-containment lattice whose ground truth is defined by the method's own propagation rule. That is an internal-consistency result, not a demonstration of real-user efficiency.\n\nThe paper does acknowledge the idealized user assumption and adds an ambiguous-feedback experiment where the user can say 'Not Care.' That helps, but the 'yes' rule is still the same subset relation, so it doesn't break the circularity. The query set is synthetic—enumerated from item attributes, not from real search logs. Minor issues: no error bars or standard deviations despite the significance stars, and the promised code link is missing from Appendix C.5.\n\nWhat the paper does well: the framework is genuinely plug-and-play, the graph construction and two propagation directions are clearly explained, and the experimental sweep is broad and internally consistent. The citation pattern looks fine. The paper is also honest about its idealized user model.\n\nBottom line: a plausible framework that deserves referee time, but the evaluation needs to be redone or reframed. The authors should run a human study, use real query logs, or at least build a simulated user whose response rule is not identical to the propagation relation. As it stands, the contribution is a new task plus a plausible method for it, not proof that real users find items faster. I'd treat it as revise, not reject.","headline":"The framework is clean and the task is new, but the simulated user answers exactly along the relation MAGUS propagates, so the reported gains are internal consistency, not real-user evidence.","tokens_in":23964,"tokens_out":3889,"would_cite":false,"duration_ms":34524,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MAGUS claims that recommending both artificial queries and items in a multi-round loop, initialized by any offline recommender and updated by label propagation over a word graph, lets users find a target item in fewer rounds than…","keywords":["multi-round recommendation","query recommendation","item recommendation","relational graph","label propagation","conversational recommender","user feedback","categorical word combinations"],"falsifier":"In the paper's simulator, replace the deterministic containment-based user response with a noisy response model (e.g., Yes with probability 0.8 when query words appear in a target item, and 0.2 otherwise) or with queries sampled from real search logs; if SA@K and RA@K gains over item-only baselines disappear, the containment assumption is the load-bearing premise.","tokens_in":22759,"feed_emoji":"🎯","tokens_out":5361,"duration_ms":47830,"temperature":0.7,"pith_summary":"MAGUS tries to establish that queries and items should be recommended together in a multi-round loop rather than treated as separate tasks. Its central claim is that any offline-tuned recommender can be wrapped in a graph-based framework that guesses user interests with artificial queries (word combinations) and items each round, and that this integrated loop lets users find a preferred item in fewer rounds than item-only or conversational baselines. If true, the practical payoff is large: existing recommender platforms could add query-plus-item interaction without retraining their core models, and users with vague preferences could iterate to a target item faster. The paper reports consistent gains across 12 base recommenders and three real-world e-commerce datasets.","feed_headline":"Queries plus items find target items in fewer rounds","feed_subtitle":"Any offline recommender can suggest queries and items together; 12-method tests show target items appear sooner.","key_machinery":"The load-bearing object is the relational graph G=(E,R), where E contains individual words and word combinations that correspond to queries or items, and R is split into mutual-improvement edges R+ (one node is part of another), mutual-inhibition edges R− (nodes from distinct items that compete), and mutual-independence edges R⊥ (compatible but non-overlapping). The mechanism that carries the argument is label propagation: initial item scores from the offline recommender spread along R+ and R− to score all word nodes, and after each user response the feedback is propagated back along the same edges, so the next recommendation reacts to the current session rather than only to history. Edge weights are either fixed (Equation 3) or learned from item embeddings (Equation 14), with the learned variant called MAGUS+.","core_discovery":"On its own terms, the paper argues that queries and items are not two recommendation tasks but two granularities of the same user-interest representation: both are combinations of categorical words. MAGUS builds a relational graph whose nodes are individual words and word combinations, connects nodes by mutual-improvement, mutual-inhibition, and mutual-independence edges, initializes item-node scores with any offline recommender, and propagates user feedback through the graph each round. The formal objective is to minimize the round K at which the recommended action lands in the target item set. The paper's headline finding is that this query-plus-item loop significantly improves round-wise and session-wise accuracy over item-only and conversational baselines on all tested datasets.","pith_inferences":["Because MAGUS treats queries as generated word combinations, its gains depend on the quality of the word vocabulary; building Q from real search logs instead of enumerating all attribute combinations is a natural stress test the paper does not run.","The containment-response user model turns the problem into a structured twenty-questions game; framing the query-selection rule as information gain over the graph could yield an even cheaper query policy than pure score ranking.","The same graph and propagation loop could be reused for conversational product search or voice assistants, where users answer attribute-style questions; the paper only sketches the LLM integration as a query-decoration step.","If real users answer queries more noisily than the simulator, the safe prediction is that the benefit persists only while item feedback remains available each round; a mixed policy that balances query and item suggestions would be the testable fix."],"forward_implications":["Any single-round recommender can be turned into a multi-round query-and-item recommender without redesigning the base model, since MAGUS only needs its item scores as initialization.","Users should reach a target item in fewer rounds when queries are recommended alongside items, because query feedback constrains the word graph before an item is hit.","More interaction rounds help more than longer recommendation lists, since each round lets the label propagation incorporate fresh feedback.","Learning edge weights from item embeddings (MAGUS+) adds a consistent gain on graph-based base recommenders.","The framework extends to ambiguous user feedback (Not Care responses) and still beats the conversational baselines, though absolute scores drop."],"supporting_citations":[{"why":"Supplies the Factorization Machine base recommender whose scores initialize MAGUS in the main experiments.","marker":"[23]"},{"why":"Supplies the DeepFM base recommender used in the ablation-style comparisons.","marker":"[9]"},{"why":"Defines the Conversational Recommender Model baseline that MAGUS is compared against after extending its action space.","marker":"[27]"},{"why":"Defines the Estimation-Action-Reflection conversational baseline that MAGUS outperforms.","marker":"[15]"},{"why":"Provides the neighborhood-propagation approach used for extracting words from item categorical features when constructing the graph.","marker":"[6]"},{"why":"Provides the context-sensitive query auto-completion baseline behind the popularity-based MPS and Hybrid methods.","marker":"[1]"},{"why":"Supplies the Relational Graph Convolutional Network base recommender that MAGUS wraps and compares against.","marker":"[25]"},{"why":"Supplies the GIPA graph base recommender used with MAGUS and MAGUS+ to test learned edge weights.","marker":"[32]"}],"fun_headline_variants":["Queries and items together shorten recommendation rounds","Multi-round loop blends queries and items for faster finds","Query-item synergy reduces rounds to target items","Combining queries and items speeds up target discovery","One framework to suggest queries and items in fewer rounds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that when a user says Yes to a suggested query, every word in that query is contained in an item they would accept, which is exactly the containment rule the graph and the score updates rely on.","fun_headline_variants_meta":{"raw":{"variants":["Queries and items together shorten recommendation rounds","Multi-round loop blends queries and items for faster finds","Query-item synergy reduces rounds to target items","Combining queries and items speeds up target discovery","One framework to suggest queries and items in fewer rounds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1198,"prompt_tokens":854,"completion_tokens":344,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":273}},"tokens_in":470,"tokens_out":344,"duration_ms":3436,"temperature":1.0,"reasoning_tokens":273,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:37:47.243748+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the paper's simulator, replace the deterministic containment-based user response with a noisy response model (e.g., Yes with probability 0.8 when query words appear in a target item, and 0.2 otherwise) or with queries sampled from real search logs; if SA@K and RA@K gains over item-only baselines disappear, the containment assumption is the load-bearing premise.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Estimation-Action-Reflection conversational baseline that MAGUS outperforms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DeepFM base recommender used in the ablation-style comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Conversational Recommender Model baseline that MAGUS is compared against after extending its action space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the neighborhood-propagation approach used for extracting words from item categorical features when constructing the graph."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the context-sensitive query auto-completion baseline behind the popularity-based MPS and Hybrid methods."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Relational Graph Convolutional Network base recommender that MAGUS wraps and compares against."},{"cited_title":"GIPA: General Information Propagation Algorithm for Graph Learning","cited_arxiv_id":"2105.06035","evidence_quote":"Supplies the GIPA graph base recommender used with MAGUS and MAGUS+ to test learned edge weights."}],"review_version":1}