{"id":"be5aab02-5e04-4164-b9e6-fd3899d61763","arxiv_id":"2506.06270","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RecGPT turns item descriptions into shared discrete tokens and trains an autoregressive transformer to predict the next item's tokens, enabling zero-shot recommendations in unseen domains.","lead":"RecGPT is a sequential recommender that represents every item as discrete tokens generated from its text description, so new items and new domains can be handled without retraining. It reports large gains over few-shot baselines in zero-shot cross-domain tests and in an industrial news deployment.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central claim of consistent zero-shot superiority is contradicted by its own Tables 1 and 2, where 10%-trained baselines beat RecGPT on Hit@5 for Washington, Steam, and cold-start Yelp.","rationale":"I read the paper as making a specific, falsifiable empirical assertion: RecGPT, with no target-domain training, beats ID-based sequential recommenders that train on 10% of the target data, consistently across six datasets and an industrial deployment. The architecture is thoughtful and the zero-shot setup is externally meaningful, so the central idea deserves serious testing. However, the paper's own numerical tables give direct counterexamples to the word 'consistent': on Washington, a 10%-trained MAERec has higher Hit@5 than RecGPT; on Steam, FDSA has higher Hit@5; and in cold-start evaluation on Yelp, several baselines are substantially ahead of RecGPT on Hit@5 and NDCG. These are not borderline differences of a few thousandths; the Yelp cold-start gap is a factor of about 1.7 in Hit@5. The reader's weakest-assumption field emphasized whether text embeddings are sufficiently domain-invariant, which is a real scientific risk, but the more immediate and decisive problem is that the paper's reported evidence contradicts its headline generalization claim. I therefore focus the stress test on that contradiction. If the authors revise the claim to a qualified one, or add confidence intervals and show that the apparent counterexamples vanish under proper evaluation, the conditional verdict could be upgraded; as presented, the central claim is overstated. This agrees with the reader's rationale, which already lists these table inconsistencies, while differing from the reader's stated weakest assumption. I recommend no change to the conditional verdict because the existing conditional already requires correction of the empirical claims; if anything, the paper should be held to that condition before acceptance.","tokens_in":17559,"tokens_out":7184,"duration_ms":74204,"concrete_test":"Extract Tables 1 and 2 into machine-readable form and run a script that, for every (dataset, metric, cutoff) cell, compares RecGPT against the best 10%-trained baseline and against each baseline individually. Output all violations of 'RecGPT > every baseline.' If the script reproduces the Washington/Steam/Yelp cells above, the 'consistent performance advantages' claim must be revised to a qualified claim (e.g., 'wins on most NDCG metrics but not Hit@5'). Independently, rerun the cold-start Yelp comparison with 5 random seeds and report mean plus-minus standard deviation; if CL4SRec or DuoRec remain above RecGPT, the cold-start advantage claim also fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RecGPT achieves genuine zero-shot generalization and 'significantly outperforms' ID-based sequential recommenders trained on 10% of target-domain data. The paper's own tables do not support this as stated. In Table 1 (zero-shot comparison), on Washington the 10%-trained MAERec reaches Hit@5 = 0.0154 versus RecGPT's 0.0130, and on Steam FDSA reaches Hit@5 = 0.1366 versus RecGPT's 0.1253; RecGPT wins NDCG@5 in these cases but loses Hit@5. In Table 2 (cold-start), RecGPT is not the best on Yelp (Hit@5 0.0131 vs 0.0221 for CL4SRec, 0.0220 for DuoRec, 0.0219 for FDSA), nor on Baby (0.0172 vs 0.0204 for DuoRec), nor on Office (0.0204 vs 0.0207 for DuoRec). Since the abstract and Section 3.2 claim 'consistent performance advantages' across all evaluated domains, these cells are direct counterexamples from the manuscript itself. No error bars or repeated-seed statistics are reported, so even in favorable comparisons we cannot tell whether RecGPT's margin is significant. The zero-shot mechanism may still work, but the headline empirical claim, as written, is not established by the evidence shown.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes RecGPT, a decoder-only transformer for sequential recommendation that replaces item IDs with text-derived discrete tokens. Item text is embedded with MPNet, quantized with Finite Scalar Quantization into K codebook tokens, and user histories are modeled with hybrid bidirectional (intra-item) and causal (inter-item) attention plus an auxiliary continuous embedding stream. RecGPT is pre-trained on eleven Amazon categories and evaluated zero-shot on three held-out Amazon categories and three non-Amazon platforms (Yelp, Washington, Steam), with additional cold-start, ablation, scaling-law, and industrial-dataset experiments. The core claim is that without any target-domain training, RecGPT significantly and consistently outperforms ID-based baselines trained on 10% of target-domain data.","tokens_in":17867,"tokens_out":5523,"duration_ms":52710,"significance":"The evaluation design has real strengths: the main zero-shot test sets are external to the pre-training corpus, so the central generalization claim is not circular; source code is released; and the scaling-law analysis is a falsifiable, practically useful target. If the comparative claims were supported, the text-only tokenization and catalog-aware beam search would be a noteworthy step toward cross-domain and cold-start recommendation. However, the manuscript's own tables contradict the 'consistent performance advantages' claim on several Hit@5 cells, and the absence of repeated-run statistics makes the significance assertions unverifiable.","major_comments":[{"comment":"The abstract and Section 3.2 state that RecGPT 'significantly outperforms' and shows 'consistent performance advantages' over 10%-trained baselines, but Table 1 contains direct counterexamples. On Steam, FDSA reaches Hit@5 0.1366 versus RecGPT's 0.1253; on Washington, MAERec reaches Hit@5 0.0154 versus RecGPT's 0.0130. In Table 2's cold-start setting, RecGPT's Hit@5 is below DuoRec on Baby (0.0172 vs 0.0204), below CL4SRec on Yelp (0.0131 vs 0.0221), and below DuoRec on Office (0.0204 vs 0.0207). These cells should either be acknowledged as exceptions or the claim should be revised to a per-metric statement such as 'generally higher NDCG@5 with mixed Hit@5 outcomes.' The wording 'consistent' and 'significantly outperforming' is not supported by the evidence shown.","section":"3.2, Table 1; Abstract"},{"comment":"No standard deviations, confidence intervals, or repeated-seed results are reported anywhere in the tables, despite the footnote claiming p<0.05 significance marked by '*'. In the absence of error bars, the stars appear to denote only that the RecGPT value is higher than the best baseline in that row, which is not a significance test. Please report mean and standard deviation over at least three seeds and specify the statistical test used, or remove the significance claim.","section":"3.2, Tables 1-2"},{"comment":"The scaling-law section fits a power-law curve to evaluation losses of the 5%, 10%, 25%, 50%, and 100% variants and states that it can 'accurately predict the performance of the full model,' but no equation, fitted exponent, goodness of fit, or prediction error is given. It is also unclear whether the solid curve in Figure 5 is the fitted power law or an interpolation between points. Please report the fitted functional form, its parameters, R^2 or residual error, and the predicted-versus-observed value for the full model; otherwise the power-law claim is not checkable.","section":"3.6, Fig. 5"},{"comment":"The 'w/o Pre' ablation removes the Trie-based prefix enhancement and reports essentially unchanged Hit@5 on Baby (0.0282 vs 0.0283), Office (0.0281 vs 0.0299), and Yelp (0.0162 vs 0.0166). The text states that this ablation demonstrates 'recommendation relevance' gains, but the table does not support a ranking-relevance benefit; the component's value is presumably computational. Please report latency or search-cost measurements to justify the decoder design, and avoid attributing ranking gains to it.","section":"3.5, Table 3"}],"minor_comments":[{"comment":"Equation (3) writes the FSQ output as an element of {0,...,L-1}^{d_fsq}, but the text and implementation describe scalar codebook indices with total vocabulary size L^{d_fsq}. Please clarify how the vector output is mapped to the token index used in the R^{15360 x 768} token embedding table.","section":"2.1.2, Eq. (3)"},{"comment":"The text refers to 'Fig. 1 (ii)' and 'Fig. 1 (iii)' when describing the architecture, but the architecture diagram is Figure 2; Figure 1 contains only the motivational chart. Please correct the cross-references.","section":"2.2 and 2.4"},{"comment":"The reported pre-training statistics for Beauty (1,464 users, 6,570 items, 13,679 interactions) and Magazine Subscriptions (380 users, 865 items) are orders of magnitude smaller than typical Amazon benchmark splits and than the other listed categories. Please verify these numbers and clarify the filtering procedure.","section":"Appendix 6.1, Table 4"},{"comment":"The industrial dataset results are shown only as bar charts without numeric values or error bars. Please report the exact Hit@5 and NDCG@5 values in a table, and clarify whether the 'deployment' claim refers to an online A/B test or an offline evaluation on proprietary data.","section":"3.3, Fig. 3"},{"comment":"Figures 6 and 7 present comparisons with pre-trained baselines without numerical values or repeated-run variance. Adding a companion table with exact values and standard deviations would make the claimed advantages verifiable.","section":"3.7, Figs. 6-7"}],"recommendation":"major_revision","confidential_remarks":"The architecture and the external zero-shot evaluation are interesting, but the manuscript's headline claims need to be recalibrated to match its own evidence. The authors should be asked to add repeated-seed statistics, report the scaling-law fit details, and either soften or qualify the 'consistent performance advantages' statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading. RecGPT combines FSQ tokenization, hybrid bidirectional-causal attention, and trie-constrained beam search in a way I haven't seen before, and the evaluation design is the right one for a zero-shot claim: held-out Amazon categories plus Yelp, Washington, and Steam are not in the pre-training data, so the main mechanism is being tested fairly. The ablation study is also well structured, and the code is released. That is real work, and the architecture is worth a serious look.\n\nBut the headline claim does not survive contact with the paper's own tables. The abstract and Section 3.2 say \"consistent performance advantages\" across all domains. On Washington, the 10%-trained MAERec hits Hit@5 0.0154 while RecGPT gets 0.0130; on Steam, FDSA gets 0.1366 vs RecGPT's 0.1253. In cold-start (Table 2), RecGPT is not the best on Yelp (0.0131 vs CL4SRec's 0.0221, DuoRec's 0.0220, or FDSA's 0.0219), nor on Baby or Office. So the consistent-superiority language is false as written. The zero-shot mechanism may still work—RecGPT wins NDCG in several of these cases and looks strong on Hit@1—but the paper needs to be honest that the advantage is not universal.\n\nTwo more soft spots. The scaling-law section claims to predict the full model's performance from smaller variants, but the figure shows the power-law curve fit to points that include the 100% full model itself. That is a fit, not a prediction. And there are no error bars anywhere, so for the wins we cannot tell if the margins are significant. The industrial result also stays at the level of a bar chart with no numbers, which is weaker than it could be.\n\nThe weakest assumption is that text alone is a sufficient domain-invariant signal for ranking. The paper treats MPNet embeddings of titles and categories as item semantics and does not test what happens when text is noisy or uninformative (e.g., generic titles). That is a limitation worth stating, but it does not by itself sink the approach; the external zero-shot results are at least a first pass at validating it.\n\nWho is this for? Anyone working on generative or text-driven recommenders, and people thinking about cross-domain transfer in sequential recommendation. It deserves a serious referee, but the authors need to fix the overclaim, add error bars, report industrial numbers, and reframe the scaling-law section before it is acceptable.","headline":"A genuinely new text-driven tokenization architecture and the right zero-shot evaluation design, undercut by an abstract that overclaims consistency and a scaling-law section that fits instead of predicts.","tokens_in":18436,"tokens_out":2184,"would_cite":true,"duration_ms":19974,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RecGPT claims that a transformer pre-trained on item text alone can recommend in unseen domains with no target-domain training, beating ID-based recommenders that saw 10% of the data.","keywords":["sequential recommendation","zero-shot generalization","foundation model","Finite Scalar Quantization","text-driven item representation","cross-domain recommendation","autoregressive recommendation","cold-start recommendation"],"falsifier":"Permute item text fields randomly while keeping user interaction sequences identical on a held-out dataset such as Yelp or Steam. If RecGPT's zero-shot Hit@5 stays near its reported level, the model is not actually using text semantics; if it collapses toward the random-token ablation (w/o FSQ), the claim that text drives transfer is confirmed. A complementary test is to pre-train on the same eleven categories with descriptions replaced by random strings and check whether zero-shot performance on Baby, Yelp, and Steam drops to chance.","tokens_in":17327,"feed_emoji":"🎯","tokens_out":9778,"duration_ms":85961,"temperature":0.7,"pith_summary":"RecGPT tries to establish that a recommender system can generalize to unseen domains without any target-domain training, by replacing item IDs with tokens derived solely from item text. The paper argues that pre-training an autoregressive transformer on interaction sequences from eleven e-commerce categories yields true zero-shot ability: on six held-out datasets, RecGPT outperforms recommenders that learn separate embeddings for each item ID and were trained on 10% of the target-domain data, and the same model also works on an industrial news platform. If the claim holds, cold-start and cross-platform recommendation would no longer require per-domain retraining, because any new item can be embedded from its description at inference time. The paper also reports that the model's loss follows an LLM-style power-law scaling curve, with a sharp improvement in zero-shot performance between 10% and 25% of pre-training data.","feed_headline":"Zero training, zero IDs: text tokens carry cross-domain recommendation","feed_subtitle":"Any new item enters from its description; it beats baselines trained on 10% of target data.","key_machinery":"The load-bearing machinery is a text-only item tokenization pipeline: MPNet maps each item's title and category text to a continuous embedding; Finite Scalar Quantization (FSQ), a rounding-based scheme, splits that embedding into four sub-vectors and rounds them into discrete tokens from a shared codebook of 15,360 tokens, so any item with text can be tokenized without retraining. On top of this, a transformer with hybrid attention lets tokens within the same item attend bidirectionally while keeping causal attention across items, preserving both item coherence and sequence order. An auxiliary stream adds the continuous semantic embeddings, layer-normalized and added to the discrete token embeddings plus positional embeddings, to compensate for quantization loss. At inference, a catalog-aware beam search with Trie-based prefix constraints searches only token sequences that correspond to real catalog items, predicting all K tokens of the next item in one pass.","core_discovery":"The paper's central claim is that text-driven, token-based autoregressive modeling is sufficient for cross-domain sequential recommendation. RecGPT represents each item by quantizing MPNet text embeddings into a small number of discrete tokens via Finite Scalar Quantization, treats a user's history as a token sequence, and trains on next-token prediction. Without any fine-tuning, it outperforms GRU4Rec, Caser, BERT4Rec, FDSA, CL4SRec, DuoRec, ICLRec, and MAERec trained on 10% of the target data across Baby, Games, Office, Yelp, Washington, and Steam, and also beats S3-Rec, UniSRec, VQ-Rec, TIGER, IDGenRec, and RecFormer in zero-shot comparisons. The authors further claim that ablation results isolate the semantic tokenizer, the hybrid bidirectional-causal attention, and the auxiliary continuous-semantic stream as individually necessary, and that a controlled comparison with VQ-Rec attributes the advantage to architecture rather than data scale. In addition, RecGPT shows power-law scaling of evaluation loss with token count and an emergent generalization threshold, both taken as evidence that recommendation foundation models obey scaling laws analogous to language models.","pith_inferences":["A direct extension of the tokenization logic: swapping the text encoder for a multimodal encoder should make the same zero-shot recipe work for music, images, and video items, which the paper lists as future work.","A boundary condition follows: on domains where choices are driven by visual or audio attributes, price, or platform-specific popularity rather than title and category text, the zero-shot advantage should shrink as the text-to-behavior correlation falls.","The scaling-law result is extrapolatable: training on more categories or languages should lower the 10%-to-25% threshold where zero-shot ability jumps, a prediction that could be tested by measuring the discontinuity under different pre-training mixtures.","Because the model is text-only, user cold-start still lacks profile signals such as age and location; combining text tokens with lightweight user features is a natural hybrid that preserves transfer while adding the signals the paper concedes are missing."],"forward_implications":["New items with a textual description can enter the catalog at inference time, making item cold-start immediate and removing the need for per-domain ID tables.","A single pre-trained model can serve e-commerce, local-business, gaming, and news recommendation without fine-tuning, at least when text carries the preference signal.","Pre-training data volume, not model size, is the cheaper lever: the reported power-law scaling means performance can be forecast and improved by adding tokens.","The beam-search decoder keeps inference fast enough for production by pruning the token space with the catalog trie.","The hybrid attention design is necessary: ablations show removing bidirectional intra-item attention or the auxiliary semantic stream degrades zero-shot performance."],"supporting_citations":[{"why":"Supplies the MPNet text encoder whose embeddings are the only item representation, so the whole zero-shot mechanism depends on it.","marker":"[10]"},{"why":"Provides the Finite Scalar Quantization method used to turn continuous item embeddings into discrete tokens.","marker":"[14]"},{"why":"Grounds the autoregressive next-token paradigm the paper adopts for sequential recommendation.","marker":"[15]"},{"why":"Defines the generative retrieval with semantic IDs that RecGPT builds on and compares against.","marker":"[24]"},{"why":"The vector-quantized pre-trained baseline; the controlled re-training study against it attributes RecGPT's advantage to architecture rather than data scale.","marker":"[23]"},{"why":"Supplies the eleven e-commerce categories used as pre-training and evaluation data.","marker":"[39]"},{"why":"Provides the notion of emergent ability used to interpret the sharp zero-shot jump between 10% and 25% pre-training data.","marker":"[19]"},{"why":"Provides the LLM power-law scaling reference used to fit RecGPT's evaluation loss curve.","marker":"[20]"}],"fun_headline_variants":["Text tokens, zero IDs: RecGPT goes cross-domain with no retraining","RecGPT: zero-shot recs from descriptions, no fine-tuning needed","Beyond IDs: RecGPT tokenizes text for universal recommendations","Zero-shot sequential recs: RecGPT's text tokens beat 10% baselines","From item text to tokens: RecGPT's cross-domain zero-shot edge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that textual descriptions are sufficient, domain-invariant signals of user preference; if title-and-category text does not encode the behavioral or platform-specific signals behind choices, the zero-shot mechanism collapses.","fun_headline_variants_meta":{"raw":{"variants":["Text tokens, zero IDs: RecGPT goes cross-domain with no retraining","RecGPT: zero-shot recs from descriptions, no fine-tuning needed","Beyond IDs: RecGPT tokenizes text for universal recommendations","Zero-shot sequential recs: RecGPT's text tokens beat 10% baselines","From item text to tokens: RecGPT's cross-domain zero-shot edge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1833,"prompt_tokens":968,"completion_tokens":865,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":768}},"tokens_in":584,"tokens_out":865,"duration_ms":8437,"temperature":1.0,"reasoning_tokens":768,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:56:50.577006+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Permute item text fields randomly while keeping user interaction sequences identical on a held-out dataset such as Yelp or Steam. If RecGPT's zero-shot Hit@5 stays near its reported level, the model is not actually using text semantics; if it collapses toward the random-token ablation (w/o FSQ), the claim that text drives transfer is confirmed. A complementary test is to pre-train on the same eleven categories with descriptions replaced by random strings and check whether zero-shot performance on Baby, Yelp, and Steam drops to chance.","supporting_citations":[{"cited_title":"Mpnet: Masked and permuted pre-training for language understanding","cited_arxiv_id":null,"evidence_quote":"Supplies the MPNet text encoder whose embeddings are the only item representation, so the whole zero-shot mechanism depends on it."},{"cited_title":"Recommender systems with generative retrieval","cited_arxiv_id":null,"evidence_quote":"Defines the generative retrieval with semantic IDs that RecGPT builds on and compares against."},{"cited_title":"Learning vector-quantized item representation for transferable sequential recommenders","cited_arxiv_id":null,"evidence_quote":"The vector-quantized pre-trained baseline; the controlled re-training study against it attributes RecGPT's advantage to architecture rather than data scale."},{"cited_title":"Emergent abilities of large language models","cited_arxiv_id":null,"evidence_quote":"Provides the notion of emergent ability used to interpret the sharp zero-shot jump between 10% and 25% pre-training data."}],"review_version":1}