{"id":"25cba82b-8090-4f77-954a-1e763fa72ff7","arxiv_id":"2506.01375","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"GNPR-SID assigns points of interest hierarchical semantic codes via a residual quantized VAE and fine-tunes an LLM to predict the next code, improving next-POI accuracy on three benchmarks.","lead":"This paper gives each place in a location-based app a smart code that reflects what kind of place it is, instead of a random number. The authors show that an AI model trained to predict a user's next visit using these smart codes is more accurate than models using random codes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main Table 2 gains may be confounded by unablated data augmentation (App. A.1) applied to GNPR-SID but not to baselines; without a controlled comparison, the SID attribution is unproven.","rationale":"The reader's weakest_assumption (cross-dataset SID construction) is a real gap, but it concerns Table 5, a generalization claim. The more fundamental risk is that the main empirical claim in the abstract and Table 2 is unfair because of an unmentioned data-augmentation asymmetry. Appendix A.1 is not referenced in Section 5, and the baseline list in Section 5.3 gives no indication that any baseline received the same augmentation. Since the fill-in-the-blank objective is a multi-task training signal, it can boost next-POI accuracy even with random IDs; therefore the gap over LLM4POI cannot be attributed to semantic IDs without a controlled run. This does not invalidate the method; it makes the headline evidence conditional, matching the reader's CONDITIONAL verdict. I retain UNCHANGED because the paper's code and ablations make a fix feasible, and the concern is testable rather than fatal.","tokens_in":18053,"tokens_out":4944,"duration_ms":47978,"concrete_test":"Retrain LLM4POI (and at least one strong non-LLM baseline such as ROTAN) using the exact Appendix A.1 augmentation pipeline (multiple cropping plus 20% fill-in-the-blank), and separately retrain GNPR-SID without augmentation, then rebuild Table 2 with identical training budgets. If the augmented LLM4POI closes the reported 7%/1%/16% gaps on NYC/TKY/CA, the headline claim must be restated as an augmentation effect; if the gaps persist, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix A.1 ('Data Augment') describes multiple cropping and a fill-in-the-blank objective (20% of samples predict a POI from an arbitrary historical position) used to construct the GNPR-SID training set. Section 5's experimental setup and the baseline descriptions in Section 5.3 never state that LLM4POI or any other baseline receives the same augmentation. Since the fill-in-the-blank objective teaches the model to predict POIs from any position in the context, it can improve next-POI accuracy independently of SID semantics. Consequently, the Table 2 margins (7%/1%/16% over LLM4POI on NYC/TKY/CA) are not clean evidence that SIDs cause the gain; the gain could come from a larger, differently distributed training set or from the additional multi-task objective. The w/o SID ablation in Table 3 controls for augmentation when comparing GNPR-SID to its own RID variant, but it does not make the comparison to LLM4POI fair. This is load-bearing because the abstract's headline claim of 'up to 16% improvement' rests directly on this comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GNPR-SID, an LLM-based next-POI recommender in which each POI is represented by a semantic ID (SID). The SID is produced by an RQVAE quantizer over a feature vector that combines POI category, Plus-Code region, frequent time slots, and frequent-visitor collaborative signals. An LLM (LLaMA3-8B) is fine-tuned with LoRA to generate the next POI's SID from a history of SIDs and timestamps. Experiments on Foursquare-NYC, Foursquare-TKY, and Gowalla-CA report Acc@1 improvements over LLM4POI, ablations of SID, time, diversity loss, quantization loss, and reconstruction loss, an analysis of diversity loss, a visualization of SID semantic clusters, an out-of-domain transfer experiment, and an efficiency comparison.","tokens_in":18423,"tokens_out":8265,"duration_ms":85045,"significance":"If the empirical claims are robust, the paper makes a useful contribution by adapting semantic ID tokenization, already successful in generative retrieval, to POI recommendation: the two-module design (semantic ID construction plus generative LLM fine-tuning) is clean, the code is released, and the ablation structure (w/o SID, w/o Time, w/o Ldiv, w/o Lquant) is a strength. However, the central 'up to 16% improvement' claim is currently confounded by an unablated data-augmentation difference between GNPR-SID and the baselines, and the cross-dataset generalization evidence is underspecified. The paper is likely to interest the KDD community once these issues are resolved, but the headline quantitative conclusion is not yet supported.","major_comments":[{"comment":"The fill-in-the-blank data augmentation described in Appendix A.1.2 (20% of cropped samples are replaced by a historical-position prediction target) is used when constructing the GNPR-SID training set, but Section 5.3 does not state that LLM4POI or any other baseline receives the same augmentation, and no ablation removes this augmentation. The w/o SID row in Table 3 keeps the augmentation and is therefore not a control for it; on CA that random-ID version already reaches 0.2296 versus LLM4POI's 0.2065, i.e., about 11% relative improvement without any semantic ID. As a result, the Table 2 margins, including the advertised 16% improvement on CA, cannot be attributed to SIDs. Please add an ablation that turns the fill-in-the-blank augmentation on and off, and run the main comparison under matched training conditions (e.g., GNPR-SID without augmentation, and LLM4POI with the same augmentation).","section":"Appendix A.1.2 / Section 5.3 / Table 2"},{"comment":"All reported Acc@1 values are single point estimates with no error bars, confidence intervals, or significance tests. The TKY comparison is especially fragile: GNPR-SID achieves 0.3062 versus LLM4POI's 0.3035, a relative difference of about 0.9% that could easily be within run-to-run noise. The claim that GNPR-SID 'consistently outperforms' the state of the art needs variance information at least for the main results and for the w/o SID ablation. Please report multiple seeds or bootstrap confidence intervals, with a paired significance test where appropriate.","section":"Tables 2-4 / Section 6.1"},{"comment":"The paper's introduction claims that SIDs improve generalization to newly added, unseen POIs, but Section 5.1 removes from the test set all users and POIs not present in the training set, so no within-dataset test POI is genuinely unseen. The only out-of-domain evidence is Table 5, yet the protocol for applying a source-trained RQVAE codebook to target-dataset POIs is not specified: it is unclear whether target features are normalized with source statistics, how the source codebook is reused, how feature-dimension mismatches across cities (different category sets, user sets) are handled, and whether the target test set also excludes POIs absent from the target training split. Please provide the exact cross-dataset encoding procedure and, if possible, add an evaluation setting that contains genuinely new POIs in the test set.","section":"Section 5.1 / Table 5 / Section 6.5"}],"minor_comments":[{"comment":"The citations for LLaMA3-8B and LoRA appear swapped: the text cites LLaMA3-8B as [15] and LoRA as [7], but in the reference list [15] is the LoRA paper and [7] is the Llama 3 report.","section":"Section 5.2 / References"},{"comment":"The phrase 'it achieves the stats-of-the-art performance' should be 'state-of-the-art'.","section":"Section 2"},{"comment":"The compactness loss notation is confusing: the denominator is k(k-1) and the sum is over codeword vectors, but k is also used for the number of codewords in a layer; please clarify the indexing and the definition of the summation range.","section":"Equation (13)"},{"comment":"Because SIDs are constructed from category, region, time-slot, and user features, the observation that similar POIs share prefixes is partly a sanity check of the construction rather than an independent validation; consider adding a quantitative evaluation, such as average prefix overlap against category or geographic similarity, on held-out POIs.","section":"Figure 3"},{"comment":"The efficiency comparison would be more informative if the paper stated whether LLM4POI was re-implemented with the same tokenizer, generation settings, and padding, or whether the numbers were taken from the original paper; otherwise the token and time reductions may reflect implementation choices rather than the proposed method.","section":"Section 6.6 / Table 6"},{"comment":"The multiple-cropping rule is stated vaguely: 'exceeds several times the given length' is not precise. Please specify the exact segmentation and cropping thresholds used in the experiments.","section":"Appendix A.1.1"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the manuscript appears to be a KDD '25 camera-ready version, but the central empirical claim is not yet clean. The main risk is the unablated fill-in-the-blank augmentation in the comparison with LLM4POI; the TKY margin is also very small and unsupported by significance testing. The issues are fixable with additional experiments, so I am not recommending rejection. There is no indication of misconduct; the problems appear to be omissions in experimental reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on LLM-based POI recommendation. The paper adapts the semantic-ID idea from TIGER and Learning to Tokenize to the next-POI setting, using RQVAE over a POI feature vector built from Plus Codes, time slots, and collaborative signals. That adaptation is reasonable and the within-framework ablation is clean: w/o SID drops accuracy on all three datasets, so the SID itself is doing something. The diversity loss analysis is also sensible, and the token-efficiency result is a practical plus. Credit where due: code is released, and the semantic-similarity visualization gives a nice sanity check.\n\nThe big caveat is the comparison to LLM4POI. Appendix A.1 describes a fill-in-the-blank augmentation used when building the GNPR-SID training set. Nothing in Section 5.3 says baselines received the same treatment. So the 7%/1%/16% margins over LLM4POI could come from the augmented training signal rather than from semantic IDs. The w/o SID ablation controls for this only within the augmented pipeline; it does not make the LLM4POI comparison fair. On TKY the margin is 1%, which without error bars or significance tests is noise. That matters because the abstract's headline claim rests on that comparison.\n\nAlso soft: the cross-dataset generalization section (Table 5) never explains how a source-trained codebook quantizes target-city POIs. And the intro promises generalization to unseen POIs, but Section 5.1 removes POIs not in the training set from the test set, so that claim is not actually tested. These are fixable but should be addressed before the results are taken as settled.\n\nBottom line: the paper is a legitimate incremental contribution with a reproducible core, but the central empirical claim is over-stated given the confound. A serious referee should engage, with the main ask being a controlled comparison where baselines get the same augmentation (or GNPR-SID doesn't use it).","headline":"Solid extension of semantic-ID generative retrieval to next-POI, but the headline gains over LLM4POI are confounded by an unablated data augmentation.","tokens_in":18881,"tokens_out":2164,"would_cite":true,"duration_ms":23316,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing random POI IDs with semantic prefix-sharing codes improves LLM next-visit prediction by up to 16%.","keywords":["next POI recommendation","semantic ID","generative recommendation","LLM fine-tuning","residual quantized variational autoencoder","diversity loss","codebook quantization","out-of-domain generalization"],"falsifier":"Retrain GNPR-SID on NYC with the learned codebook but with the codeword indices randomly permuted within every codebook level, so each POI keeps its quantized feature vector yet its SID string loses prefix semantics; if top-1 accuracy remains near 0.3618 rather than dropping toward the random-ID baseline's 0.3372, the reported gain is not caused by semantic prefix sharing.","tokens_in":17843,"feed_emoji":"📍","tokens_out":11788,"duration_ms":115564,"temperature":0.7,"pith_summary":"The paper argues that an LLM can predict the next place a user will visit—the next point of interest (POI)—far better if each place is given a meaningful semantic ID instead of a random number. It proposes GNPR-SID, which first converts each POI's category, region, visiting-time pattern, and frequent visitors into a short hierarchical code, so similar places share code prefixes, and then fine-tunes a large language model to generate the next code from a user's check-in history. On check-in data from New York, Tokyo, and California, the method reports top-1 accuracy of 0.3618, 0.3062, and 0.2403, beating the strongest random-ID LLM baseline by 7%, 1%, and 16% respectively. The point of the design is that the ID itself carries semantic and collaborative information, so the LLM can reason about places instead of treating each ID as an opaque token. A reader should care because the same trick—replacing opaque tokens with structured, semantics-carrying symbols—may transfer to other generative retrieval tasks.","feed_headline":"Semantic place IDs lift next-POI accuracy up to 16 percent","feed_subtitle":"Giving each location a code that shares prefixes with similar venues helps LLMs predict the next check-in.","key_machinery":"The load-bearing mechanism is the semantic ID (SID) viewed as a hierarchical code: a short tuple of codeword indices produced by residual quantization, in which the first index gives the coarsest semantic bucket and each later index refines it. The construction module is a residual quantized variational autoencoder (RQVAE): an encoder maps the concatenated POI features to a latent vector, each codebook layer quantizes the residual left by the previous layer, the selected codewords are summed and decoded to reconstruct the features, and the index sequence becomes the SID. A diversity loss balances two constraints—codewords should be used roughly uniformly across the codebook, and vectors assigned to the same codeword should stay close—so the discrete space does not collapse. The generative module then treats each codeword index as a token, so the LLM is trained to generate the next POI by completing the SID sequence with timestamps.","core_discovery":"The central claim, stated the way the authors would state it, is that semantic IDs make generative next-POI recommendation accurate and portable in a way random IDs cannot. Each POI is described by a one-hot concatenation of category, grid region, top frequent time slots, and top frequent visitors; an RQVAE encoder maps this to a latent vector, and multiple layers of codebooks quantize the residuals so that the POI's SID is a tuple of codeword indices. The design deliberately lets similar POIs share early indices: a tea shop near a coffee shop will receive the same leading codewords and differ only in a later refinement, and any remaining same-ID collisions are resolved by appending a unique symbol. A large language model is fine-tuned with low-rank adapters on prompts that interleave the user's historical SIDs with their visit timestamps, and the next POI is the next SID the model generates. The paper reports Acc@1 of 0.3618, 0.3062, and 0.2403 on NYC, TKY, and CA, and its ablations show that swapping SIDs for random IDs, dropping timestamps, removing the diversity loss, or removing the quantization loss all lower accuracy, while removing the reconstruction loss prevents training from converging.","pith_inferences":["An extension not tested in the paper: the out-of-domain experiment trains on one city and tests on another, so it mixes two transfer questions—whether the codebook's quantization geometry transfers, and whether the LLM's learned mapping from SIDs to POIs transfers; retraining one component while freezing the other would separate them.","A testable extension of the paper's motivation: keep a held-out set of truly unseen POIs, since the current protocol deletes test POIs absent from the training set, to measure cold-start generalization directly.","A design option the paper leaves implicit: predict only the first codeword (region or category), filter candidate POIs, then decode the remaining indices—a coarse-to-fine decoding that would shrink the output vocabulary at each step.","A general transfer of the recipe: quantize any entity's features into prefix-sharing tokens and fine-tune an LLM to emit them, which could apply to products, documents, or other generative retrieval settings where entities are currently assigned random IDs."],"forward_implications":["On the three datasets, replacing random numeric IDs with SIDs raises top-1 accuracy by 7%, 1%, and 16% over the strongest random-ID LLM baseline, and removing the SID in ablation lowers accuracy on every dataset.","A model fine-tuned on one city transfers to another city with less accuracy loss than the random-ID baseline, as shown by the cross-dataset rows of the comparison table.","Because all POI metadata is stored in a few code tokens, the SID format cuts total prompt tokens, training time, and test time by roughly half compared to the random-ID baseline on the NYC benchmark.","The diversity loss has a usable operating range: too little produces codeword collisions and drops accuracy, while too much forces dissimilar POIs together and also drops accuracy.","The SID prefixes align with POI semantics: visualization on TKY shows different first-codeword values correspond to distinct venue categories such as transportation versus entertainment."],"supporting_citations":[{"why":"Supplies the residual quantized variational autoencoder (RQVAE) with hierarchical codebooks that the SID construction is built on.","marker":"[47]"},{"why":"Provides the LLM-based next-POI baseline using random numeric IDs that GNPR-SID is compared against and must outperform.","marker":"[18]"},{"why":"Supplies the New York (NYC) and Tokyo (TKY) check-in datasets used in all experiments.","marker":"[44]"},{"why":"Supplies the California (CA) check-in dataset used in all experiments.","marker":"[5]"},{"why":"Defines the preprocessing and evaluation protocol, including the filters on POIs and users and the removal of test POIs absent from training.","marker":"[43]"},{"why":"Provides the Acc@1 evaluation convention and a time-aware attention baseline included in the comparison.","marker":"[10]"},{"why":"Supplies the stop-gradient operation used in the quantization and commitment losses.","marker":"[31]"}],"fun_headline_variants":["Semantic IDs give LLMs a 16% boost in next-POI picks","Semantic POI IDs sharpen next-location forecasts by up to 16%","LLMs predict next POI better when places share semantic codes","Semantic ID codes make next-POI recommendation up to 16% more accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a semantic codebook and the LLM's SID vocabulary trained on one city's POIs can be applied to another city's POIs and still yield semantically meaningful IDs; the paper does not specify how target-city POIs are quantized with a source codebook, how features are normalized, or how the LLM can emit codewords it never saw for the target domain.","fun_headline_variants_meta":{"raw":{"variants":["Semantic IDs give LLMs a 16% boost in next-POI picks","Semantic POI IDs sharpen next-location forecasts by up to 16%","LLMs predict next POI better when places share semantic codes","Semantic ID codes make next-POI recommendation up to 16% more accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000698,"raw_usage":{"total_tokens":3208,"prompt_tokens":1052,"completion_tokens":2156,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":668,"completion_tokens_details":{"reasoning_tokens":2082}},"tokens_in":668,"tokens_out":2156,"duration_ms":15157,"temperature":1.0,"reasoning_tokens":2082,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:42:55.900327+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain GNPR-SID on NYC with the learned codebook but with the codeword indices randomly permuted within every codebook level, so each POI keeps its quantized feature vector yet its SID string loses prefix semantics; if top-1 accuracy remains near 0.3618 rather than dropping toward the random-ID baseline's 0.3372, the reported gain is not caused by semantic prefix sharing.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the New York (NYC) and Tokyo (TKY) check-in datasets used in all experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the California (CA) check-in dataset used in all experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the preprocessing and evaluation protocol, including the filters on POIs and users and the removal of test POIs absent from training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Acc@1 evaluation convention and a time-aware attention baseline included in the comparison."}],"review_version":1}