{"id":"86733ebb-ab0e-49da-8361-19b626a03550","arxiv_id":"2412.16502","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"STKDRec combines a spatial-temporal knowledge graph encoder with a spatial-enhanced transformer via knowledge distillation and reports consistent ranking improvements across three takeaway recommendation datasets.","lead":"This paper introduces STKDRec, a two-stage model that first trains a graph encoder on a spatial-temporal knowledge graph and then distills that knowledge into a transformer that models users' takeaway purchase sequences with spatial information. It reports consistent improvements over nine baselines on three food delivery datasets from Ele.me.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential test-label leakage into the STKG is the load-bearing uncertainty: the paper never states that held-out purchases are excluded from the teacher graph.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the paper never states that the held-out last purchase is excluded from the STKG used to pre-train the teacher. I agree that this is the most serious threat to the central claim because it can inflate every result in Table 2 through the teacher's soft labels, not just one metric. I do not see a stronger challenge: the architecture is coherent, the ablations are consistent with the design, and the absence of error bars, while important for the word 'significantly,' is secondary to a possible test-label leak. Since the concern is resolvable by inspecting the public code and would decide between ACCEPT and REJECT, the reader's CONDITIONAL verdict is appropriate. I therefore keep the existing verdict unchanged.","tokens_in":12041,"tokens_out":4660,"duration_ms":46053,"concrete_test":"Inspect the released code's data pipeline and verify whether the STKG is constructed before or after the train/validation/test split. Concretely, insert an assertion before STKG construction on the Wuhan dataset: for every user, the user's last purchased takeaway must not appear as the tail v in any (u, time, v) or (u, distance, v) triple used to build the STKG. If the assertion fails, rebuild the STKG with all test purchases excluded, retrain teacher and student with the same hyper-parameters, and compare Table 2's Wuhan HR@10 and NDCG@10 against BSARec. If the gap largely disappears, the reported superiority is an artifact of leakage; if the margins persist, the leakage concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that STKDRec 'significantly outperforms' all baselines on all 18 metric-dataset cells. That claim is load-bearing on the teacher never seeing the held-out last purchase. The data-processing paragraph only says that the last purchase is test data, the second-to-last is validation, and the rest is training; the Spatial-Temporal Knowledge Graph section defines triples of the form (u, time, v) and (u, distance, v) without stating that test purchases are removed before graph construction. If the STKG contains those test edges, then during pre-training Eq. (10) trains the teacher with the true label of the item that will be used for evaluation, and the teacher's soft labels in Eq. (4) can assign high probability to the held-out item via graph paths such as training-item → user → test-item, which are reachable at sampling depth m=2. The student learns those soft labels through the KL distillation loss in Eq. (11), so the Table 2 margins (e.g., Wuhan HR@10 0.8229 versus 0.7958 for BSARec) could reflect answer leakage rather than learned spatial-temporal knowledge. This is a correctness risk because it directly bears on whether the empirical contribution exists at all.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes STKDRec, a two-stage takeaway recommendation model. In the first stage, a spatial-temporal knowledge graph (STKG) encoder is pre-trained to capture high-order spatial-temporal and collaborative associations from triples such as (user, time, item) and (user, distance, item). In the second stage, a spatial-temporal Transformer (ST-Transformer) is trained as a student, using a spatial-enhanced sequence representation that combines region and distance embeddings, and it is supervised by both ground-truth next-item labels and a KL distillation loss against the teacher's soft labels. The method is evaluated on three Ele.me datasets (Wuhan, Sanya, Taiyuan) against nine baselines, with HR@k and NDCG@k metrics, plus ablations and a parameter sensitivity study. Code is released.","tokens_in":12278,"tokens_out":9649,"duration_ms":88487,"significance":"If the empirical results are not confounded by label leakage, the paper makes a practical contribution by combining graph-based and sequence-based spatial-temporal modeling through knowledge distillation, with a plausible computational-efficiency argument and broad evaluation across three datasets and nine baselines. The ablations are helpful in isolating the contributions of the spatial position embedding and the distillation loss, and the released code is a strength. However, the headline claim that STKDRec 'significantly outperforms' state-of-the-art baselines is currently not well supported because of a potentially load-bearing data-leakage issue in the STKG construction and because the experimental protocol lacks statistical rigor and omits the closest spatial-temporal baselines.","major_comments":[{"comment":"The data-split description in the Datasets paragraph states only that the last purchased takeaway per user is test data, the second-to-last is validation, and the rest is training; it never states that the STKG is constructed exclusively from the training portion. The STKG contains triples (u, time, v) and (u, distance, v), and the subgraph sampling procedure treats each sequence item as a center and recursively samples neighbors to depth m=2. Because the user node is adjacent to every item the user purchased, a depth-2 path from any context item to the target item passes through the user. If the STKG includes held-out purchases, the teacher's soft labels in Eq. (4) can assign high probability to the test item, and the distillation loss in Eq. (11) can leak that answer into the student; if the STKG excludes held-out purchases but still contains training-time purchase edges, Eq. (10) still allows the teacher to see the answer to each training prediction. The authors must specify how the STKG is split and should remove the user-to-target edge for the item being predicted. As written, the Table 2 margins cannot be cleanly attributed to learned spatial-temporal knowledge.","section":"Spatial-Temporal Knowledge Graph Encoder; Eq. (4), (10)-(13)"},{"comment":"The abstract and the Experimental Results section use the phrase 'significantly outperforms', but Table 2 reports a single run with no error bars, standard deviations, or significance tests. Several margins are small enough that they could be within seed-to-seed variation; for example, Sanya HR@5 is 0.8770 for STKDRec versus 0.8623 for BSARec, and Taiyuan HR@10 is 0.8789 versus 0.8661. The authors should report mean and standard deviation over multiple random seeds and, if 'significant' is claimed, a paired statistical test.","section":"Experimental Results; Table 2"},{"comment":"The Related Work discusses StEN (Lin et al. 2022) and BASM (Du et al. 2023a) as spatial-temporal methods for online food ordering, yet neither method appears in the baseline comparison in Table 2. Since the paper's central claim is improved spatial-temporal takeaway recommendation, omitting these closest spatial-temporal baselines leaves the 'state-of-the-art' claim incomplete. The authors should add these methods or explicitly justify why they are not applicable to this sequential recommendation setting.","section":"Related Work; Table 2"},{"comment":"The Metrics paragraph says that evaluation pairs the actual test takeaway with 100 randomly sampled negative takeaways and ranks them. This candidate-sampling protocol yields different absolute HR/NDCG values than full-corpus ranking and can favor models that rank popular items highly. The text does not explicitly state that this protocol is applied identically to every baseline method, so the comparisons in Table 2 may not reflect the top-k recommendation task as defined in the Problem Formulation. The authors should confirm that all methods use the same sampling procedure and, ideally, also report full-corpus metrics.","section":"Experimental Setting; Metrics"}],"minor_comments":[{"comment":"The ground-truth label vector Y_x is used in the pre-training, distillation, and recommendation losses but is never formally defined; the authors should state that it is the one-hot encoding of the next purchased takeaway in the training sequence.","section":"Eq. (10)-(12)"},{"comment":"Figure 3 uses 'Recall@10' on the y-axes, while the metrics are defined as HR@k (Hit Rate) in the Experimental Setting; these names should be made consistent.","section":"Figure 3"},{"comment":"The ablation discussion refers to a variant '-w/o KD+SP', while Table 3 labels the same variant '-w/o SP+KD'; the notation should be unified.","section":"Ablation Studies; Table 3"},{"comment":"The '-w/o SP+KD' row in Table 3 is numerically identical to the SASRec row in Table 2 for all datasets (e.g., Wuhan HR@10=0.7796 and NDCG@10=0.6911). The authors should clarify whether this variant is exactly SASRec and, if so, why it is not listed among the baselines in Table 2.","section":"Table 3; Table 2"},{"comment":"The label 'Absolute Positon Embedding' in Figure 2 contains a typo and should read 'Absolute Position Embedding'.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The primary risk in this manuscript is the unstated handling of held-out purchases in the STKG construction; the stress-test concern about test-label leakage is real and must be addressed by a clear graph-split statement and, ideally, rerunning the experiments after removing target edges. The single-run evaluation and the absence of significance tests also need strengthening before the 'significantly outperforms' claim can be accepted. The paper is otherwise within scope and the core idea is viable if the leakage concern is resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a competent, incremental recommendation paper with a clean two-stage design, three real-world datasets, nine baselines, ablations, and a public code link. The genuinely new piece is the spatial position embedding that mixes region and distance embeddings, plus the specific recipe of pre-training a GNN on a spatial-temporal KG and distilling its soft labels into a lightweight Transformer. That recipe is sensible and the ablations are consistent with it.\n\nWhere it falls short is in the evidence supporting the word 'significantly' and in one glaring ambiguity. There are no error bars or significance tests anywhere; the 'significantly outperforms' in the abstract is just a claim. That's a common sin in this literature, but it matters here because the margins over the best baseline are modest (e.g., Wuhan HR@10 0.8229 vs 0.7958). The bigger issue is the potential test-label leakage. The paper designates the last purchase as test and the second-to-last as validation, but the STKG construction section never says that test purchases are excluded from the graph triples. In a KG where a user node connects to a takeaway node via a time/distance relation at purchase time, a 2-hop path from a training item to the test item is reachable. The teacher's soft labels in Eq. (4) could then assign high probability to the held-out item, and the student inherits that through the distillation loss in Eq. (11). The authors probably did the standard thing and built the KG from training data only, but they need to say so explicitly. This is a clarifying statement away from being fixed, but as written it undermines the empirical contribution.\n\nI also agree with the reader that the two closest spatial-temporal baselines (StEN and BASM) are discussed in related work but absent from Table 2. Their absence is hard to justify without a sentence explaining why. The candidate-sampling evaluation with 100 negatives is standard and not a real flaw.\n\nBottom line: this paper deserves a serious referee. If the leakage ambiguity is resolved and the comparison is expanded, the contribution is real though incremental. Send it out, but the reviewer needs to pin the authors down on the KG construction and ask for at least one significance test or variance estimate.","headline":"A clean two-stage distillation recipe for spatial-temporal recommendation that is one clarifying sentence away from being trustworthy.","tokens_in":12793,"tokens_out":2268,"would_cite":true,"duration_ms":19772,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"STKDRec, a two-stage teacher-student model, outperforms nine takeaway recommenders on three real-world city datasets by distilling spatial-temporal knowledge from a knowledge graph into a lightweight Transformer.","keywords":["takeaway recommendation","sequential recommendation","knowledge distillation","knowledge graph","spatial-temporal modeling","Transformer","graph neural networks","geohash"],"falsifier":"Rebuild the STKG after removing, for every user, all triples that mention their last purchased takeaway, retrain the teacher and student, and re-evaluate. If HR@10 and NDCG@10 for STKDRec on Wuhan, Sanya, and Taiyuan drop to within noise of the best baseline, the original result depended on leakage. Additionally, check the teacher's top-k soft-label probabilities for test items: high mass on a user's held-out purchase before any training update would be direct evidence of contamination.","tokens_in":1587,"feed_emoji":"🍜","tokens_out":2332,"duration_ms":59442,"temperature":0.7,"pith_summary":"The paper tries to show that a takeaway recommender can combine the relational structure of a knowledge graph with the sequential logic of order histories, without paying the full inference cost of graph neural networks. It proposes STKDRec, which pre-trains a spatial-temporal knowledge graph encoder as a teacher and distills its knowledge into a lighter spatial-temporal Transformer that reads purchase sequences. Across the Wuhan, Sanya, and Taiyuan datasets, STKDRec ranks first on every reported metric, for example raising Wuhan HR@10 from 0.7976 (best baseline) to 0.8229. The spatial position embeddings for region and distance, plus the distillation loss, carry the performance gain.","feed_headline":"Teacher-student model beats nine takeaway recommenders","feed_subtitle":"Graph knowledge is distilled into a lightweight Transformer, lifting top-10 hit rates above all baselines.","key_machinery":"The machinery is a two-stage teacher-student setup. The teacher is a GNN-based STKG encoder that aggregates messages over a sampled subgraph of user-takeaway-time-distance-attribute triples, producing user-specific gated representations and soft label distributions over all takeaways. The student is a spatial-temporal Transformer that adds a learnable spatial position embedding (a linear combination of embeddings for the geohash region and the spherical distance of each purchased item) to the usual token and absolute position embeddings, then applies masked self-attention. The two are joined by a distillation loss that minimizes the KL divergence between the teacher's softened soft labels and the student's predictions, $\\mathcal{L}_{KD} = \\mathrm{KL}(Y'_x/\\tau \\,\\|\\, \\hat{Y}_x/\\tau)$, weighted by a coefficient $\\alpha$ against the supervised cross-entropy loss. This carries graph-level collaborative and spatial-temporal knowledge into a lightweight sequential model without running GNN inference at serving time.","core_discovery":"STKDRec's central claim is that spatial-temporal knowledge can be transferred from a graph to a sequence model through distillation, yielding better takeaway recommendations than either representation alone. The paper constructs a spatial-temporal knowledge graph whose edges are user-takeaway purchases with time, distance, and attribute relations, and pre-trains a GNN encoder on sampled subgraphs. A student Transformer then models the user's purchase sequence with learnable spatial position embeddings built from geohash regions and spherical distances, and is trained to imitate the teacher's soft label distribution in addition to the true next-item labels. On the three city datasets the full model outperforms all nine baselines on all eighteen HR@k and NDCG@k cells, with notable margins over the strongest baseline BSARec.","pith_inferences":["A check the paper does not run: the data split marks each user's last purchase as test, but the STKG construction section never states that test purchases are excluded from the graph triples; if they are included, the teacher's soft labels could leak the answer into the distillation loss, inflating the reported margins.","A natural extension is to apply the same distillation scheme to other knowledge-rich side information, such as review text or merchant attributes, where the teacher and student have different inductive biases.","The teacher-student asymmetry also points to a compression direction: the student could be made smaller than the tested 256-dimension setting, trading a little accuracy for larger serving-time savings, a trade-off the paper does not explore."],"forward_implications":["If STKDRec's gains hold, a food-delivery platform can serve a Transformer-level recommender at inference time with the accuracy of a graph model, since the teacher runs only during training.","The method's spatial position embedding suggests a general recipe for injecting continuous geographic variables into sequential recommenders, with applications in ride-hailing, delivery routing, and visit prediction.","The 0.03 to 0.05 absolute improvements in HR@10 over the best baseline, on dense city datasets, indicate that spatial-temporal context carries information that pure sequence models miss.","The parameter sensitivity tests show that performance is robust to the number of sampled neighbors but degrades with large distillation temperature, so practitioners can fix the sampling size while tuning only the temperature."],"supporting_citations":[{"why":"Supplies the GraphSAGE neighborhood sampling used to build the STKG subgraph that the teacher encoder aggregates over.","marker":"Hamilton, Ying, and Leskovec 2017"},{"why":"Defines the teacher-student distillation loss with softened soft labels that STKD adapts to transfer graph knowledge.","marker":"Hinton, Vinyals, and Dean 2015"},{"why":"Provides the BERT4Rec baseline and the absolute position embedding that the ST-Transformer extends with spatial position embeddings.","marker":"Sun et al. 2019"},{"why":"SASRec is the self-attention sequential baseline whose masked attention design the student model builds upon.","marker":"Kang and McAuley 2018"},{"why":"BSARec is the strongest baseline with an attentive inductive bias; its results are the main comparison target STKDRec outperforms.","marker":"Shin et al. 2024"},{"why":"BASM is the prior spatial-temporal takeaway model that the paper argues STKDRec beats by using fine-grained distance information.","marker":"Du et al. 2023a"}],"fun_headline_variants":["Graph distills into Transformer for better takeaway picks","Distilling graph knowledge lifts takeaway recommendations","Two-stage model transfers spatial-temporal know-how to Transformer","STKDRec: graph teacher, sequence student, top recommendations","Spatial-temporal distillation sharpens takeaway recommendations"],"cache_read_input_tokens":14976,"weakest_assumption_plain":"The load-bearing premise is that the spatial-temporal knowledge graph contains no information about the held-out last purchase of any user; if test purchases appear in the graph's triples, the teacher's soft labels can leak the answer into training, and the reported gains would not reflect genuine learning.","fun_headline_variants_meta":{"raw":{"variants":["Graph distills into Transformer for better takeaway picks","Distilling graph knowledge lifts takeaway recommendations","Two-stage model transfers spatial-temporal know-how to Transformer","STKDRec: graph teacher, sequence student, top recommendations","Spatial-temporal distillation sharpens takeaway recommendations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000215,"raw_usage":{"total_tokens":1424,"prompt_tokens":934,"completion_tokens":490,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":414}},"tokens_in":550,"tokens_out":490,"duration_ms":4644,"temperature":1.0,"reasoning_tokens":414,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:30:54.522728+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rebuild the STKG after removing, for every user, all triples that mention their last purchased takeaway, retrain the teacher and student, and re-evaluate. If HR@10 and NDCG@10 for STKDRec on Wuhan, Sanya, and Taiyuan drop to within noise of the best baseline, the original result depended on leakage. Additionally, check the teacher's top-k soft-label probabilities for test items: high mass on a user's held-out purchase before any training update would be direct evidence of contamination.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BSARec is the strongest baseline with an attentive inductive bias; its results are the main comparison target STKDRec outperforms."}],"review_version":1}