{"id":"dc07904c-06b2-4f70-a559-89aecacdc128","arxiv_id":"1908.09972","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"CosRec encodes sequences as pairwise item tensors and applies 2D convolutions, achieving reported state-of-the-art results on two benchmark datasets, though without variance estimates or several cited baselines.","lead":"CosRec turns a user's recent item history into a grid of all item pairs and runs a 2D image-style neural network over that grid to predict the next item. On MovieLens-1M and Gowalla it reports large gains over a strong baseline, but it omits several recent comparison models and includes no statistical significance testing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'state-of-the-art' claim is untested against SASRec and BERT4Rec, which are cited but absent from Table 3; without those comparisons under the same protocol the central claim is not supported.","rationale":"The reader's verdict is CONDITIONAL, and I agree with that disposition. The weakest point is not the convolutional construction described in Sections 3.2 and 3.3, which is internally coherent and accompanied by a released code link; it is the empirical support for 'state-of-the-art.' The paper's own related-work section names two stronger recent sequence models, SASRec and BERT4Rec, that are absent from Table 3. Since the central claim is a ranking claim, excluding those methods makes the claim conditional on a selected baseline list. A single head-to-head rerun would settle the issue: if CosRec beats both under the exact Caser protocol, the claim is restored; if not, the paper should be read as 'outperforms the implemented baselines' rather than 'state of the art.' The reader's weakest assumption about protocol comparability is related, but the missing-baseline gap is more decisive because even a perfectly clean protocol would not tell us whether CosRec is state of the art. Hence I mark partial agreement and keep the verdict unchanged.","tokens_in":6573,"tokens_out":7276,"duration_ms":79356,"concrete_test":"Run SASRec (Kang and McAuley, 2018) and BERT4Rec (Sun et al., 2019) on the same preprocessed ML-1M and Gowalla data, using the exact Caser/CosRec evaluation script: first 80% of actions for training/validation, last 20% for test, L=5, T=3, full-item ranking, and the same MAP, Precision@N, and Recall@N metrics as Table 3. Use the released implementations or faithful re-implementations with hyperparameters tuned on the validation portion. If either omitted baseline matches or exceeds CosRec on both datasets, the 'state-of-the-art' claim is unsupported; if CosRec still leads on both datasets, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and Section 5, is that CosRec 'outperforms both conventional methods and recent sequence-based approaches' and achieves 'state-of-the-art performance.' However, Section 2 cites SASRec and BERT4Rec as related sequence models, and neither appears in Table 3. The numbers in Table 3 are only meaningful relative to the baselines actually run; because SASRec and BERT4Rec use different evaluation conventions in their original papers (e.g., leave-one-out or sampled negative items), published numbers cannot be substituted for a direct comparison. The paper also provides no statistical significance testing or repeated-seed variance, so even the reported margin over Caser could be within run-to-run noise. This is not an internal inconsistency, but it is a correctness risk: if either omitted baseline outperforms CosRec under the identical 80/20 split and full-item MAP/Precision/Recall protocol, the abstract's 'state-of-the-art' claim is false. The pairwise-encoding architecture may be sound; the empirical claim is the load-bearing part.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CosRec, a sequential recommendation model that encodes the last L items into an L x L x 2d tensor of pairwise concatenated item embeddings, applies standard 2D convolutions to this tensor, and conditions the resulting sequential representation on a user embedding for next-item prediction. The model is trained with binary cross-entropy loss and negative sampling. Experiments on ML-1M and Gowalla compare CosRec against PopRec, BPR, FMC, FPMC, GRU4Rec, Caser, and an MLP-based ablation CosRec-base, reporting MAP, Precision@N, and Recall@N. The authors claim state-of-the-art performance and include a filter visualization intended to show that pairwise encoding captures non-adjacent, \"skip\" dependencies.","tokens_in":6814,"tokens_out":3651,"duration_ms":36606,"significance":"If the empirical results are robust, the paper makes a useful empirical contribution: pairwise encoding followed by standard 2D convolution is a simple and plausible alternative to Caser-style vertical/horizontal filters and NextItNet-style 1D dilated convolutions. The authors should be credited for releasing code, following the Caser evaluation protocol, including an ablation of the 2D CNN module, and reporting metrics on two public benchmarks. However, the contribution is primarily empirical, and the central claim of state-of-the-art performance is currently supported only against the baselines that were actually run; the strongest cited recent sequence models are not evaluated, and no uncertainty quantification is provided.","major_comments":[{"comment":"The abstract and Section 5 claim that CosRec 'outperforms both conventional methods and recent sequence-based approaches' and achieves 'state-of-the-art performance,' but SASRec and BERT4Rec, both cited in Section 2 as recent sequence models, are absent from Table 3. Because those models use different evaluation conventions in their original papers, published numbers cannot be substituted for a direct comparison under the protocol of Section 4.1.1. The claim should be either supported by direct same-protocol comparisons with these models or explicitly restricted to the evaluated baselines.","section":"Section 4.2, Table 3"},{"comment":"All metrics in Table 3 appear to be single-run point estimates; no error bars, confidence intervals, or significance tests are reported. Given that Section 4.2 describes the improvement over Caser as 'significant' (e.g., MAP 0.1883 vs. 0.1507 on ML-1M), the absence of variance information makes it impossible to determine whether the margins are beyond run-to-run noise. The authors should provide results over multiple random seeds and, where appropriate, paired significance tests.","section":"Section 4.2, Table 3"},{"comment":"The experimental protocol is not fully specified in ways that directly affect comparability. The manuscript states that the first 80% of each user's actions are used for training and validation, but it does not state the validation split or early-stopping rule, nor does it report how the baseline hyperparameters were selected or tuned. In addition, the paper says the model predicts the next T=3 items at once, but it does not explain how multiple target timestamps per user are incorporated into the reported MAP, Precision@N, and Recall@N. Without these details, the reader must assume that the protocol is exactly the Caser protocol; if any part differs, the reported margins over the baselines could be overstated.","section":"Section 4.1.1 and 4.1.3"}],"minor_comments":[{"comment":"The formula contains 'loд' instead of '\\log', and the notation 'j < Su' for sampled negative items is introduced without a formal definition; please clarify the negative sampling notation.","section":"Eq. (1)"},{"comment":"The definition of the pairwise tensor says the (i,j)-th vector is the concatenation of embeddings of 'the item pair (i,j), i,j in (S^u_{t-L}, ..., S^u_{t-1})', which is imprecise because i and j are indices, not items; please index the tensor positions explicitly.","section":"Section 3.2"},{"comment":"The kernel sizes are listed as '1 x 1' and '3 x 3' but the first layer is described as 'enriching feature representations'; it would help to state explicitly that these are 2D kernels applied over the L x L spatial dimensions with 2d input channels.","section":"Table 1"},{"comment":"The caption says 'Darker colors indicate higher values' while the colorbar shows values increasing from 0.000 to 0.027, but the actual colormap direction is not labeled on the figure; please clarify or label the colorbar.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical architecture contribution, and the core idea is clearly presented. The main risk is that the 'state-of-the-art' claim depends on omitted comparisons and single-run results; if the authors cannot add the missing baselines or rigorous uncertainty quantification, the claim should be substantially softened. Given that this is a short CIKM paper, the missing comparisons may be addressable in a revision, but they are load-bearing for the abstract's central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: CosRec is a legitimate small architectural contribution. Pairwise-encoding the last L items into an L×L×2d tensor and then sliding a standard 2D CNN over it is a real twist on Caser and NextItNet, not a re-coordinatization. Within the baselines actually run, it beats Caser, GRU4Rec, and FPMC by consistent margins on ML-1M and by smaller margins on Gowalla. The 'state-of-the-art' claim is bigger than the evidence, but the method itself is worth engaging.\n\nWhat is genuinely new: prior CNN recommenders operate on the item embedding matrix directly, with Caser's vertical/horizontal filters and NextItNet's 1D dilated convolutions. CosRec constructs pairwise combinations of all positions first, so small 2D kernels can see skip/adjacent correlations. The CosRec-base result—an MLP on the same pairwise encoding, without any CNN—also beating Caser is a clean ablation, though it cuts both ways: it suggests a lot of the gain comes from the pairwise encoding, not from the 2D convolution per se. The code is linked and the hyperparameters are stated, so reproduction is plausible.\n\nThe soft spots, in order of severity. First, the missing recent baselines. SASRec and BERT4Rec are cited in related work, and the abstract and Section 5 claim superiority over 'recent sequence-based approaches,' but neither appears in Table 3. Their original papers use different evaluation conventions, so you cannot patch the table with published numbers. If either method outperforms CosRec under the same 80/20 split and full-item MAP/Precision/Recall protocol, the abstract's claim is false. This is the load-bearing gap. Second, there are no error bars, confidence intervals, or significance tests; every number in Table 3 is apparently a single-run point estimate. On Gowalla, where improvements are 5–13%, run-to-run noise could matter. Third, hyperparameter fairness is not documented: the paper says it follows Caser's protocol but does not say how GRU4Rec and Caser were tuned, leaving the usual under-tuning worry. Minor: Table 1 labels the input as D×5×5 while the pairwise tensor has 2d channels per cell; likely a typo, not a substantive issue.\n\nOn balance, the central architectural claim is sound, and the included experiments support it over the baselines that were actually run. The citation pattern is appropriate; no red flags there. The paper is for people working in sequential recommendation and would be a useful read for anyone designing sequence encoders.\n\nRecommendation: send it to peer review, and ask for SASRec and BERT4Rec under the same protocol, plus repeated-seed variance. That kind of revision is exactly what this paper needs; rejecting the idea would be wrong.","headline":"A clean, small architectural idea with a credible ablation, but the 'state-of-the-art' claim outruns the experiments because SASRec and BERT4Rec are cited but never run and there are no repeated trials.","tokens_in":7336,"tokens_out":2663,"would_cite":true,"duration_ms":29680,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CosRec claims that encoding item histories as pairwise tensors lets a standard 2D CNN outperform prior sequential recommenders on MovieLens-1M and Gowalla.","keywords":["sequential recommendation","2D convolutional neural networks","pairwise encoding","next-item prediction","implicit feedback","MovieLens-1M","Gowalla","top-N metrics"],"falsifier":"Run the released code on the same 80/20 split and the same T=3 prediction protocol with the same hyperparameter search for every baseline, and check whether CosRec still exceeds Caser, GRU4Rec, and FPMC on both ML-1M and Gowalla; if the margins disappear under a matched protocol, the central claim would be refuted.","tokens_in":6372,"feed_emoji":"🎯","tokens_out":7646,"duration_ms":70156,"temperature":0.7,"pith_summary":"This paper tries to establish that the one-directional chain structure assumed by most sequential recommender models is not necessary, and that relaxing it improves next-item prediction. It proposes CosRec, which turns a user's last L interactions into a three-way tensor of concatenated item-pair embeddings, then applies standard 2D convolutional filters to this image-like map before predicting the next items. On MovieLens-1M and Gowalla, the paper reports that CosRec exceeds all evaluated baselines, including the CNN-based Caser, with the largest margin on MovieLens-1M (25.0% MAP improvement over Caser). A version of CosRec that replaces the 2D CNN with an MLP also outperforms prior methods on MovieLens-1M, which the authors read as evidence that the pairwise encoding itself supplies much of the benefit. If correct, the result means sequential recommendation can borrow off-the-shelf 2D CNN architectures rather than special filter designs.","feed_headline":"Pairwise encoding lets 2D CNNs beat sequential baselines","feed_subtitle":"On MovieLens-1M, CosRec reports 25% higher MAP than Caser by convolving item pairs, not raw sequences.","key_machinery":"The load-bearing mechanism is the pairwise encoding module that converts an ordered sequence into an image-like three-way tensor. For the previous $L$ items, the tensor $T^L_{(u,t)}\\in\\mathbb{R}^{L\\times L\\times 2d}$ has entry $(i,j)$ equal to the concatenated embedding $[\\mathbf{e}_i;\\mathbf{e}_j]$, so every ordered pair of recent items becomes a position in a feature map. Standard 2D convolutional kernels then slide over this map; because pairs like (camera, lens) are adjacent in the tensor even when an unrelated item (bike) sits between them in the original sequence, small filters such as $3\\times3$ can capture skip correlations that a chain-structured model would have to propagate through the intervening item. The same encoding turns the problem into one that can reuse ordinary CNN architectures, including stacking and padding.","core_discovery":"On the paper's own terms, the discovery is that pairwise encoding of a user's recent history is a stronger substrate for sequential recommendation than the ordered embedding matrix used by Caser and NextItNet. CosRec constructs a tensor $T^L_{(u,t)}\\in\\mathbb{R}^{L\\times L\\times 2d}$ whose $(i,j)$-th vector is the concatenation $[\\mathbf{e}_i;\\mathbf{e}_j]$ of the embeddings of the $i$-th and $j$-th previous items, applies two blocks of $1\\times1$ and $3\\times3$ 2D convolutions, concatenates the resulting sequential vector with the user embedding, and predicts the next $T=3$ items under binary cross-entropy with negative sampling. The paper reports that this outperforms all tested baselines on MovieLens-1M and Gowalla, including a 25.0% MAP gain over Caser on ML-1M, and that even the MLP-based CosRec-base beats prior methods on ML-1M.","pith_inferences":["The pairwise tensor has $L^2$ entries, so CosRec's memory and compute grow quadratically with the Markov order $L$; a natural extension the paper leaves implicit is replacing the dense tensor with sparse or windowed pairs, which would keep the skip-capture property at much longer horizons.","The reported gap is much larger on ML-1M than Gowalla, suggesting the benefit of pairwise encoding is tied to data density and reliable item co-occurrence; this predicts that on sparser or noisier datasets the margin over Caser would shrink, and regularizing the pairwise weights might restore it.","Since CosRec-base already beats prior methods on ML-1M, a focused ablation varying the number of 2D convolution blocks and filter sizes could separate how much of the gain is the pairwise encoding and how much is the convolution itself; the paper does not report such an ablation."],"forward_implications":["Sequential recommenders can stop relying on special vertical/horizontal or dilated 1D filter shapes and instead reuse standard 2D CNN architectures, including deeper stacks.","Irrelevant or noisy 'skip' items in a user's history no longer need to interrupt the signal between two related items, because the pairwise tensor makes those items adjacent to each other.","Because the architecture supports stacked convolutions with padding, it can be extended to longer histories and larger data streams than a single-layer vertical-filter design.","Even a plain MLP over the pairwise encoding can outperform strong sequential baselines on dense data, suggesting that how the sequence is represented matters as much as the convolution operator."],"supporting_citations":[{"why":"Supplies the Caser baseline and the preprocessing and evaluation protocol that CosRec adopts.","marker":"[14]"},{"why":"GRU4Rec is the recurrent baseline that CosRec reports beating on both datasets.","marker":"[5]"},{"why":"Supplies the FMC and FPMC Markov-chain baselines that CosRec claims to outperform.","marker":"[12]"},{"why":"BPR is the non-sequential personalized ranking baseline against which the value of order is demonstrated.","marker":"[11]"},{"why":"MovieLens-1M dataset used for the main evaluation.","marker":"[2]"},{"why":"Gowalla dataset used for the second evaluation.","marker":"[1]"}],"fun_headline_variants":["2D CNN on pairwise tensor lifts sequential recommendation","CosRec: using 2D convolutions on item pairs beats sequence models","Pairwise encoding with 2D CNN enhances next-item prediction","Sequential recommendation gains from 2D convolutional pairs","2D CNN with pairwise interactions outperforms recurrent baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the evaluation protocol is exactly comparable to Caser's—first 80% of each user's actions for training, last 20% for testing, with all baselines tuned under the same conditions; if any baseline was under-tuned or the metric computation differs, the reported margins would be overstated.","fun_headline_variants_meta":{"raw":{"variants":["2D CNN on pairwise tensor lifts sequential recommendation","CosRec: using 2D convolutions on item pairs beats sequence models","Pairwise encoding with 2D CNN enhances next-item prediction","Sequential recommendation gains from 2D convolutional pairs","2D CNN with pairwise interactions outperforms recurrent baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000703,"raw_usage":{"total_tokens":3154,"prompt_tokens":908,"completion_tokens":2246,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":2163}},"tokens_in":524,"tokens_out":2246,"duration_ms":17331,"temperature":1.0,"reasoning_tokens":2163,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:55:34.291422+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released code on the same 80/20 split and the same T=3 prediction protocol with the same hyperparameter search for every baseline, and check whether CosRec still exceeds Caser, GRU4Rec, and FPMC on both ML-1M and Gowalla; if the margins disappear under a matched protocol, the central claim would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Caser baseline and the preprocessing and evaluation protocol that CosRec adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the FMC and FPMC Markov-chain baselines that CosRec claims to outperform."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MovieLens-1M dataset used for the main evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gowalla dataset used for the second evaluation."}],"review_version":1}