Pith. sign in

REVIEW 5 major objections 5 minor 8 cited by

Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single formula predicts recommender ranking performance.

desk verdict A promising data-quality idea wrapped in a predictive law that isn't actually tested; the in-sample fits don't carry the weight of the claims. read the letter →

arxiv 2412.00430 v6 pith:OXQNM2B3 submitted 2024-11-30 cs.AI cs.IR

Tingjia Shen , Hao Wang , Chuhan Wu , Jin Yao Chin , Wei Guo , Yong Liu , Huifeng Guo , Defu Lian
show 2 more authors
Ruiming Tang Enhong Chen
This is my paper · ORCID
classification cs.AIcs.IR
keywords sequentialrecommendationscalinglawsperformancelawapproximateentropydataqualityhitrateNDCGtransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a Performance Law for sequential recommendation models: a closed-form equation that predicts ranking metrics (HR@10 and NDCG@10) from the number of transformer layers, the item embedding dimension, the number of training tokens, and a data-complexity measure called Approximate Entropy. The practical payoff would be large: a few small training runs plus cheap data statistics would let developers forecast accuracy, locate the best model configuration, and avoid exhaustive sweeps. The authors argue that ordinary scaling laws fail for recommenders because loss and ranking performance diverge and because data quality matters as much as data quantity. They fit the law on four datasets and report that it tracks HR@10 and NDCG@10 across dataset scales and model sizes, including the U-shaped decay that appears when models grow too large.

What carries the argument

The load-bearing object is Equation (7), a parametric curve with a depth term in $N$, an embedding term in $d_{\mathrm{emb}}$, and a data term in $D' = \#\mathrm{Tokens}\cdot\mathrm{ApEn}'$. Approximate Entropy (with tolerance $r=0$ here) is a time-series regularity statistic: it counts how often nearby patterns recur, and the paper uses $\mathrm{ApEn}' = 1/\mathrm{ApEn}$ so that larger values mean cleaner, less redundant data. Theorem 3.3 argues that the effective data scale is bounded by tokens times $\mathrm{ApEn}'$, using Kraft's inequality and Markov-chain entropy; Theorem 3.5 bounds loss by $\log(n) + 1/n$ with $n = S_{\max}d_{\mathrm{emb}}$, which produces the log-plus-decay shape of the model-size terms. The paper assumes Performance $= 1 - kL$ to convert loss into ranking metrics, and Equation (7) is the assembled fit.

What would settle it

On a held-out dataset, compute ApEn and token counts, fit Equation (7) using only a few small runs, then predict HR@10 and NDCG@10 at larger depths and embedding sizes; if the predicted values fall outside the fitted residual band, or if the measured loss and HR pairs are visibly non-monotone, the law fails.

Watch

Extended reading notes

Core claim

The central claim is the Performance Law, Equation (7): $$\text{Performance} = w_1\left(\log N + \frac{p_1}{$N^{{w_3}}$}\right) + w_2\left(\log d_{\mathrm{emb}} + \frac{p_1}{d_{\mathrm{emb}}^{w_4}}\right) + \log D' + \frac{p_2}{D'^{w_5}},$$ with $D' = \#\mathrm{Tokens}\cdot \mathrm{ApEn}'$ (where $\mathrm{ApEn}' = 1/\mathrm{ApEn}$, the reciprocal of Approximate Entropy). The paper argues that this single equation describes HR@10 and NDCG@10 of decoder-only transformer sequential recommenders across datasets, model depths, and embedding sizes, and that it can locate both a global optimum and the best configuration under a fixed model-size budget. The law is assembled by combining a log-and-decay loss bound (Theorem 3.5) with a data-scale bound (Theorem 3.3) that replaces raw token count by tokens times reciprocal Approximate Entropy, joined through the assumed linear relationship between ranking performance and loss.

Load-bearing premise

The whole law rests on the assumption that ranking metrics rise and fall linearly with the training loss; if that relationship bends, the predictions have no foundation.

Editorial extensions

If this is right

  • If Equation (7) holds, HR@10 and NDCG@10 for unseen configurations can be predicted from a few small training runs, token counts, and Approximate Entropy, without full grid searches.
  • The same fitted law can identify both the global optimum and the constrained optimum (for example, a fixed depth and embedding size), which the paper demonstrates by selecting a practical online-style configuration.
  • By comparing fitted exponents across model architectures and numerical precisions, the law ranks how much each model family stands to gain from scaling up.
  • Because the data term is $\log D' + p_2/D'^{w_5}$, the law predicts diminishing returns and eventual decay from both model-size and data-size directions, explaining when bigger models or more data stop helping.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated extension of the linear-loss assumption: a calibrated link between softmax loss and thresholded ranking metrics, such as a logistic or sampling-corrected transform, might make the Performance Law hold outside the fitted regimes.
  • Setting Approximate Entropy's tolerance to zero counts only exact repeats; for noisy interaction logs, a small positive tolerance would make the quality measure less brittle and is a testable variant of the paper's recipe.
  • If the effective-data bound is universal, the same $D' = \mathrm{Tokens}\cdot\mathrm{ApEn}'$ substitution could be applied to other recommendation families and other metrics, turning the law into a general data-curation tool (for example, predicting when de-duplicating interactions helps more than adding new ones).
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper introduces a 'Performance Law' for sequential recommendation models, proposing that ranking metrics HR@10 and NDCG@10 can be predicted from model depth N, embedding dimension d_emb, token count, and Approximate Entropy (ApEn) through Eq. (7), with effective data parameter D' = #Tokens · ApEn' = #Tokens/ApEn. The authors give theorems linking ApEn to data scale and loss to model dimensions, fit Eq. (7) on four datasets, and use the fitted law to search for optimal configurations and compare architectures.

Significance. If the Performance Law were valid, it would be practically valuable: a small number of training runs plus token count and ApEn could predict ranking metrics and identify compute-optimal configurations without exhaustive training. The paper also makes a plausible contribution in proposing ApEn as a data-quality measure for recommender data, and it reports a broad experimental matrix across four datasets and several architectures. However, the theoretical derivation has load-bearing gaps and the empirical validation is entirely in-sample, so the central predictive claim is not established by the evidence in the manuscript.

major comments (5)
  1. [Section 3.3, Lemma 3.2, Eqs. (10)-(11)] Lemma 3.2 is false for r<1, the regime used by the paper (ApEn is computed with r=0). With two-point distributions x=(0.5,0.5), y=(0.9,0.1) and r=0.5, the left side is -1 and the right side is approximately -1.737, violating the claimed inequality. The proof reverses the inequality after dividing by the negative number ln r, and it invokes ln(y/x)≤y/x−1 in the wrong direction. Since Theorem 3.3 relies on this lemma to conclude that data scale is bounded by #Tokens·ApEn', the data-parameter part of Eq. (7) is not supported.
  2. [Section 3.3, Theorem 3.3, Eqs. (15)-(21)] Even apart from Lemma 3.2, the proof of Theorem 3.3 contains unjustified steps. The requirement that 'p(si) and pxy are minimal' is undefined and is an assumption rather than a property of the data. In Eq. (19), the inequality log(1/p_xy)≥1 is not valid for arbitrary transition probabilities (it requires p_xy≤1/e), and the claim that 'each term has a minimum value of 1/(|U|·Smax)' is asserted without proof; states with zero observed transitions have p_xy=0. The bound in Eq. (21) that the entropy of the sequence distribution is at least S_max is also not established. The theorem therefore does not prove D=#Tokens·ApEn'.
  3. [Section 3.3, linear performance-loss assumption] The paper states 'we assume a linear relationship between model performance and loss, expressed as Performance = 1−kL'. No derivation, measurement, or citation supports this fixed linear map between sampled-softmax training loss and thresholded ranking metrics HR@10/NDCG@10. The constant k is never fitted or used anywhere in the subsequent equations, and the arguments in Section 3.3 only concern loss, so the transfer from loss scaling to ranking-metric scaling has no demonstrated basis.
  4. [Theorem 3.5, Eq. (24), Eq. (7)] Theorem 3.5 does not justify the functional form of Eq. (7). The proof assumes Zt≤1, which is asserted to be 'common in the overfitting phenomenon' but is not proven for the models studied. Even if Eq. (24) held, it is a bound of the form log(Zt)+1/Zt between two expressions, not an exact additive identity; the final sentence 'Analogously to (Wu & Tang, 2024), we factorize the product into the structure of Equation 7' supplies the missing step by reference to a paper by two of the present authors, with no derivation. The same symbol p1 appears in both the N and d_emb terms of Eq. (7), while the fitted examples in Eqs. (29)-(30) contain additional constants, so even the notational form is inconsistent.
  5. [Section 5.3, Table 2] All reported validation of Eq. (7) is in-sample: the same runs used to fit the seven or more free parameters are those whose R² and fitted curves are displayed, and no configuration is held out. Table 2 shows the fitted exponent w5 ranging from 0.0595 to 70.35 for HR@10 and from -29.95 to 18.46 for NDCG@10 across datasets, with sign changes. A law whose fitted parameters vary this widely across datasets does not support the abstract's claim that the method 'enables accurate predictions across various dataset scales and model sizes'; out-of-sample prediction of held-out configurations or datasets is required and is not reported. Section 5.4.1 similarly reports chosen optima but does not compare predicted metric values with later measurements.
minor comments (5)
  1. [Eq. (7)] The parameter p1 appears in both the N and d_emb terms but likely is intended to be two different parameters; the fitted equations (29)-(30) use different constants, so the notation in Eq. (7) should be corrected.
  2. [Figures 2-5 and Table 1] The caption of Fig. 2 refers to 'KuaiRand-1K' while Table 1 and the text refer to 'KuaiRand-pure'; please make the dataset names consistent across the manuscript.
  3. [References] Several references are incomplete; for example, 'Wang, H., et al.' lacks a year and venue, and the ICML submission template header remains in the arXiv version.
  4. [Section 5.4.2, Table 4] The text states 'both w1 and w2 are positive values,' but Table 4 reports w3 and w4, not w1 and w2; the relationship between the table and the statement needs clarification.
  5. [Table 1] The column heading '1/DL' is not defined in the text or caption, and it is unclear how it relates to the fitted data parameter D' discussed later.

Circularity Check

2 steps flagged · score 6.0 of 10

Eq. 7's functional form is imported from a self-cited prior work, and the optimal-configuration 'prediction' is the argmax of the same in-sample fit.

  1. ansatz smuggled in via citation [Section 3.3, Theorem 3.5 proof, sentence after Eq. 26 (Eq. 7)]
    "Analogously to (Wu & Tang, 2024), we factorize the product into the structure of Equation 7, which represents the ultimate form of our fitting model."

    Equation 7 is the central Performance Law that the paper claims 'enables accurate predictions across various dataset scales and model sizes.' The preceding derivation (Theorem 3.5, Eq. 24) only bounds log(Zt)+1/Zt by a term like log(demb^w4)+1/(demb^w4); it does not produce the additive separation into layer and embedding terms, the log(D')+p2/(D'^w5) data term, or the linear combination with coefficients w1, w2. The paper bridges this gap by appealing 'analogously' to Wu & Tang (2024), a paper by two co-authors of this manuscript (Chuhan Wu and Ruiming Tang). That cited work is itself an empirical proposal, not an independent external theorem, so the claimed functional form is an ansatz imported through a self-citation rather than a derived consequence of the lemmas.

  2. fitted input called prediction [Section 5.4.1, Application 1 (Eqs. 29-30, Table 3)]
    "From the aforementioned fitting, we derived two distinct performance law formulations from HR and NG, as follows: ... From the results in the Table, it is evident that our formulation can achieve performance maxima both globally and under constraints."

    Equations 29-30 are fitted on the same measured runs that appear in Table 3. The 'global optimal' and 'local optimal' configurations are obtained by taking the argmax of these fitted curves, and the table's bottom row is then presented as 'the optimal parameters as calculated using the Performance Law.' Because no configuration is held out and no unseen run is predicted, the claimed prediction reduces to the fitted function by construction. The in-sample R^2 and correlation figures do not establish that Eq. 7 predicts across dataset scales or model sizes; Table 2 additionally shows the fitted exponent w5 varying by orders of magnitude and sign across datasets, underscoring that the law is dataset-specific rather than a general predictive law.

full rationale

The paper has an independent, non-circular contribution in the ApEn data-quality analysis: D' = #Tokens * ApEn' is computed from data and compared to fitted scaling-law parameters in Figure 4, which is a testable empirical check. However, the paper's central predictive claim rests on Eq. 7, and the chain from the lemmas to Eq. 7 is not a derivation: Theorem 3.5 provides only a bound, and the full factorization is imported by analogy from Wu & Tang (2024), prior work by two of the present authors. The validation of the Performance Law is entirely in-sample: the curves are fitted to the same runs whose metrics are plotted, and the optimal-configuration 'prediction' in Section 5.4.1 is simply the argmax of those fitted curves. No held-out configuration, dataset, or external benchmark is used to establish predictive accuracy. This is not a case of deliberate circular definition, but the central claim's predictive content does reduce substantially to the fit and to a self-cited ansatz, so a score of 6 reflects partial circularity.

Assumptions & free parameters 5 free parameters · 6 assumptions · 2 invented entities

The Performance Law rests on a large set of fitted constants and unproven assumptions: the linear loss-performance bridge, the first-order Markov idealization, the condition Zt <= 1, and the jump from a bound to the exact Eq 7 via the authors' own prior performance-law paper. The ApEn quality factor is well defined, but its role as a substitute for data scale is established only by in-sample fits. No free parameter is derived from first principles.

free parameters (5)
  • w1, w2, w3, w4, w5, p1, p2 (Performance Law coefficients) = Example fit (Eq 29): HR = 0.50(34 N^-0.0001 + ln N) - 19.01(3 d^-0.0365 + ln d) + 2.5 D'^-12.74 + ln D' + 628.12
    Seven coefficients in Eq 7 are fitted to the same HR/NDCG curves they are used to predict. Table 2 also fits per-dataset D' and w5 values.
  • slope and intercept of linear relation between fitted data parameter D' and #Tokens/ApEn = not reported
    Section 5.2 and Figure 4 assert a 'clear linear growth trend' between fitted D' and Tokens/ApEn, but the regression parameters and R^2 for this linear relation are not reported.
  • ApEn embedding dimension m = unspecified (r fixed to 0)
    The Approximate Entropy computation requires m and r; only r=0 is stated in Section 3.2.2. Without m, the ApEn values in Tables 1 and 2 cannot be reproduced.
  • data parameter D' fitted per dataset and metric = e.g., KuaiRand-pure HR@10: D'=6074772.573, w5=0.4864; ML-1m HR@10: D'=27496352.75, w5=2.6036
    D' and w5 are fit separately for each dataset and metric to make the Performance Law match; this is exactly fitting the quantity that the theory purports to derive.
  • constant k in Performance = 1 - kL = not reported
    The assumed linear relation is introduced in Section 3.3, but k is never estimated or validated against data.
assumptions (6)
  • domain assumption User sequences are first-order aperiodic stationary Markov chains
    Used in Lemma 3.1 and Theorem 3.3 to equate ApEn with the entropy rate and to bound encoding length. Real recommendation sequences are not first-order Markov; this idealization is not tested.
  • ad hoc to paper HR@10 and NDCG@10 are linear in loss: Performance = 1 - kL
    Stated in Section 3.3 with no derivation. It is the bridge between loss scaling laws and the performance law and is the weakest load-bearing premise.
  • ad hoc to paper Zt <= 1 during overfitting
    Assumed in the proof of Theorem 3.5 to make the bound log(Zt)+1/Zt <= O(log(d * demb^-1/2) + ...) hold; the paper calls this 'common' but gives no condition under which it holds.
  • ad hoc to paper The p(si) and pxy in Theorem 3.3 are minimal, and each transition probability is at least 1/(|U| Smax)
    Introduced in the proof of Theorem 3.3, Eq 17; not justified and generally false for Markov chains with many rare transitions.
  • ad hoc to paper The functional form of Eq 7 follows from Theorem 3.5 by analogy to Wu and Tang 2024
    The theorem only proves existence of exponents bounding log(Zt)+1/Zt; the specific additive log terms, shared p1, and D' term in Eq 7 are asserted, citing the authors' own prior Performance Law paper.
  • standard math Kraft inequality, Gibbs/log-sum inequalities, Stirling approximation
    Standard tools used in Lemma 3.2, Theorem 3.3, and Theorem 3.5. They are standard, but Lemma 3.2 is misapplied for r<1.
invented entities (2)
  • Effective data parameter D' = #Tokens * ApEn' = #Tokens / ApEn
    purpose: Replaces the data scale D in scaling laws to account for data quality and redundancy.
    D' is a constructed proxy whose equivalence to the fitted data parameter is validated only by the in-sample linear fit in Section 5.2 and Figure 4. No out-of-sample dataset or architecture is used to confirm it.
  • ApEn' = 1/ApEn, the inverted Approximate Entropy measure
    purpose: Presented as a data-quality measure where higher values mean less redundant data.
    The inversion is chosen by hand to make larger values correspond to better data. The paper provides no independent criterion that ApEn' is the right quality scale beyond the same fits.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy." pith.science (2026). https://pith.science/paper/OXQNM2B3

@misc{pith2026241200430,
  author       = {Pith},
  title        = {Pith review of: Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OXQNM2B3}},
  note         = {Machine review of arXiv:2412.00430}
}
read the original abstract

Scaling Laws have emerged as a powerful framework for understanding how model performance evolves as they increase in size, providing valuable insights for optimizing computational resources. In the realm of Sequential Recommendation (SR), which is pivotal for predicting users' sequential preferences, these laws offer a lens through which to address the challenges posed by the scalability of SR models. However, the presence of structural and collaborative issues in recommender systems prevents the direct application of the Scaling Law (SL) in these systems. In response, we introduce the Performance Law for SR models, which aims to theoretically investigate and model the relationship between model performance and data quality. Specifically, we first fit the HR and NDCG metrics to transformer-based SR models. Subsequently, we propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics. Our method enables accurate predictions across various dataset scales and model sizes, demonstrating a strong correlation in large SR models and offering insights into achieving optimal performance for any given model configuration.

Figures

Figures reproduced from arXiv: 2412.00430 by the authors.

Figure 1
Figure 1. Distinction between Performance Law and Scaling Law. Performance typically shows decay as the model size increases. However, as the volume of user data increases, more expan￾sive recommendation models are being implemented. The high computational requirements of these recommendation models (Ding et al., 2023) lead to considerable expenses and unpredictability during development. This places added stress on developer… view at source ↗
Figure 2
Figure 2. The relationship between model loss and the number of layers (horizontal axis, H), as well as the embedding dimensions (different colored lines, demb), the plot includes annotations of the coefficient of determination R 2 [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The relationship between model HR performance and the number of layers (x-axis, N), as well as the embedding dimensions (y-axis, demb), the plot includes annotations of the fitted parameter w. 𝑃𝑒𝑟𝑓𝑜𝑟𝑚𝑎𝑛𝑐𝑒 = 𝑓(𝑁, 𝑑𝑒𝑚𝑏 ) + log𝐷 ′ + 𝑝2 𝐷′𝑤5 𝐷′ ∝ 𝑇𝑜𝑘𝑒𝑛𝑠/𝐴𝑝𝐸𝑛 [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The linear correlation between parameter D and Token￾s/Apen. The upper figure validates this relationship within the context of the Scaling Law Loss, while the lower figure verifies it within the Performance Law Metric. enhancements. It marks a significant step forward…
Figure 5
Figure 5. Figure 5: The relationship between model NDCG performance and the number of layers (x-axis, N), as well as the embedding dimensions (y-axis, demb), the plot includes annotations of the fitted parameter w [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FuXi-\beta: Towards a Lightweight and Fast Large-Scale Generative Recommendation Model

    cs.IR 2025-08 conditional novelty 6.0 of 10

    FuXi-β shows that removing query-key attention and using a functional relative time bias makes generative recommendation Transformers faster and, on industrial datasets, more accurate.

  2. Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling

    cs.IR 2025-08 conditional novelty 6.0 of 10

    Generative recommender systems using COSETTE tokenization and the MARIUS architecture reach or exceed the accuracy of a strong ID-based SASRec baseline on standard Amazon benchmarks.

  3. TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation

    cs.IR 2025-02 conditional novelty 6.0 of 10

    TD3 factorizes a synthetic sequence summary into user, time, item, and core factors via Tucker decomposition, and trains recommenders on this summary with a feature-alignment meta-objective.

  4. Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens

    cs.CV 2024-11 conditional novelty 6.0 of 10

    The middle layers of LVLMs process visual information in two stages, and amplifying image attention in the first 'enrichment' stage reduces object hallucinations.

  5. SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Adding a KL penalty between fine-tuned and original model logits on input tokens during RAG fine-tuning reduces catastrophic forgetting while preserving downstream performance.

  6. DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction

    cs.IR 2025-05 conditional novelty 5.0 of 10

    DLF is a CTR prediction architecture that combines low-rank, high-rank, and implicit interaction blocks with layer-wise attention fusion, reporting state-of-the-art results on Criteo, Avazu, Movielens, and Frappe.

  7. Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model

    cs.IR 2025-04 conditional novelty 5.0 of 10

    UniGRF trains one generative model to output both next-item predictions and click probabilities, and reports consistent gains over separate retrieval and ranking models on MovieLens and Amazon-Books.

  8. FuXi-$\alpha$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer

    cs.IR 2025-02 conditional novelty 5.0 of 10

    FuXi-alpha, a sequential recommender with decoupled temporal, positional, and semantic attention channels plus a two-stage FFN, reports gains over HSTU and positive online engagement results.

Reference graph

Works this paper leans on

27 extracted references · 6 canonical work pages · cited by 8 Pith papers

  1. [1]

    Understanding training efficiency of deep learning recommendation models at scale

    Acun, B., Murphy, M., Wang, X., Nie, J., Wu, C.-J., and Hazelwood, K. Understanding training efficiency of deep learning recommendation models at scale. In 2021 IEEE International Symposium on High-Performance Com- puter Architecture (HPCA), pp. 802–814. IEEE,

    work page 2021

  2. [5]

    Compressed interaction graph based framework for multi-behavior recommendation

    Guo, W., Meng, C., Yuan, E., He, Z., Guo, H., Zhang, Y ., Chen, B., Hu, Y ., Tang, R., Li, X., et al. Compressed interaction graph based framework for multi-behavior recommendation. In Proceedings of the ACM Web Con- ference 2023, pp. 960–970,

    work page 2023

  3. [8]

    URL https://doi.org/ 10.1145/3269206.3271761

    1145/3269206.3271761. URL https://doi.org/ 10.1145/3269206.3271761. Hidasi, B., Karatzoglou, A., Baltrunas, L., and Tikk, D. Session-based recommendations with recurrent neural networks. In Bengio, Y . and LeCun, Y . (eds.),4th Inter- national Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings,

    arXiv

  4. [10]

    URL https://doi.org/ 10.1145/2911451.2911489

    1145/2911451.2911489. URL https://doi.org/ 10.1145/2911451.2911489. He, Z., Liu, W., Guo, W., Qin, J., Zhang, Y ., Hu, Y ., and Tang, R. A survey on user behavior modeling in rec- ommender systems. arXiv preprint arXiv:2302.11087,

    arXiv

  5. [11]

    and McAuley, J

    Kang, W.-C. and McAuley, J. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM) , pp. 197–206,

    work page 2018

  6. [12]

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T

    1109/ICDM.2018.00035. Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361,

    arXiv 2018

  7. [13]

    Generalization through memorization: Nearest neighbor language models

    Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M. Generalization through memorization: Nearest neighbor language models. arXiv preprint arXiv:1911.00172,

    arXiv 1911

  8. [15]

    Deep double descent: Where bigger models and more data hurt

    Nakkiran, P., Kaplun, G., Bansal, Y ., Yang, T., Barak, B., and Sutskever, I. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechan- ics: Theory and Experiment, 2021(12):124003,

    2021

Show all 27 references
  1. [16]

    Beyond scaling laws: Understanding transformer performance with associative memory

    Niu, X., Bai, B., Deng, L., and Han, W. Beyond scaling laws: Understanding transformer performance with associative memory. arXiv preprint arXiv:2405.08707,

    arXiv

  2. [18]

    W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al

    Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446,

    arXiv

  3. [22]

    Llama 2: Open foundation and fine- tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288 ,

    arXiv

  4. [23]

    Decoupled representation learning for attributed networks

    Wang, H., Lian, D., Tong, H., Liu, Q., Huang, Z., and Chen, E. Decoupled representation learning for attributed networks. IEEE Transactions on Knowledge and Data Engineering, 35(3):2430–2444, 2021a. Wang, H., Lian, D., Tong, H., Liu, Q., Huang, Z., and Chen, E. Hypersorec: Exp...

    arXiv

  5. [24]

    URL https://doi.org/ 10.1145/3397271.3401142

    1145/3397271.3401142. URL https://doi.org/ 10.1145/3397271.3401142. Wu, C. and Tang, R. Performance law of large language models. arXiv preprint arXiv:2408.09895,

  6. [27]

    Breaking determinism: Fuzzy modeling of sequen- tial recommendation using discrete state space diffusion model

    Xie, W., Wang, H., Zhang, L., Zhou, R., Lian, D., and Chen, E. Breaking determinism: Fuzzy modeling of sequen- tial recommendation using discrete state space diffusion model. arXiv preprint arXiv:2410.23994, 2024a. Xie, W., Zhou, R., Wang, H., Shen, T., and Chen, E. Bridg- ing...

    arXiv

  7. [28]

    A unified framework for adaptive representation enhancement and inversed learn- ing in cross-domain recommendation

    Zhang, L., Wang, H., Zhang, S., Yin, M., Han, Y ., Zhang, J., Lian, D., and Chen, E. A unified framework for adaptive representation enhancement and inversed learn- ing in cross-domain recommendation. arXiv preprint arXiv:2404.00268,

    arXiv

  8. [29]

    X., and Wen, J.-R

    Zhou, K., Yu, H., Zhao, W. X., and Wen, J.-R. Filter- enhanced mlp is all you need for sequential recommenda- tion. In Proceedings of the ACM web conference 2022, pp. 2388–2399,

    2022

  9. [1949]

    K., Lee, J., Lundell, J., Kim, C., Kejariwal, A., and Owens, J

    Lin, Z., Feng, L., Ardestani, E. K., Lee, J., Lundell, J., Kim, C., Kejariwal, A., and Owens, J. D. Building a performance model for deep learning recommendation model training on gpus. In 2022 IEEE 29th International Conference on High Performance Computing, Data, and Analyti...

    2022

  10. [1991]

    Grokking: Generalization beyond overfit- ting on small algorithmic datasets

    Power, A., Burda, Y ., Edwards, H., Babuschkin, I., and Misra, V . Grokking: Generalization beyond overfit- ting on small algorithmic datasets. arXiv preprint arXiv:2201.02177,

    arXiv

  11. [2016]

    10 Submission and Formatting Instructions for ICML 2024 Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D

    URL http://arxiv.org/ abs/1511.06939. 10 Submission and Formatting Instructions for ICML 2024 Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A., et al. Training compute-optimal large langua...

    2024 arXiv

  12. [2017]

    Exploring user retrieval integration towards large language models for cross-domain sequen- tial recommendation

    Shen, T., Wang, H., Zhang, J., Zhao, S., Li, L., Chen, Z., Lian, D., and Chen, E. Exploring user retrieval integration towards large language models for cross-domain sequen- tial recommendation. arXiv preprint arXiv:2406.03085,

    arXiv

  13. [2018]

    Wu, S., Tang, Y ., Zhu, Y ., Wang, L., Xie, X., and Tan, T

    URL http: //arxiv.org/abs/1811.00855. Wu, S., Tang, Y ., Zhu, Y ., Wang, L., Xie, X., and Tan, T. Session-based recommendation with graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 346–353,

    arXiv

  14. [2019]

    ISBN 9781450369763

    Associa- tion for Computing Machinery. ISBN 9781450369763. doi: 10.1145/3357384.3357895. URL https://doi. org/10.1145/3357384.3357895. Tang, J. and Wang, K. Personalized top-n sequential recom- mendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM ...

  15. [2020]

    URL https: //doi.org/10.1145/3426723

    doi: 10.1145/3426723. URL https: //doi.org/10.1145/3426723. Gadre, S. Y ., Smyrnis, G., Shankar, V ., Gururangan, S., Wortsman, M., Shao, R., Mercat, J., Fang, A., Li, J., Keh, S., et al. Language models scale reliably with over-training and on downstream tasks. arXiv preprint...

    arXiv

  16. [2021]

    Reconciling modern machine-learning practice and the classical bias– 9 Submission and Formatting Instructions for ICML 2024 Table

    Belkin, M., Hsu, D., Ma, S., and Mandal, S. Reconciling modern machine-learning practice and the classical bias– 9 Submission and Formatting Instructions for ICML 2024 Table

    2024

  17. [2022]

    Integrating large language models into recommendation via mutual aug- mentation and adaptive aggregation

    Luo, S., Yao, Y ., He, B., Huang, Y ., Zhou, A., Zhang, X., Xiao, Y ., Zhan, M., and Song, L. Integrating large language models into recommendation via mutual aug- mentation and adaptive aggregation. arXiv preprint arXiv:2401.13870,

  18. [2023]

    Understanding emergent abilities of language models from the loss per- spective

    Du, Z., Zeng, A., Dong, Y ., and Tang, J. Understanding emergent abilities of language models from the loss per- spective. arXiv preprint arXiv:2403.15796,

    arXiv

  19. [2024]

    Mdap: A multi-view disentangled and adaptive prefer- ence learning framework for cross-domain recommenda- tion

    Tong, J., Yin, M., Wang, H., Pan, Q., Lian, D., and Chen, E. Mdap: A multi-view disentangled and adaptive prefer- ence learning framework for cross-domain recommenda- tion. arXiv preprint arXiv:2410.05877,

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.