REVIEW 4 major objections 6 minor 16 references
A Deep, Forgetful Novelty-Seeking Movie Recommender Model
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Novelty from the last k movies improves next-movie prediction.
desk verdict A clean, clearly-specified novelty-window idea, but the main comparison confounds the window with the deep architecture, so the central claim needs an ablation before it holds. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Action Novelty Index (ANI) matrix is the load-bearing object: for each user and time step it stores, for every tag, the reciprocal of how many times that tag appeared in the previous k actions plus one. This single formula implements both forgetfulness (only k actions count) and continuous novelty (values are frequencies, not ranks). The User Novelty-Seeking Index (UNI) is a secondary, interpretive quantity derived from entropy of the normalized ANI row and is not used in prediction. The rest of the machinery is a deep network: a CNN over the movie title plus summed tag embeddings forms each movie's latent vector, an MLP over demographics and ANI forms the user's time-varying latent vector, and the predicted rating is their dot product, trained by mean-squared error and gradient descent.
What would settle it
Take users whose histories contain a genre they watched heavily, then abandoned for more than k actions, then revisited. If a small-k DFNSM ranks that genre below a full-history NSM on those return actions, the forgetfulness assumption is contradicted; the same test run on a random sample of hundreds of users, not just the first twenty, would show whether the nDCG gap persists.
Extended reading notes
Core claim
DFNSM's central claim is that next-movie choice is best predicted by combining demographic and movie-content representations with a novelty score computed from a limited memory window: each tag's novelty at time t is $\mathrm{ANI}_{t,i} = \frac{1}{\#x_{t-k,t-1}^{i} + 1}$, where the count covers only actions $t-k$ through $t-1$. The paper claims this 'forgetful' novelty measure is more accurate than the full-history Novelty-Seeking Model because it treats novelty as a continuous scale rather than a ranking and because it drops stale information; NSM's dynamic choice novelty matrix is presented as the special case $k$ equal to the whole sequence length. The model projects the user (demographics plus the current ANI row) and each movie (title plus embedded tags) into a shared latent space, predicts ratings by a dot product, and ranks movies accordingly. The paper's evidence is mean nDCG@all on the first twenty users: 0.3887 for DFNSM versus 0.2696 for NSM and 0.1781 for random ranking.
Load-bearing premise
The load-bearing premise is that a user's next movie choice is set by novelty computed only from the previous k actions, with every older action ignored; if long-ago choices still shape current taste, or if k only works because the evaluation happens on the same few users, the reported 0.3887 advantage is not established.
Editorial extensions
If this is right
- If DFNSM's result holds, next-action recommenders should replace full-history novelty matrices with a sliding k-action window, since the full-history model is a special case with k set to sequence length.
- The reported nDCG advantage implies that novelty, treated as a scale rather than a ranking, carries usable signal for ranking the single next action even when demographics and content features are included.
- The k value that maximizes accuracy can be read as a per-user memory or fickleness parameter, so model comparison should report k separately per user rather than one global value.
- The finding that per-user k produces accuracy spikes while a shared k does not suggests users differ in how quickly past choices stop feeling fresh; recommendation quality may improve by personalizing the window size.
Reading between the lines
- A natural extension the paper does not develop is to make k a learned, per-user parameter rather than a tuned constant; if forgetfulness varies as the UNI plots suggest, a network could output k from the action history.
- The ANI formula could transfer to other sequential choice domains with sparse tag-like attributes, such as news reading or music listening, where a short recency window is computationally cheap.
- A direct test of the paper's scope is to rerun the same comparison on a random sample of hundreds of users rather than the first twenty, which would show whether the nDCG gap persists on longer, heavier histories.
- UNI's entropy-based smoothness could serve as a stable user-level covariate in churn or exploration models, since the paper shows the index is user-distinguishable over time.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the Deep Forgetful Novelty-Seeking Model (DFNSM), a next-movie recommender that combines an Action Novelty Index (ANI) computed from only the k most recent actions, a User Novelty-Seeking Index (UNI) derived from the entropy of ANI rows, and a deep latent-factor architecture using user demographics and movie titles/tags. The model is evaluated on MovieLens 1M against a previously proposed Novelty-Seeking Model (NSM) and a random predictor, reporting mean nDCG@all scores of 0.3887, 0.2696, and 0.1781 respectively for the first 20 users. The paper also studies UNI trends and the sensitivity of performance to the forgetfulness parameter k, concluding that different users have different optimal memory windows.
Significance. If the central claim holds, the paper makes a conceptually interesting contribution: recency-windowed action novelty, rather than full-history novelty, may better capture how users choose their next movie, and a continuously valued novelty measure may outperform rank-based novelty. The formal definitions of ANI and UNI in Sections 3.2.1 and 3.2.2 are clear, and the k-sensitivity analysis in Section 4.4 is an honest attempt to study a model parameter. However, the current evidence is not sufficient to establish the claim: the comparison against NSM conflates the novelty-window change with a change of predictive architecture, the evaluation is limited to 20 users with no uncertainty quantification, and the k-selection procedure uses the same metric on which the model is judged. These issues are fixable within the scope of the manuscript, so the appropriate revision is major rather than rejection.
major comments (4)
- [Section 3.3.1 and Table 3] The main comparison in Table 3 does not isolate the paper's stated contribution. DFNSM differs from NSM in at least two independent ways: (i) ANI uses only the previous k actions (Section 3.2.1) whereas NSM uses full history, and (ii) DFNSM uses an MLP/CNN trained by gradient descent while NSM uses Gibbs sampling. The reported improvement of 0.3887 over 0.2696 could be entirely due to the deep architecture, the windowed novelty, or their interaction. Since the introduction claims that considering only k previous actions improves accuracy, an ablation is necessary: for example, DFNSM with k set to the full sequence length, or NSM fed with windowed DCN values, would disentangle the two factors. Without such an experiment, the central claim is not established.
- [Section 4.2.2 and Table 3] The evaluation on the first 20 MovieLens users is a convenience sample, and the reported superiority of DFNSM over NSM is not accompanied by any measure of variability or significance. Figure 12 shows that NSM outperforms DFNSM on users 7 and 9, but the paper does not report standard deviations, confidence intervals, or a paired significance test across users. Given that the entire empirical case rests on this table, the authors should report per-user variances, run a paired test (e.g., Wilcoxon signed-rank or bootstrap), and ideally evaluate on a larger and more representative user sample. The current presentation does not allow a reader to assess whether the mean difference is robust.
- [Section 4.4] The k-selection procedure in Section 4.4 is circular with respect to the evaluation. The authors train on the first three users and select the k that maximizes nDCG@all on the same users, then interpret the optimal k as the user's 'memory'. Selecting the hyperparameter on the same metric used for the final comparison (Table 3) can overstate the model's performance and makes the 'optimal k measures forgetfulness' claim unfalsifiable without a held-out evaluation. Moreover, the text states that when a single k is used across the first four users the model 'was never able to achieve a high metric score', yet Table 3 reports a single global k (value unspecified) for DFNSM. The manuscript must specify which k was used for Table 3, and should evaluate k selection on a validation set distinct from the test users, reporting the chosen k explicitly.
- [Section 4.2.2, NSM baseline] The NSM baseline is not described with enough detail to be reproduced or judged as a strong baseline. The paper references Zhang et al. [2] but does not state hyperparameters, the number of Gibbs iterations, the handling of the MovieLens tag vocabulary, or whether novelty values were computed from the same tag-based ANI representation used for DFNSM. Without this information, the comparison in Table 3 is not verifiable, and the reader cannot rule out that an unfair or weak configuration of NSM explains the gain.
minor comments (6)
- [Abstract] The abstract says 'watching list' and 'predict the most likely actions', which is imprecise; since the model predicts the rating and ranks movies, the phrasing should be aligned with Section 3.1.
- [Section 4.2.1, Eq. for nDCG@all] The notation in the DCG@all formula uses 'all actions' in the summation limit but does not explicitly state that the summation runs over the full ranked list of movies; also the sentence 'The IDCGall is one because at any time, the user only takes one action' should clarify that the ideal ranking places the chosen item first.
- [Section 4.2.2, Figure 12] Figure 12 is described as showing individual-user nDCG@all values, but the axis labels and legend are not described in the text; please add explicit labels for users and metric values, and consider plotting error bars or a confidence band around the means.
- [Section 4.4, Figure 14] The caption for Figure 14 does not state which nDCG@all variant is plotted, nor whether the curves are averaged over multiple random initializations; since the text mentions random initializers as a source of variation, report the number of runs and display error bars.
- [Section 4.1] The description of data preprocessing (duplicating low ratings and the rating normalization n(r) = r - R_mean) is introduced in the dataset analysis but not referenced in the model section; please make explicit where these steps enter the training pipeline and whether they are applied to both DFNSM and NSM.
- [Conclusion] The word 'phycological' in the conclusion is a typo for 'psychological'; throughout the manuscript, light copyediting is needed for grammar and article usage.
Circularity Check
One definitional step equates the tuned k with user memory; the central DFNSM/NSM prediction comparison is not circular.
-
self definitional
[Section 4.4 (Optimal k-value), paragraphs 1 and 3]
"This k value can also be thought of as how good we assume a user’s memory of past actions is. ... From this, the value of k that produces the best overall metrics score is also the best approximation of this particular person’s memory. ... These would indicate that an optimal k-value exists but is different over many users and shows that different users have different memories."
The paper first defines 'memory' as the k that maximizes the nDCG@all metric, then reports that the k-values producing spikes differ across users and concludes that users have different memories. Since 'memory' was operationalized as the argmax k over the evaluation metric, the existence of an optimal k is guaranteed by construction for any finite grid, and 'different users have different memories' merely restates that the selected argmax values differ. The psychological conclusion is therefore a relabeling of the model-selection outcome rather than an independent empirical measurement. This step is interpretive and does not affect the headline DFNSM-versus-NSM comparison, which is why the overall circularity score is low.
full rationale
The central predictive chain is not circular. DFNSM computes ANI_t from the k prior actions, combines the current ANI row with demographic information through an MLP to form a user representation, and predicts ratings as a dot product with a movie representation; recommendations are rankings of those predicted ratings. This is a standard supervised time-split setup in which novelty features are computed from past actions and no fitted parameter is renamed as the next-action target. The headline result in Table 3 compares DFNSM against an independently described NSM baseline and a random predictor; no text states that the Table 3 k was chosen by maximizing the reported nDCG@all on the same evaluation users. The only concrete reduction found is in Section 4.4, where k is equated with memory and the subsequent 'different memories' finding is true by that definition. The absence of an ablation isolating the windowing effect from the deep architecture is a validity concern, not a circularity, so it does not raise the circularity score. There is no load-bearing self-citation chain in the paper.
Assumptions & free parameters
free parameters (2)
- k (forgetfulness window) =
not reported; selected per user by nDCG@all in Section 4.4
- neural network hyperparameters (embedding size, layer widths, learning rate, batch size, epochs) =
not reported
assumptions (5)
- ad hoc to paper Only the previous k actions affect current novelty; older actions are forgettable.
- ad hoc to paper Tag novelty is 1/(1 + tag count in previous k actions).
- ad hoc to paper Entropy of the Frobenius-normalized ANI row measures user novelty-seeking.
- domain assumption Dot-product similarity between user and movie latent vectors ranks next-movie choice.
- standard math Neural networks can represent the needed predictive functions.
invented entities (2)
-
Action Novelty Index (ANI) matrix
-
User Novelty-Seeking Index (UNI)
Cite this review
Pith. "Pith review of A Deep, Forgetful Novelty-Seeking Movie Recommender Model." pith.science (2026). https://pith.science/paper/FXKPGDHN
@misc{pith2026190901811,
author = {Pith},
title = {Pith review of: A Deep, Forgetful Novelty-Seeking Movie Recommender Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/FXKPGDHN}},
note = {Machine review of arXiv:1909.01811}
}
read the original abstract
As more and more people shift their movie watching online, competition between movie viewing websites are getting more and more intense. Therefore, it has become incredibly important to accurately predict a given user's watching list to maximize the chances of keeping the user on the platform. Recent studies have suggested that the novelty-seeking propensity of users can impact their viewing behavior. In this paper, we aim to accurately model and describe this novelty-seeking trait across many users and timestamps driven by data, taking into consideration user forgetfulness. Compared to previous studies, we propose a more robust measure for novelty. Our model, termed Deep Forgetful Novelty-Seeking Model (DFNSM), leverages demographic information about users, genre information about movies, and novelty-seeking traits to predict the most likely next actions of a user. To evaluate the performance of our model, we conducted extensive experiments on a large movie rating dataset. The results reveal that DFNSM is very effective for movie recommendation.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[2]
Zhang, F., Yuan, N. J., Lian, D., & Xie, X. (2014). Mining novelty-seeking trait across heterogeneous domains. Proceedings of the 23rd International Conference on World Wide Web - WWW 14. doi: 10.1145/2566486.2567976
arXiv 2014
- [1]
-
[3]
J., Xie, X., Chen, E., & Zhou, X
Zhang, F., Zheng, K, Yuan, N. J., Xie, X., Chen, E., & Zhou, X. (2015). A Novelty-Seeking based Dining Recommender System. 1362-1372. 10.1145/2736277.2741095
arXiv 2015
-
[4]
Rosenblatt, F. (1962). Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Washington: Spartan Books
work page 1962
-
[5]
Madden, M. (2014, February 11). The Audience for Online Video-Sharing Sites Shoots Up. Retrieved from https://www.pewinternet.org/2009/07/29/the-audience-for-online-video-sharing-sites-shoots-up/
work page 2014
-
[6]
Zhang, W., Itoh, K., Tanida, J., & Ichioka, Y. (1990). Parallel distributed processing model with local space-invariant interconnections and its optical architecture. Applied Optics, 29(32), 4790. doi: 10.1364/ao.29.004790
-
[7]
Cloninger, R., Przybeck, T. R., & Svrakic D. 1994. The Temperament and Character Inventory (TCI): A guide to its development and use. St. Louis, MO: Center for Psychobiology of Personality, Washington University
work page 1994
-
[8]
Breese, J. S., Heckerman, D., & Kadie, C. (2013). Empirical Analysis of Predictive Algorithm for Collaborative Filtering. UAI
work page 2013
Show all 16 references
-
[9]
Aggarwal, C. C. (2018). Recommender Systems: the textbook. Switzerland: Springer
2018
-
[10]
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. 2017. Neural collaborative filtering. In Proceedings of the WWW. 173–182. https://doi.org/10.1145/3038912.3052569
2017
-
[11]
K., & Roy, D
Dziugaite, G. K., & Roy, D. M. 2015. Neural network matrix factorization. arXiv preprint arXiv:1511.06443 (2015)
2015 arXiv
-
[12]
Ying, H., Zhuang, F., Zhang, F., Liu, Y., Xu, G., Xie, X., … Wu, J. (2018). Sequential Recommender System based on Hierarchical Attention Networks. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. doi: 10.24963/ijcai.2018/546
2018 doi
-
[13]
numpy.linalg.norm
(n.d.). numpy.linalg.norm. Retrieved from https://docs.scipy.org/doc/numpy- 1.13.0/reference/generated/numpy.linalg.norm.html
-
[14]
scipy.stats.entropy
(n.d.). scipy.stats.entropy. Retrieved from https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html
-
[15]
(1970, January 1)
Nielsen, & A., M. (1970, January 1). Neural Networks and Deep Learning. Retrieved from http://neuralnetworksanddeeplearning.com/chap4.html
1970
-
[16]
M., & Konstan, J
Harper, F. M., & Konstan, J. A. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems (TiiS) 5, 4, Article 19 (December 2015), 19 pages. DOI=http://dx.doi.org/10.1145/2827872
2015 doi
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.