{"id":"c1283f53-85b0-4974-89ea-e332ed9a7565","arxiv_id":"1909.01811","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DFNSM combines a windowed, reciprocal-frequency novelty score with a deep latent-factor model and reports nDCG@all of 0.3887 versus 0.2696 for NSM and 0.1781 for random on 20 MovieLens users.","lead":"A movie recommender model that guesses the next film a user will watch by scoring how novel movie genres feel, based only on the user's last k watched films. It reports higher next-movie ranking accuracy than a novelty-seeking baseline and a random predictor on a small 20-user slice of MovieLens data, but with important limitations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported DFNSM advantage over NSM confounds the forgetful k-window with the deep-learning architecture, so the central claim that recency-windowed novelty improves recommendation is not yet established.","rationale":"The reader identified the recency-window assumption as load-bearing, but the more fundamental problem is that the experiment never isolates that assumption. The comparison DFNSM versus NSM varies both the novelty-window and the entire prediction architecture, so even if the numerical result is reproducible, it cannot support the paper's causal claim about forgetfulness. This is not an accusation of misconduct; it is a standard confound. The concern is addressable with a simple ablation (full-history DFNSM or windowed NSM), which makes the appropriate verdict conditional rather than reject. The reader's conditional already reflects a need for more evidence, so I do not change the verdict, but I add a sharper condition: the reported advantage must be shown to come from the k-window, not merely from deep learning capacity. Secondary issues (small sample of 20 users, missing hyperparameters, and k selection tied to the metric) are real but less central to the scientific claim about forgetfulness.","tokens_in":10393,"tokens_out":4239,"duration_ms":48630,"concrete_test":"Re-run the Table 3 evaluation on the same first 20 MovieLens users with two DFNSM configurations: (a) k set to the maximum action-sequence length (effectively full history, no forgetfulness) and (b) k set to the value used for the reported 0.3887 result (or, if not reported, to several values including the per-user optimal k from Section 4.4). If the full-history configuration achieves mean nDCG@all equal to or greater than the windowed configuration within random-initialization variance, then the forgetful mechanism is not responsible for the claimed improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central comparison (Table 3) contrasts DFNSM, a deep MLP/CNN model trained with gradient descent, against NSM, a probabilistic model using full-history novelty and Gibbs sampling. These differ in at least two independent respects: (i) the novelty measure is computed from only the last k actions in DFNSM versus the full action history in NSM, and (ii) the prediction architecture is entirely different. The reported nDCG@all improvement (0.3887 vs 0.2696) could therefore be due to the deep learning component, the windowed novelty, or their interaction. Section 3.3.1 introduces the MLP/CNN architecture without any ablation, and Section 4.2.2 only reports the end-to-end comparison. Section 4.4 examines k-sensitivity but only for the first three users and without a full-history control condition; its own text notes that a single k across users 'was never able to achieve a high metric score,' which further muddies what k was used in Table 3. Because the paper's stated contribution is the 'forgetful' window (Section 1: 'there is no need to consider the impact of actions a long time ago'), the absence of an experiment isolating the window effect leaves the central claim unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Deep Forgetful Novelty-Seeking Model (DFNSM), a next-movie recommender that combines an Action Novelty Index (ANI) computed from only the k most recent actions, a User Novelty-Seeking Index (UNI) derived from the entropy of ANI rows, and a deep latent-factor architecture using user demographics and movie titles/tags. The model is evaluated on MovieLens 1M against a previously proposed Novelty-Seeking Model (NSM) and a random predictor, reporting mean nDCG@all scores of 0.3887, 0.2696, and 0.1781 respectively for the first 20 users. The paper also studies UNI trends and the sensitivity of performance to the forgetfulness parameter k, concluding that different users have different optimal memory windows.","tokens_in":10771,"tokens_out":3072,"duration_ms":34535,"significance":"If the central claim holds, the paper makes a conceptually interesting contribution: recency-windowed action novelty, rather than full-history novelty, may better capture how users choose their next movie, and a continuously valued novelty measure may outperform rank-based novelty. The formal definitions of ANI and UNI in Sections 3.2.1 and 3.2.2 are clear, and the k-sensitivity analysis in Section 4.4 is an honest attempt to study a model parameter. However, the current evidence is not sufficient to establish the claim: the comparison against NSM conflates the novelty-window change with a change of predictive architecture, the evaluation is limited to 20 users with no uncertainty quantification, and the k-selection procedure uses the same metric on which the model is judged. These issues are fixable within the scope of the manuscript, so the appropriate revision is major rather than rejection.","major_comments":[{"comment":"The main comparison in Table 3 does not isolate the paper's stated contribution. DFNSM differs from NSM in at least two independent ways: (i) ANI uses only the previous k actions (Section 3.2.1) whereas NSM uses full history, and (ii) DFNSM uses an MLP/CNN trained by gradient descent while NSM uses Gibbs sampling. The reported improvement of 0.3887 over 0.2696 could be entirely due to the deep architecture, the windowed novelty, or their interaction. Since the introduction claims that considering only k previous actions improves accuracy, an ablation is necessary: for example, DFNSM with k set to the full sequence length, or NSM fed with windowed DCN values, would disentangle the two factors. Without such an experiment, the central claim is not established.","section":"Section 3.3.1 and Table 3"},{"comment":"The evaluation on the first 20 MovieLens users is a convenience sample, and the reported superiority of DFNSM over NSM is not accompanied by any measure of variability or significance. Figure 12 shows that NSM outperforms DFNSM on users 7 and 9, but the paper does not report standard deviations, confidence intervals, or a paired significance test across users. Given that the entire empirical case rests on this table, the authors should report per-user variances, run a paired test (e.g., Wilcoxon signed-rank or bootstrap), and ideally evaluate on a larger and more representative user sample. The current presentation does not allow a reader to assess whether the mean difference is robust.","section":"Section 4.2.2 and Table 3"},{"comment":"The k-selection procedure in Section 4.4 is circular with respect to the evaluation. The authors train on the first three users and select the k that maximizes nDCG@all on the same users, then interpret the optimal k as the user's 'memory'. Selecting the hyperparameter on the same metric used for the final comparison (Table 3) can overstate the model's performance and makes the 'optimal k measures forgetfulness' claim unfalsifiable without a held-out evaluation. Moreover, the text states that when a single k is used across the first four users the model 'was never able to achieve a high metric score', yet Table 3 reports a single global k (value unspecified) for DFNSM. The manuscript must specify which k was used for Table 3, and should evaluate k selection on a validation set distinct from the test users, reporting the chosen k explicitly.","section":"Section 4.4"},{"comment":"The NSM baseline is not described with enough detail to be reproduced or judged as a strong baseline. The paper references Zhang et al. [2] but does not state hyperparameters, the number of Gibbs iterations, the handling of the MovieLens tag vocabulary, or whether novelty values were computed from the same tag-based ANI representation used for DFNSM. Without this information, the comparison in Table 3 is not verifiable, and the reader cannot rule out that an unfair or weak configuration of NSM explains the gain.","section":"Section 4.2.2, NSM baseline"}],"minor_comments":[{"comment":"The abstract says 'watching list' and 'predict the most likely actions', which is imprecise; since the model predicts the rating and ranks movies, the phrasing should be aligned with Section 3.1.","section":"Abstract"},{"comment":"The notation in the DCG@all formula uses 'all actions' in the summation limit but does not explicitly state that the summation runs over the full ranked list of movies; also the sentence 'The IDCGall is one because at any time, the user only takes one action' should clarify that the ideal ranking places the chosen item first.","section":"Section 4.2.1, Eq. for nDCG@all"},{"comment":"Figure 12 is described as showing individual-user nDCG@all values, but the axis labels and legend are not described in the text; please add explicit labels for users and metric values, and consider plotting error bars or a confidence band around the means.","section":"Section 4.2.2, Figure 12"},{"comment":"The caption for Figure 14 does not state which nDCG@all variant is plotted, nor whether the curves are averaged over multiple random initializations; since the text mentions random initializers as a source of variation, report the number of runs and display error bars.","section":"Section 4.4, Figure 14"},{"comment":"The description of data preprocessing (duplicating low ratings and the rating normalization n(r) = r - R_mean) is introduced in the dataset analysis but not referenced in the model section; please make explicit where these steps enter the training pipeline and whether they are applied to both DFNSM and NSM.","section":"Section 4.1"},{"comment":"The word 'phycological' in the conclusion is a typo for 'psychological'; throughout the manuscript, light copyediting is needed for grammar and article usage.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is clearly a student project with a strong conceptual idea, but the empirical verification is not yet at the level expected for a journal publication. The central claim about the forgetful window is genuinely interesting and testable; a focused revision with ablations and a more rigorous evaluation would make it publishable. Please also consider whether the journal's scope expects more extensive related-work positioning on sequential recommendation and novelty-aware recommenders."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper's headline claim is not yet supported, because the comparison bundles two changes at once. But the ANI formulation is a clean, clearly specified variant of earlier DCN novelty, and the paper is honest about its own limits. It reads like a solid first project, not a completed research claim.\n\nThe actual new thing is the fixed-k window: ANI_t,i = 1/(count of tag i in actions t-k..t-1 + 1). That is a simple, well-defined extension of Zhang et al.'s DCN, and the paper correctly notes that DCN is the k=full-history special case. Also new is using continuous novelty values in a neural recommender rather than integer rankings. The writing is unusually clear about what is being computed.\n\nWhere it falls short is the evaluation. Table 3 compares DFNSM (MLP/CNN trained with gradient descent) against NSM (probabilistic model with Gibbs sampling) and random. Those differ in architecture and in the novelty window simultaneously, so the 0.3887 vs 0.2696 gap cannot be attributed to forgetfulness. No ablation isolates the window. The k-sensitivity test in Section 4.4 only uses three users, and the paper itself says a single k across users \"was never able to achieve a high metric score.\" That admission sits awkwardly with the average results in Table 3. There are also no error bars or significance tests, the NSM baseline implementation is not specified, and the evaluation uses only the first 20 users. No code or hyperparameters are provided. These are addressable, not fatal, but as is the central claim is unsupported.\n\nThe reader's take is fair, and the stress-test concern is correct: the missing ablation is a load-bearing gap. The paper's own Section 4.4 text should be flagged in any review.\n\nWho is this for? Someone working on novelty in sequential recommendation might find the ANI definition worth a short discussion, and a student venue could develop it. A serious referee could turn this into a decent short paper by requiring a full-history control, ablations over architecture, and proper evaluation.\n\nI'd send it to review rather than desk reject, mainly because the hypothesis is testable and the write-up makes the math precise. But it needs major revision.","headline":"A clean, clearly-specified novelty-window idea, but the main comparison confounds the window with the deep architecture, so the central claim needs an ablation before it holds.","tokens_in":11189,"tokens_out":2133,"would_cite":false,"duration_ms":24027,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Novelty from the last k movies improves next-movie prediction.","keywords":["Novelty-seeking","Movie recommendation","Action Novelty Index","User Novelty-Seeking Index","Forgetfulness","Deep learning","nDCG","Sequential recommendation"],"falsifier":"Take users whose histories contain a genre they watched heavily, then abandoned for more than k actions, then revisited. If a small-k DFNSM ranks that genre below a full-history NSM on those return actions, the forgetfulness assumption is contradicted; the same test run on a random sample of hundreds of users, not just the first twenty, would show whether the nDCG gap persists.","tokens_in":10225,"feed_emoji":"🎬","tokens_out":6008,"duration_ms":142875,"temperature":0.7,"pith_summary":"The paper tries to show that a movie recommender can predict a user's next movie better by measuring how novel each candidate movie is relative only to the user's k most recent watched movies, rather than to the entire viewing history. It argues that users are forgetful: long-ago actions should not make a present movie feel less novel. On the first twenty users of a large public movie-rating dataset, the proposed Deep Forgetful Novelty-Seeking Model reaches mean nDCG@all 0.3887, against 0.2696 for the full-history novelty-seeking model NSM and 0.1781 for random ranking. The result matters because it suggests that a cheap, interpretable novelty signal, a sliding window of past choices, can carry much of the predictive weight in next-action recommendation.","feed_headline":"Forgetful novelty model beats full-history baseline on next-movie picks","feed_subtitle":"Using only a user's last k actions to score novelty lifts next-movie ranking from 0.270 to 0.389.","key_machinery":"The Action Novelty Index (ANI) matrix is the load-bearing object: for each user and time step it stores, for every tag, the reciprocal of how many times that tag appeared in the previous k actions plus one. This single formula implements both forgetfulness (only k actions count) and continuous novelty (values are frequencies, not ranks). The User Novelty-Seeking Index (UNI) is a secondary, interpretive quantity derived from entropy of the normalized ANI row and is not used in prediction. The rest of the machinery is a deep network: a CNN over the movie title plus summed tag embeddings forms each movie's latent vector, an MLP over demographics and ANI forms the user's time-varying latent vector, and the predicted rating is their dot product, trained by mean-squared error and gradient descent.","core_discovery":"DFNSM's central claim is that next-movie choice is best predicted by combining demographic and movie-content representations with a novelty score computed from a limited memory window: each tag's novelty at time t is $\\mathrm{ANI}_{t,i} = \\frac{1}{\\#x_{t-k,t-1}^{i} + 1}$, where the count covers only actions $t-k$ through $t-1$. The paper claims this 'forgetful' novelty measure is more accurate than the full-history Novelty-Seeking Model because it treats novelty as a continuous scale rather than a ranking and because it drops stale information; NSM's dynamic choice novelty matrix is presented as the special case $k$ equal to the whole sequence length. The model projects the user (demographics plus the current ANI row) and each movie (title plus embedded tags) into a shared latent space, predicts ratings by a dot product, and ranks movies accordingly. The paper's evidence is mean nDCG@all on the first twenty users: 0.3887 for DFNSM versus 0.2696 for NSM and 0.1781 for random ranking.","pith_inferences":["A natural extension the paper does not develop is to make k a learned, per-user parameter rather than a tuned constant; if forgetfulness varies as the UNI plots suggest, a network could output k from the action history.","The ANI formula could transfer to other sequential choice domains with sparse tag-like attributes, such as news reading or music listening, where a short recency window is computationally cheap.","A direct test of the paper's scope is to rerun the same comparison on a random sample of hundreds of users rather than the first twenty, which would show whether the nDCG gap persists on longer, heavier histories.","UNI's entropy-based smoothness could serve as a stable user-level covariate in churn or exploration models, since the paper shows the index is user-distinguishable over time."],"forward_implications":["If DFNSM's result holds, next-action recommenders should replace full-history novelty matrices with a sliding k-action window, since the full-history model is a special case with k set to sequence length.","The reported nDCG advantage implies that novelty, treated as a scale rather than a ranking, carries usable signal for ranking the single next action even when demographics and content features are included.","The k value that maximizes accuracy can be read as a per-user memory or fickleness parameter, so model comparison should report k separately per user rather than one global value.","The finding that per-user k produces accuracy spikes while a shared k does not suggests users differ in how quickly past choices stop feeling fresh; recommendation quality may improve by personalizing the window size."],"supporting_citations":[{"why":"Defines the Novelty-Seeking Model (NSM) and dynamic choice novelty matrix that DFNSM extends and compares against.","marker":"[2]"},{"why":"Provides the public movie-rating dataset with ratings, timestamps, user demographics, and movie genres used in all experiments.","marker":"[16]"},{"why":"Supplies the neural collaborative filtering approach that motivates adding deep networks to user-item prediction.","marker":"[10]"},{"why":"Supplies neural-network matrix factorization, another deep recommendation component the model builds on.","marker":"[11]"},{"why":"Provides the sequential hierarchical attention recommender, a sequence-based contrast for modeling user interests.","marker":"[12]"},{"why":"Establishes the novelty-seeking construct in consumer behavior that the paper quantifies.","marker":"[1]"},{"why":"Supplies the convolutional network architecture used to extract text information from movie titles.","marker":"[6]"}],"fun_headline_variants":["Forgetful novelty lifts movie ranking by 44%","Short-term memory boosts novelty-based movie picks","DFNSM: limited window beats full-history scoring","Forgetting old data improves movie recommendation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a user's next movie choice is set by novelty computed only from the previous k actions, with every older action ignored; if long-ago choices still shape current taste, or if k only works because the evaluation happens on the same few users, the reported 0.3887 advantage is not established.","fun_headline_variants_meta":{"raw":{"variants":["Forgetful novelty lifts movie ranking by 44%","Short-term memory boosts novelty-based movie picks","DFNSM: limited window beats full-history scoring","Forgetting old data improves movie recommendation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1556,"prompt_tokens":946,"completion_tokens":610,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":551}},"tokens_in":562,"tokens_out":610,"duration_ms":6040,"temperature":1.0,"reasoning_tokens":551,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:39:10.007245+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take users whose histories contain a genre they watched heavily, then abandoned for more than k actions, then revisited. If a small-k DFNSM ranks that genre below a full-history NSM on those return actions, the forgetfulness assumption is contradicted; the same test run on a random sample of hundreds of users, not just the first twenty, would show whether the nDCG gap persists.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the sequential hierarchical attention recommender, a sequence-based contrast for modeling user interests."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the novelty-seeking construct in consumer behavior that the paper quantifies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the convolutional network architecture used to extract text information from movie titles."}],"review_version":1}