{"id":"cd20d16a-f360-4d7f-a6c6-c656247843cf","arxiv_id":"2608.11555","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A screen-and-confirm protocol certifies that continuous-time decay, not additional conditioning signals, almost fully explains customer-return timing on four datasets.","lead":"This paper tests whether adding customer and calendar features actually improves predictions of when a customer returns, beyond a simple time-decay model. It introduces a positive-control protocol that certifies when a null result means 'no signal' rather than 'weak method', and finds that decay nearly suffices while extra conditioning is redundant or harmful.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing per-pathway positive controls for category/RFM/LTV leave the central 'no conditioning helps under decay' claim uncertified for those signals.","rationale":"The paper is honest about the missing per-pathway controls, and this concern does not invalidate its method contribution or the decay result, which is independently supported by Table 1 and the taxi positive control. But it does mean C3 is broader than what C1 certifies: the screen-and-confirm protocol validates calendar/geography, while category, RFM, and proxy-LTV nulls rely on the same pipeline plus the model-free ceiling, which does not bound NLL-style distributional gains. The reader's seed-std concern is real but secondary; even a proper bootstrap would likely leave the tiny measured deltas practically negligible, whereas a failed category/RFM positive control would undermine the interpretability of those deltas entirely. The proposed synthetic control would settle this directly: monotone recovery would make the existing nulls credible, while failure would require narrowing the certified claim. Since the current CONDITIONAL verdict already asks for additional validation, the appropriate adjustment is to keep it unchanged, with the per-pathway positive control as a more specific condition than the reader's wording suggests.","tokens_in":12063,"tokens_out":5785,"duration_ms":64215,"concrete_test":"Run the §5.5 synthetic positive-control procedure for the category and RFM encodings exactly as done for season (Fig. 1a), using the THP-D backbone and the §4 optimizer: plant gap_{j+1} ~ Exp(r0 exp(beta * category_j)) and gap_{j+1} ~ Exp(r0 exp(beta * RFM_j)) for beta = 0, 0.5, 1, 2 across the same three seeds. If THP-D fails to recover beta monotonically (e.g., delta NLL at beta=2 is not clearly negative), then the Table 4 nulls for CatOnly-D and RFM-D cannot be certified as 'no signal,' and C3 must be narrowed to calendar/geography or extended with per-pathway controls.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that no conditioning signal improves return-timing likelihood once decay is present—is certified for calendar/geography by the §5.5 screen-and-confirm protocol, but the paper explicitly does not run that screen for category, RFM, or proxy-LTV: §5.5 states these nulls 'were not separately screened—per-pathway positive controls are future work.' The no-decay CatOnly baseline (Table 4: −0.32/−0.02/−1.70) shows the model can recover category when decay is absent, but it does not establish that THP-D can recover a category-to-gap coupling when decay is present. The Fig. 1a positive control does this only for an i.i.d. season field, not for history-derived category/RFM encodings. C2's model-free ceiling is R² on the mean gap and, as §5.4 concedes, does not bound distributional NLL gains. Thus the headline negative for category/RFM/LTV rests on an unvalidated null, exactly the failure mode the screen was designed to rule out. The seed-std null rule (the reader's concern) is a second, softer issue: it affects the word 'statistically' but not the measured ≤0.06 NLL magnitude, whereas a failed category/RFM positive control would affect whether the real-data nulls are interpretable at all.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-part methodology for evaluating conditioning signals in temporal point process (TPP) models of customer-return timing: (C1) a screen-and-confirm protocol using synthetic positive controls, and (C2) a model-free ceiling on the predictability of inter-event gaps. Applying these to three public benchmarks and one proprietary marketplace dataset, the authors report that continuous-time decay is the dominant timing signal and that additional conditioning (calendar, geography, category, RFM, proxy-LTV) is redundant or harmful on top of a decay backbone. The manuscript is unusually transparent, explicitly retracting earlier leaked readouts and flagging limitations, and it frames the contribution as a certification discipline rather than an architectural novelty.","tokens_in":12327,"tokens_out":4406,"duration_ms":49393,"significance":"If its claims hold, the paper offers a reusable template for interpreting null results in TPP conditioning, which is genuinely valuable to both practitioners and the evaluation literature. The authors deserve explicit credit for shipping a positive-control screen, for running a real-clock positive control (NYC taxi), for distinguishing point-prediction from distributional likelihood gains, and for honestly retracting artifacts that inflated earlier results. The model-free ceiling result is a useful sanity check on the field's tendency to add covariates. The main risks are that the claim of certification is broader than the experiments actually certify, and that the statistical language is not backed by a proper uncertainty analysis.","major_comments":[{"comment":"The screen-and-confirm protocol is validated only for calendar/geography signals. As the manuscript itself states, 'the LTV/category/RFM nulls are measured under the same pipeline and bounded by the same ceiling but were not separately screened—per-pathway positive controls are future work.' The no-decay CatOnly baseline (Table 4) shows the model can recover a category signal when decay is absent, but it does not establish that the decay-conditioned model can recover a category-to-gap coupling when decay is present. Since the abstract and C3 claim to certify that 'the conditioning practitioners keep adding—LTV, category, RFM, calendar, geography—is redundant or harmful,' this is a load-bearing gap between evidence and claim. The manuscript should either run per-pathway positive controls for category, RFM, and proxy-LTV, or explicitly narrow the certified claim to calendar/geography and describe the other nulls as measured but not screen-certified.","section":"§5.5"},{"comment":"The null rule 'we read a conditioning delta as null when |Δ| < 2× its seed std' ( §4) treats seed-to-seed optimization variance as the only source of uncertainty. With only three seeds, this ignores sampling variability across customers and does not correct for multiple comparisons across the several conditioning signals tested. Because the central claim uses the words 'statistically null' (abstract and §5.3), the current procedure does not support that statistical phrasing. The measured deltas of ≲0.06 NLL on public benchmarks are certainly small and may justify 'negligible' or 'practically null,' but the authors should either provide a proper uncertainty estimate (e.g., bootstrap over customers or a permutation test on the NLL delta) or replace the 'statistically null' wording with a magnitude-based statement.","section":"§4 and §5.3"}],"minor_comments":[{"comment":"The y-axis label 'ΔNLL (↓)' could be clearer, since negative values indicate improvement and the caption already explains the sign convention. Consider adding a zero line and an explicit arrow direction to help readers avoid confusion.","section":"Fig. 1"},{"comment":"The values in Table 5 are percentages but the column entries like '1.4%' are sometimes followed by the notation '𝑟2' without specifying that these are R²×100. Adding a note in the caption that all entries are percentages of variance explained would improve readability.","section":"Table 5 and §5.4"},{"comment":"The list of retracted bugs is valuable and should be preserved, but the paragraph is dense. A small table with columns such as 'artifact', 'cause', 'impact', and 'correction' would make the honest-evaluation account easier for readers to verify.","section":"§7"}],"recommendation":"major_revision","confidential_remarks":"The paper is well written and unusually honest about its own pipeline failures. The main issue is that the headline claim of certified redundancy is broader than the evidence, because the positive-control screen is only run for calendar/geography. The statistical null rule also needs strengthening or rewording. These are fixable within the manuscript's scope, so major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one. It's a genuinely useful measurement paper. The screen-and-confirm protocol is a good idea: plant a synthetic coupling, verify the model recovers it, then trust the real-data null. The authors validate it for categorical and continuous encodings, and the taxi hour-of-day real positive control is a nice touch. The model-free ceiling—single-digit percent of gap variance explained, previous gap always the best—is a clean, simple result that explains why conditioning fails. And the paper is unusually honest: they document leaked read-outs and retract the inflated MAE numbers. That alone earns respect.\n\nThe central finding—decay is nearly sufficient, conditioning is redundant—is plausible and mostly supported. On public benchmarks, every decay-regime delta is at most 0.06 NLL. The Thumbtack category delta is +0.47, which they read as overfitting.\n\nBut there are two soft spots. First, the certification claim is broader than what was actually screened. Section 5.5 says calendar/geography passed the planted-coupling screen; category/RFM/LTV nulls were \"not separately screened—per-pathway positive controls are future work.\" So the headline that \"conditioning is redundant\" is certified only for the exogenous signals. For category/RFM, the null rests on direct measurement plus the model-free ceiling, not on a demonstrated ability to recover those couplings. Given that CatOnly-D on RetailRocket is −0.06 and the positive control only covers i.i.d. season, the category null is not fully validated. Second, the statistical rule for \"null\" is |Δ| < 2× seed std over three seeds. That treats seed variance as the full uncertainty and ignores sampling variability of the NLL estimate. For the public deltas near zero this may not change the practical conclusion, but it doesn't support the word \"statistically null.\"\n\nI don't think either flaw sinks the paper. The magnitude claims are small and the transparency is exemplary. For a practitioner, the message is: don't expect LTV/category/RFM to sharpen return-timing once you have a decay backbone. That's worth testing on your own data. The screen-and-confirm protocol is worth adopting.\n\nBring it to reading group, cite it. Send it to review with a request for clearer statistical language and either per-pathway positive controls or a scope statement that limits the certified null to calendar/geography.","headline":"A transparent, useful measurement study with a genuinely good validation protocol, but the 'certified null' is over-broad for category/RFM signals and the seed-std rule is weaker than 'statistically null.'","tokens_in":12833,"tokens_out":1548,"would_cite":true,"duration_ms":15407,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that once a decay-based inter-event clock is in a temporal point process, conditioning signals such as category, recency/frequency, calendar, and geography add no measurable timing-likelihood gain, and it offers a…","keywords":["temporal point processes","customer-return timing","conditioning signals","screen-and-confirm protocol","positive controls","model-free predictability ceiling","continuous-time decay","near-memoryless returns"],"falsifier":"Take the same decay backbone and datasets, then use a bootstrap over customers or many more seeds to estimate the full sampling distribution of each conditioning delta; alternatively, plant a synthetic season-to-gap coupling of realistic strength into held-out data and check whether the screen fails to recover it at a comparable magnitude. A positive finding would be a strongly seasonal dataset where a calendar-conditioned decay model beats the decay-only backbone by more than twice the estimated uncertainty while the positive control still recovers.","tokens_in":11864,"feed_emoji":"🕰️","tokens_out":5648,"duration_ms":59317,"temperature":0.7,"pith_summary":"This paper asks whether the extra signals practitioners add to customer-return timing models—category, lifetime value, recency/frequency, calendar, geography—actually improve when a customer returns, and it answers no on four datasets once a continuous-time decay backbone is present. To make a null result meaningful it builds a screen-and-confirm protocol: plant a synthetic coupling of known strength, confirm the model recovers it monotonically, and only then read a flat real-data result as 'no signal' rather than 'weak method.' It also reports a model-free ceiling: no available covariate explains more than a single-digit percentage of inter-return gap variance, so returns are near-memoryless beyond the time since the last event. The central claim is that the inter-event clock—continuous-time decay—is nearly sufficient for return timing, and that conditioning on top of it is statistically null or mildly harmful. The contribution is not discovering that decay helps; it is turning 'conditioning doesn't help' into a checkable, certified statement.","feed_headline":"Decay clock beats extra customer signals in return-timing models","feed_subtitle":"A screen-and-confirm test shows added covariates are null once the inter-event clock is in the model.","key_machinery":"The central object is the screen-and-confirm test: a positive control that plants a feature-to-timing coupling of tunable strength in synthetic data, shows the conditioned model recovers it monotonically, then runs the identical pipeline on the real feature; a flat real result after a passing control certifies 'no signal in that encoding.' The second machinery piece is a model-free predictability ceiling: the fraction of log-gap variance explained by each covariate, which bounds any mean-shift signal available to any model. The third is the decay head, which lets the hidden state decay between events and relaxes the frozen-intensity assumption of an attention-based backbone. Together these separate the mechanism claim (the clock carries the timing signal) from the measurement claim (nothing else is there to find).","core_discovery":"On the paper's own terms, the discovery is a certified negative: in temporal point-process models of customer return timing, the conditioning components that are standard in practice and in the literature are redundant once the model contains continuous-time decay. On three public e-commerce/web benchmarks, every conditioning delta on top of a decay backbone moves temporal negative log-likelihood by at most about 0.06, within seed noise; on a proprietary marketplace dataset the same additions are null to mildly harmful, with category worsening likelihood by about 0.47 relative and large seed variance read as overfitting. The search for point predictions fails harder: a model-free regression ceiling shows that less than about 5 percent of inter-event gap variance is explainable by any measured covariate, with the previous gap itself the best predictor. Decay's large likelihood gains are distributional—correctly placing probability mass over when the next event occurs—not point-predictive. The paper is explicit that it does not claim to have discovered that decay helps; the certified claim is that the usual enrichments do not help on top of it, and the screen-and-confirm protocol is what makes that null interpretable.","pith_inferences":["The screen-and-confirm protocol could be adopted as a standard audit step for any proposed point-process feature: certify recoverability on synthetic couplings first, then interpret real-data gains or nulls.","The model-free ceiling suggests that datasets with strong external seasonality, such as holiday retail or weather-driven demand, may sit under a higher ceiling; re-running the ceiling and screen per dataset would settle when conditioning genuinely pays.","If the inferred mechanism that decay already encodes RFM and category is right, representation-probing or mutual-information tests should show near-zero additional information in those hand-crafted features given the decayed hidden state.","The passive-covariate null does not close the question of interventions such as marketing nudges; causal counterfactual data could reveal timing effects even when offline covariates are uninformative."],"forward_implications":["Adding category, LTV, RFM, or calendar/geography conditioning to a decay-equipped customer-return model will not improve timing likelihood; on public benchmarks any movement is within about 0.06 NLL of zero.","Point-timing predictions cannot be improved by measured covariates: predicting the global mean gap matches the neural TPPs once leakage is removed.","Decay's whole-nat gains in temporal NLL are distributional calibration effects, attainable even when point predictability is zero.","A flat conditioning result should not be trusted unless a positive control shows the model can recover a planted signal of comparable encoding.","On data with genuine exogenous timing drivers, the same screen does fire, so the customer-return null is data-specific rather than a method artifact."],"supporting_citations":[{"why":"Defines the neural Hawkes decay mechanism that the paper isolates as the dominant timing lever.","marker":"[2]"},{"why":"Provides the Transformer Hawkes backbone whose frozen-intensity ablation is the controlled contrast for the decay head.","marker":"[3]"},{"why":"Provides the benchmark and evaluation practice and the public data splits used for the public datasets.","marker":"[8]"},{"why":"Supplies the state-space point-process method and the prior backbone ranking that the paper reproduces as validation.","marker":"[9]"},{"why":"Represents the covariate-conditioning approach whose real-data timing null this paper certifies.","marker":"[13]"},{"why":"Supplies the memoryless exponential-gap assumption that the model-free ceiling independently confirms on four datasets.","marker":"[15]"},{"why":"Supports the claim that neural TPP architectures differ little in time-NLL, so the multi-nat contrast comes from the frozen-intensity ablation.","marker":"[21]"},{"why":"Supplies the sanity-check discipline that screen-and-confirm ports to conditioning nulls.","marker":"[23]"}],"fun_headline_variants":["Decay clock: extra signals null in return-timing models","Return timing: decay nearly enough, added signals null","Certified: decay nearly enough, added signals redundant","Screen-and-confirm: added signals null after decay"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The main premise is that a model improvement is counted as real only if it is larger than twice the variation across three random training runs; if that seed variation is smaller than the true uncertainty from customer sampling, model choice, or encoding choice, the conclusion that no signal helps could be wrong.","fun_headline_variants_meta":{"raw":{"variants":["Decay clock: extra signals null in return-timing models","Return timing: decay nearly enough, added signals null","Certified: decay nearly enough, added signals redundant","Screen-and-confirm: added signals null after decay"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000799,"raw_usage":{"total_tokens":3614,"prompt_tokens":1144,"completion_tokens":2470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":760,"completion_tokens_details":{"reasoning_tokens":2405}},"tokens_in":760,"tokens_out":2470,"duration_ms":18211,"temperature":1.0,"reasoning_tokens":2405,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:35:16.358749+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same decay backbone and datasets, then use a bootstrap over customers or many more seeds to estimate the full sampling distribution of each conditioning delta; alternatively, plant a synthetic season-to-gap coupling of realistic strength into held-out data and check whether the screen fails to recover it at a comparable magnitude. A positive finding would be a strongly seasonal dataset where a calendar-conditioned decay model beats the decay-only backbone by more than twice the estimated uncertainty while the positive control still recovers.","supporting_citations":[{"cited_title":"Mei and J","cited_arxiv_id":null,"evidence_quote":"Defines the neural Hawkes decay mechanism that the paper isolates as the dominant timing lever."},{"cited_title":"Zuo et al","cited_arxiv_id":null,"evidence_quote":"Provides the Transformer Hawkes backbone whose frozen-intensity ablation is the controlled contrast for the decay head."},{"cited_title":"Xue et al","cited_arxiv_id":null,"evidence_quote":"Provides the benchmark and evaluation practice and the public data splits used for the public datasets."},{"cited_title":"Chang, A","cited_arxiv_id":null,"evidence_quote":"Supplies the state-space point-process method and the prior backbone ranking that the paper reproduces as validation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Represents the covariate-conditioning approach whose real-data timing null this paper certifies."},{"cited_title":"Counting Your Customers","cited_arxiv_id":null,"evidence_quote":"Supplies the memoryless exponential-gap assumption that the model-free ceiling independently confirms on four datasets."},{"cited_title":"Bosser and S","cited_arxiv_id":null,"evidence_quote":"Supports the claim that neural TPP architectures differ little in time-NLL, so the multi-nat contrast comes from the frozen-intensity ablation."},{"cited_title":"Adebayo, J","cited_arxiv_id":null,"evidence_quote":"Supplies the sanity-check discipline that screen-and-confirm ports to conditioning nulls."}],"review_version":1}