{"id":"fb9c7d91-4638-4cc9-a809-a2e2356ab6b5","arxiv_id":"2411.15056","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Grouping long payment histories by merchant and learning within- and across-merchant patterns improves default prediction accuracy on a large Tencent dataset.","lead":"This paper proposes LBSF, which reorganizes a user's long payment history by merchant before learning risk patterns with transformers, and reports better default prediction on Tencent payment data. It matters for online lenders that lack traditional credit records and must infer creditworthiness from payment behavior alone.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never states the temporal cutoff between the payment-behavior window and the default label, so the central LBSF-vs-Mamba gap may reflect label leakage rather than merchant folding.","rationale":"The reader's verdict is CONDITIONAL, and the reader identified label timing as the most serious issue. My independent reading of the full text confirms that this is the single most load-bearing concern. The paper's own framing in Section III-A says the model predicts default to decide whether to grant access, and Section V-A defines positives as users who default after access, but the manuscript never states the temporal relation between the behavior sequence and either the access date or the default outcome. Without that statement, the 180-day AUC advantage of LBSF over Mamba is not interpretable. The dataset split in Table I (training, validation, test by calendar dates) is consistent with many standard setups where users are sampled by application date, but it does not rule out behavior windows that overlap the outcome window. This is not an internal inconsistency in the method, but it is a missing boundary condition for the empirical claim. A secondary concern, the merchant-selection rule for M=74, is also genuine, but it is less load-bearing because a per-user frequency-based selection would not necessarily leak label information; the timing issue alone can invalidate the comparison. I therefore agree with the reader's weakest_assumption and see no reason to change the CONDITIONAL verdict. The paper has coherent ablations and a plausible mechanism, but the absence of a stated temporal cutoff, combined with no code and no error bars, leaves the central claim unverified. The concrete test above would settle whether the reported gap survives a clean temporal split.","tokens_in":14218,"tokens_out":3259,"duration_ms":35429,"concrete_test":"Request the authors to (a) report the per-user service-access date and default-date distributions, and (b) rerun the 180-day comparison with a strict temporal split: for every user, use only payment behaviors with timestamp strictly before the service-access date (ideally with a 7-day buffer), keep the merchant set fixed from the training period, and retrain LBSF and Mamba under identical conditions. If the AUC gap and Recall@10% gap between LBSF and Mamba remain at least 0.005 AUC on this clean split, the central claim survives; if the gap collapses, the reported result is explained by label timing rather than by merchant folding.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that LBSF outperforms Mamba on 180-day data (AUC 0.7678 vs 0.7554). This comparison is only meaningful if each user's behavior sequence contains no information about the default outcome. Section III-A frames the task as predicting default before granting access, and Section V-A labels users 'who default after being granted access' as positive, but nowhere does the paper state that payment behaviors are truncated at the service-access date, or at any fixed offset before the default label. The data splits in Table I are calendar-window splits of the collection period, not per-user observation/outcome windows. If a positive user's 180-day behavior window overlaps the period after access and before or after default, the model can exploit direct signals of the default event (e.g., missed payments, balance drops, atypical merchant patterns) rather than predictive long-term habits. Because LBSF's merchant folding is designed to highlight exactly such changing patterns, the reported 1.24pp AUC gap could be an artifact of label leakage. The missing temporal boundary is thus load-bearing: it is the condition that converts the empirical comparison into evidence for the claimed mechanism. A secondary concern is the unstated rule for selecting M=74 merchants per user in Section V-C; if that selection uses label information, it would also inflate results, but the timing ambiguity alone is sufficient to undermine the headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LBSF (Long-term Payment Behavior Sequence Folding) for financial default prediction from long payment histories. LBSF reorganizes each user's payment sequence by merchant, encodes payment descriptions, timestamps, and amounts jointly, melds behavior embeddings within each merchant with a Transformer, then applies a second Transformer plus a ViT-style [CLS] token to perform relational learning across merchants. The method is evaluated on a proprietary Tencent Mobile Payment dataset of 458,744 users with 45-day, 90-day, and 180-day behavior windows, compared against 13 baselines (basic, sequential, and pre-training models). The main reported result is that LBSF outperforms all baselines on the 90-day and 180-day subsets, with the largest gain on 180 days (AUC 0.7678 vs. 0.7554 for Mamba, Recall@10% 0.3467 vs. 0.3208). An ablation study and two qualitative case studies are also presented. The paper concludes that folding long behavior sequences by merchant effectively models long-term financial patterns and yields more accurate user financial profiles.","tokens_in":14479,"tokens_out":2430,"duration_ms":26099,"significance":"If the central claim holds, LBSF offers a practical, architecture-agnostic way to model very long payment sequences: merchant-level folding provides an internal grouping criterion that enables parallel processing, and multi-field encoding exploits rich payment details. The method is evaluated end-to-end on a held-out test split, so there is no derivation-versus-fit circularity, and the attention-based case studies give some interpretability. However, the empirical significance is currently constrained by three load-bearing gaps: the temporal relationship between the behavior window and the default label is never stated, the rule for selecting M=74 merchants per user is unspecified, and the experimental evidence rests on single runs of a proprietary dataset without code or error bars. These issues must be resolved before the reported LBSF-vs-Mamba gap can be attributed to merchant folding rather than to leakage or selection artifacts.","major_comments":[{"comment":"The label timing assumption is never stated. Section III-A frames the task as predicting default before granting access to financial services, and Section V-A labels users who 'default after being granted access' as positive, but the paper does not say that the payment-behavior window is truncated at the service-access date or at any fixed offset before the default event. Table I reports only calendar splits of users into training/validation/test sets, not per-user observation-versus-outcome windows. If a positive user's 180-day behavior window overlaps the period after access or around the default event, the model can exploit direct signals of the default (missed payments, balance drops, unusual merchant patterns), and the claimed 1.24pp AUC gap in Table II (0.7678 vs. 0.7554 for Mamba) could be an artifact of label leakage. The authors must specify the exact cutoff between the behavior window and the default determination, and ideally report results under truncated windows as a robustness check.","section":"Section III-A and Section V-A"},{"comment":"The selection rule for M=74 merchants per user is not described. The paper states only that 'we set M to 74,' with no explanation of how the merchants {m'_n1,...,m'_nM} are chosen, whether the merchant list is global or per-user, or whether selection happens before or after label assignment. If merchant selection uses label information or future behavior, the reported improvements would be inflated. The authors need to specify the selection procedure and demonstrate that it is label-independent and time-independent.","section":"Section V-C and Algorithm 1"},{"comment":"The evidence for consistent gains across windows is incomplete. XGBoost and MLP are reported only on the 90-day subset (with '——' on 45-day and 180-day), yet the text claims a consistent trend across all models with longer sequences. Additionally, no error bars, confidence intervals, or multiple-seed variance are reported; the 45-day case shows LBSF behind Mamba by 0.46%, and the headline 180-day AUC gap of 1.24pp may be within run-to-run noise. Reporting results over multiple seeds and completing the missing baseline cells is necessary to support the central empirical claim.","section":"Table II and Section VI-A"},{"comment":"The dataset description and the table are inconsistent. The text says the dataset is sampled 'to maintain a positive rate among all included users of approximately 10.0%,' but Table I reports positive rates of 7.2% for training, 12.2% for validation, and 12.2% for testing. The authors should reconcile this discrepancy and state whether the sampling was done within each split or globally.","section":"Section V-A and Table I"}],"minor_comments":[{"comment":"Equation (2) has a formatting error: it is missing a closing parenthesis (it reads '(cos(2πt/T, sin(2πt/T )') and the comma placement after 'cos(2πt/T' is wrong; it should be the two-component vector (cos(2πt/T), sin(2πt/T)).","section":"Eq. (2)"},{"comment":"The notation is confusing: the sentence 'the embeddings of d_ni, t_ni are denoted by d_ni, t_ni for simplicity' uses the same symbols for raw fields and embeddings; using bold or different symbols would improve clarity.","section":"Section IV-C"},{"comment":"The case study example in Section IV-B says the user's 'new choice is cheap train tickets,' but Figure 3 and the surrounding text do not clearly mark the temporal ordering across merchants; adding explicit timestamps or arrows to the figure would help.","section":"Section IV-B and Figure 3"},{"comment":"The case studies are qualitative and based on two manually selected defaulters; the paper should state that these examples are illustrative and not a systematic evaluation of interpretability, and ideally report how often the described patterns occur among detected defaulters.","section":"Section VI-C"},{"comment":"The dataset and code are not publicly available, and the paper does not include an ethics or reproducibility statement beyond the data footnote; a clear statement about anonymization and any plan for sharing features or code would strengthen the work.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the architecture is clearly presented, but the empirical claim hinges on temporal-integrity details that are currently absent. I do not view this as a case of intentional leakage, but the manuscript as written does not rule it out, and the proprietary data plus missing baseline cells make it hard to assess robustness. A revision that adds the temporal cutoff definition, merchant-selection rule, multi-seed results, and complete baseline table would be suitable for reconsideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The merchant-folding idea is the real novelty here, and it is not a restatement of prior hierarchy methods. Grouping payment sequences by an intrinsic field before hierarchical transformer encoding is simple and plausible, and the ablation supports the mechanism: removing merchant folding costs about 5 AUC points on the 90-day subset, the largest drop among the ablated components. The paper is also honest enough to report the 45-day exception where Mamba edges out LBSF. That is a good sign. The related work is fair, and the writing is clear.\n\nThe soft spots are real but not all equal. The load-bearing one is timing: Section III-A says the goal is to predict default before granting access, and Section V-A labels users who default after access, but the paper never states that the 180-day behavior window is truncated before the access date or before the default date. The data splits are calendar-window splits of the collection period, not per-user observation/outcome windows. If a positive user's behavior window overlaps the post-access period, the model can pick up direct signals of strain—missed payments, balance drops, unusual merchant patterns—rather than predictive long-term habits. Because merchant folding is explicitly designed to highlight exactly those changes, the reported 1.24pp AUC gap over Mamba could be an artifact. This is a fixable omission, but it is currently the difference between a credible result and a leaked one. The secondary concern, the unstated rule for selecting M=74 merchants per user in Section V-C, could also inflate results if selection uses label information.\n\nBeyond that, the evaluation is thinner than I would like: single runs without error bars, no code, no data, and XGBoost and MLP are simply missing on the 45-day and 180-day subsets with no explanation. None of these is fatal by itself, but together they mean the empirical comparison rests on a single proprietary run.\n\nWho is this for? Applied ML researchers in fintech, especially people working with long user behavior sequences. If the authors confirm the temporal alignment and release code or detailed preprocessing, this becomes a solid applied contribution. As it stands, it deserves a serious referee—conditional on the authors addressing the leakage risk. I would want the timing question answered before trusting the headline numbers.","headline":"Merchant-level folding is a genuinely new idea and the paper is clearly written, but the central performance claim is currently undercut by an unstated temporal boundary between behavior sequences and default labels.","tokens_in":15016,"tokens_out":1558,"would_cite":false,"duration_ms":17866,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LBSF claims that folding long payment behavior sequences by merchant, with multi-field encoding and hierarchical Transformer melding, yields more accurate default-risk predictions than standard long-sequence models.","keywords":["financial risk assessment","long-term behavior sequence","sequence folding","merchant-level modeling","multi-field encoding","default prediction","payment behaviors","Transformer"],"falsifier":"Take the 180-day subset, fix a cutoff date well before each user's default event, discard all payments after it, and re-run LBSF versus Mamba; also rerun with merchants chosen by a label-blind rule such as top-frequency. If AUC and Recall@10% margins vanish, the longer-window gains are leakage or merchant-selection artifacts rather than genuine folding benefits.","tokens_in":14016,"feed_emoji":"💳","tokens_out":5111,"duration_ms":45259,"temperature":0.7,"pith_summary":"The paper argues that default risk in online inclusive finance can be predicted more accurately when models use long payment histories rather than short windows, and that the best way to use those long histories is to reorganize them by merchant before feeding them to a Transformer. It proposes LBSF, a method that folds each user's payment sequence into merchant-level sub-sequences, encodes payment description, time, and amount jointly, then melds within-merchant and across-merchant information to build a user representation. On a large real-world dataset, LBSF outperforms 13 baselines on 90-day and 180-day windows, with the biggest gain coming from the merchant-folding component. If this is right, platforms without traditional credit records can extract more signal from everyday payment behavior, and other sequential-modeling tasks on long behavior logs have a cheap, knowledge-free reorganization to try.","feed_headline":"Folding payment histories by merchant improves default-risk forecasts","feed_subtitle":"On 180 days of payment data, LBSF hits AUC 0.7678 and Recall@10% 0.3467, topping all baselines.","key_machinery":"The load-bearing object is merchant-level sequence folding: the user's raw payment sequence is regrouped into sub-sequences by merchant, keeping chronological order within each merchant, so the chaotic global timeline becomes a structured set of per-merchant histories. This folding is what makes parallel Transformer processing feasible and, more importantly, exposes shifts in spending patterns, such as moving from high-end dining to convenience-store purchases, which the paper argues are markers of financial stress. It is paired with multi-field behavior encoding, using description text, sine-cosine periodicity embeddings for month, day, week, and hour, and transaction amount, plus a two-stage Transformer melding with a ViT-style [CLS] token for across-merchant relational learning.","core_discovery":"The central claim is that the internal structure of payment behavior itself, specifically the merchant a user pays, is a better grouping cue for long-sequence modeling than external hierarchies such as web-page structure or extracted events. LBSF reorganizes the chronological payment sequence by merchant, preserving time order inside each merchant, encodes each payment through concatenated description, sine-cosine time, and amount embeddings, and then applies two levels of Transformer aggregation: one to summarize each merchant's sub-sequence, and a second, ViT-style pass over merchant embeddings with a [CLS] token to produce the user representation for a binary default classifier. The paper reports that this folding consistently beats sequential baselines on 90-day and 180-day windows, reaching AUC 0.7575 and 0.7678 respectively, and that removing merchant folding causes the largest drop in an ablation study, supporting the claim that merchant-level organization carries the signal.","pith_inferences":["The paper leaves unresolved whether the behavior window ends before the default event is determined; if the windows overlap, part of the observed gain could come from label leakage rather than modeling.","The rule for choosing the 74 merchants per user is never stated; if merchants are selected by frequency, the method may partly reproduce recency and frequency signals that simpler feature-based baselines could also capture.","The same folding recipe could transfer to other long behavior-log tasks such as churn prediction or fraud detection wherever a merchant or category field provides a natural grouping, though the paper does not test this.","A testable extension is to compare folding by merchant against folding by category, time-of-day, or amount band to see whether the grouping criterion or the two-level structure itself drives the gain."],"forward_implications":["Longer payment windows, from 45 days to 90 days to 180 days, monotonically improve AUC across nearly all compared models, so platforms should retain and use longer behavior histories rather than truncating them.","Merchant-level folding contributes more than any single field embedding: ablating it drops 90-day AUC from 0.7575 to 0.7063, a larger loss than removing amount, timing, or description.","Because folding is field-based and knowledge-free, it can be applied on top of any end-to-end sequential backbone, including newer state-space models.","LBSF's attention weights over merchants give a pattern-like explanation, such as a shift from high-end spending to consumer-finance-funded shopping, which could support operational review of flagged users.","On 45-day histories LBSF does not beat Mamba, so the folding advantage appears primarily when histories are long enough to contain meaningful within-merchant trends."],"supporting_citations":[{"why":"Supplies the Transformer backbone used for within-merchant and across-merchant melding and defines the quadratic-complexity problem that folding addresses.","marker":"[19]"},{"why":"Supplies the [CLS] token and patch-style relational learning adopted for across-merchant aggregation.","marker":"[38]"},{"why":"The strongest baseline LBSF must beat on 90-day and 180-day windows; the reported margins are measured against it.","marker":"[45]"},{"why":"A baseline Transformer variant compared in the experiments, representing the efficient-attention approach the paper distinguishes from data-structure folding.","marker":"[34]"},{"why":"A baseline long-sequence Transformer variant compared in the experiments.","marker":"[37]"},{"why":"An attention-based sequential fraud-detection baseline included in the model comparison.","marker":"[15]"}],"fun_headline_variants":["Merchant folding of payment histories improves default-risk models","Folding payments by merchant beats sequential baselines in risk","Internal merchant cues in payment folding beat external hierarchies","Payment folding via merchant lifts AUC for financial risk tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result stands on an unstated boundary: the payment behaviors used for prediction must end before the default event is determined, because if the behavior window overlaps the label window, the model is reading the default rather than predicting it; a second unstated premise is that the 74 merchants chosen per user are selected without using the default label.","fun_headline_variants_meta":{"raw":{"variants":["Merchant folding of payment histories improves default-risk models","Folding payments by merchant beats sequential baselines in risk","Internal merchant cues in payment folding beat external hierarchies","Payment folding via merchant lifts AUC for financial risk tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000706,"raw_usage":{"total_tokens":3179,"prompt_tokens":937,"completion_tokens":2242,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":2179}},"tokens_in":553,"tokens_out":2242,"duration_ms":17383,"temperature":1.0,"reasoning_tokens":2179,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:33:31.526465+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the 180-day subset, fix a cutoff date well before each user's default event, discard all payments after it, and re-run LBSF versus Mamba; also rerun with merchants chosen by a label-blind rule such as top-frequency. If AUC and Recall@10% margins vanish, the longer-window gains are leakage or merchant-selection artifacts rather than genuine folding benefits.","supporting_citations":[{"cited_title":"Reformer: The efficient trans- former,","cited_arxiv_id":null,"evidence_quote":"A baseline Transformer variant compared in the experiments, representing the efficient-attention approach the paper distinguishes from data-structure folding."},{"cited_title":"Spatio- temporal attention-based neural network for credit card fraud detection,","cited_arxiv_id":null,"evidence_quote":"An attention-based sequential fraud-detection baseline included in the model comparison."}],"review_version":1}