{"id":"51285859-5994-4bf1-b69e-a83f7ad511e1","arxiv_id":"2507.08835","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A contrastively pre-trained transformer on raw transaction time series is reported to improve money-laundering detection and FDR control over tabular and LSTM baselines, though the evaluation protocol limits the strength of the claim.","lead":"This paper pre-trains a transformer on raw bank transaction histories using contrastive learning without labels, then uses the learned representations to score accounts for money laundering. It adds a two-threshold Benjamini-Hochberg procedure to control false positives, claiming better detection of both fraudsters and non-fraudsters than tabular and LSTM baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The FDR-controlled detection counts are not established because BH thresholds are not shown to be calibrated independently of the test set and realized test FDR is never reported.","rationale":"The reader identified the most load-bearing concern: the reported detection counts at controlled FDR levels depend on thresholds whose calibration is not shown to be independent of the test set, and the realized test FDR is never reported. I checked whether the paper resolves this anywhere. Section 3.3 derives p-values from labels and gives the BH FDR guarantee for the calibration population; Section 4.4 reports detection counts on test data but not the realized FDR. Appendix Table 3 documents a five-fold change in fraud prevalence between training and test, which makes transfer of the training-calibrated FDR guarantee especially doubtful. Therefore the reported gains in Tables 1 and 2 cannot be interpreted as 'more detections at the same FDR'. Additional weaknesses exist, including the absence of a rule-based baseline despite the abstract's claim of contrast with rule-based procedures, and the non-zero detections at FDR_l = 0 in Table 2, which are difficult to reconcile with Algorithm 4 at level zero. The paper does release the dataset and reports multiple random seeds, which is genuinely useful, but it does not provide code or a machine-checked protocol that would let the reader verify the FDR claims. The central empirical claim is not supported as presented, so the REJECT verdict stands.","tokens_in":1095,"tokens_out":785,"duration_ms":133387,"concrete_test":"Using the released Kaggle dataset, rerun the Section 4.4 evaluation with a strict protocol: fit each classifier on the training split, compute BH p-values and thresholds (Algorithm 4) from training scores and training labels only, then apply the fixed thresholds to test scores. Report realized FDR = V/R on the test set for each model and each nominal alpha in Tables 1 and 2. If any realized FDR materially exceeds the nominal level (e.g., transformer at alpha = 0.4), the headline claim fails; if it does not, the protocol ambiguity is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is that the transformer's two-threshold rule detects more fraudsters and non-fraudsters than LSTM or tabular baselines at a controlled FDR (Section 4.4, Tables 1 and 2). This depends on the BH thresholds of Section 3.3 having a known false-discovery rate on the test population. The paper never reports the realized FDR on the test set, and the protocol is ambiguous: Section 3.3 computes BH p-values from labels and states the guarantee of Eq. (10) for the population used to estimate them, while Section 4.4 says the five strategies were applied on the test set data without stating whether thresholds were fixed using only training labels. If test labels were used to choose thresholds, the counts are in-sample oracle counts, not out-of-sample predictions. If thresholds were fixed from training labels, the guarantee does not transfer: Appendix Table 3 changes fraud prevalence from 26.76% in training to 5.03% in test. BH's FDR bound is proportional to the null proportion in the calibration set, and a fixed threshold chosen under the training prior can have a substantially higher false-discovery proportion under the test prior. Thus the comparison across models is not at the same realized FDR, and the 'controlled FDR' claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-step pipeline for anti-money laundering detection: a transformer encoder is pre-trained with an unsupervised contrastive loss on raw transaction time series, using a similarity criterion built from tabular descriptors to sample positives and negatives; a logistic regression on the learned embeddings produces a fraud score; and a two-threshold rule, calibrated with the Benjamini-Hochberg procedure, declares high-score accounts as fraudsters and low-score accounts as non-fraudsters while ostensibly controlling the false discovery rate. The experiments compare the transformer, a fine-tuned transformer, an LSTM autoencoder, and tabular baselines on an anonymized real-life dataset, reporting the number of detected fraudsters and non-fraudsters at several nominal FDR levels.","tokens_in":27084,"tokens_out":7050,"duration_ms":81040,"significance":"If the empirical claims were fully supported, the paper would be a useful demonstration that self-supervised contrastive pretraining can produce transferable representations for a highly imbalanced financial detection task, and the two-threshold BH treatment of the false-discovery rate is a sensible way to frame the operational decision problem. The manuscript also contains positive reproducibility elements: the dataset is released on Kaggle, the preprocessing and anonymization steps are described, and detailed hyperparameter tables are provided in Appendix A.2. The main quantitative claim, however, is currently weakened by an ambiguous evaluation protocol: it is not established that the thresholds are calibrated independently of the test labels, and the realized FDR on the test set is never reported. The significance of the work therefore depends on whether the evaluation can be clarified and repaired.","major_comments":[{"comment":"The paper does not establish that the BH thresholds are calibrated independently of the test set. Section 3.3 defines the p-values through label-dependent empirical distributions, and Algorithm 4 takes the labels as an input, while Section 4.4 only states that the five strategies were applied to the test set data. If the thresholds are recomputed from the test labels, then the reported detection counts are in-sample oracle counts, not out-of-sample detections, and the comparison across models is circular. If the thresholds are fixed using only training labels, the guarantee in Eq. (10) does not automatically transfer to the test population because Appendix Table 3 changes the fraud prevalence from 26.76% in training to 5.03% in test; the BH FDR bound is proportional to the null proportion in the calibration set. The realized test-set FDR is never reported for any model or level. Please clarify the exact protocol and report, for each model and each nominal level, the realized false discovery proportion on the test set.","section":"§4.4, Tables 1 and 2"},{"comment":"The pseudocode for the BH procedure appears to apply the low-threshold correction to the high threshold. For Th the null class is non-fraudsters, as defined in Eq. (6), so the relevant null proportion is the non-fraudster proportion; the correction in Eq. (11), α' = N / (Σ yi) α, is derived for Tl, where Σ yi is the number of fraudsters. Algorithm 4 is headed 'for Th' yet computes α' with Nl = Σ yi. If this pseudocode were used as written for Th, it would inflate the target level and invalidate the advertised FDR control. Please correct the pseudocode and state explicitly which correction, if any, was used to produce Tables 1 and 2.","section":"§3.3, Algorithm 4"},{"comment":"The FDR guarantee is invoked without discussing the conditions under which the BH bound holds. The guarantee in Eq. (10) requires p-values that are valid under the null and satisfy the independence or PRDS condition; here the p-values are estimated from the same scores and labels, with a leave-one-out construction in the denominators, and the resulting dependence is not addressed either theoretically or empirically. Given that the thresholds are later applied to a test population with substantially different class proportions, the paper should either justify the applicability of the BH bound or treat the procedure as heuristic and evaluate the realized FDR directly.","section":"§3.3, Eq. (10)"}],"minor_comments":[{"comment":"The notation is inconsistent: the dataset is first written as D = {x0, ..., x_{N-1}}, but the text then says 'for each 1 ≤ i ≤ N', and later equations use indices 0 ≤ i ≤ N-1 and 1 ≤ i ≤ N interchangeably. Please normalize the indexing.","section":"§2.1"},{"comment":"The decision rule for accounts whose scores lie between Tl and Th is not explicitly described. The text says low scores are declared non-fraudsters and high scores are declared fraudsters, but it should state clearly whether the middle region is left for manual review or is assigned to a third category.","section":"§3.2"},{"comment":"In the row for Logistic Regression on tabular data at FDRh = 0.80, the '% total frauds' column reports 0.41, which appears to be a fraction rather than a percentage; other rows in the same column use values such as 37.06 and 42.80. Please make the units consistent.","section":"Appendix A.4, Table 7"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is the evaluation protocol in Section 4.4: unless the thresholds are fixed using only training labels and the realized test-set FDR is reported, the central quantitative claim of controlled detection power is not supported as written. The issue is potentially fixable by re-running the comparison with a clearly out-of-sample threshold calibration and reporting realized FDRs, which is why I recommend major revision rather than outright rejection. The public release of the dataset is a positive element that could make the revised version valuable to the AML and representation-learning communities."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: This paper combines a contrastively pre-trained transformer on raw transaction sequences with a two-threshold BH rule for AML—a new configuration—but the headline FDR-controlled detection counts are not supported because the calibration protocol is never pinned down.\n\nWhat's new and good: The combination of a single transformer encoder, InfoNCE, a memory bank, kNN-based positive sampling, and cluster-based negative sampling on raw transaction time series is genuinely new for AML. The authors release a public Kaggle dataset, which is a real contribution. The qualitative results in Section 4.3 (score distributions and t-SNE plots) do suggest the transformer representations separate fraudsters from non-fraudsters better than LSTM autoencoder or tabular baselines. The writing is clear, and the discussion of HAMLET and contrastive learning is fair.\n\nWhere it goes soft: Tables 1 and 2 report detection counts at nominal FDR levels, but the paper never states whether the BH thresholds were calibrated on training or test labels. Section 3.3 describes the procedure on a training set; Section 4.4 says the strategies were 'applied on the test set data' without saying how thresholds were fixed. If test labels were used to select thresholds, the counts are in-sample oracle numbers, not predictions. If training labels were used, the FDR guarantee in Eq. (10) does not transfer to the test population: the prevalence drops from 26.76% in training to 5.03% in test, and the realized test FDR is never reported. Either way, the comparison across models is not at the same achieved FDR, so the 'controlled FDR' claim is unsupported. That is the central pillar of the paper. There are also smaller gaps: the abstract and conclusion claim superiority over rule-based procedures, but no rule-based baseline is actually run; and the high-threshold FDR values (0.4, 0.3, 0.2) are not what a regulator would call 'under control.' The hyperparameter sensitivity is acknowledged but not analyzed. On the credit side, the dataset release and the honest limitation statements (subset of full data, label noise) indicate an honest attempt.\n\nWho this is for: Researchers in financial AI and AML who want a new self-supervised pipeline for transaction sequences. The paper is not ready to be taken as a result, but the idea and the dataset justify referee time.\n\nRecommendation: Send it to peer review, but insist that the authors clarify the calibration protocol, report realized FDR on the test set, and ideally add a simple rule-based baseline. If they can do that, the paper could be valuable.","headline":"New contrastive-transformer pipeline for AML, but the headline FDR-controlled detection counts rest on an unverified BH calibration protocol.","tokens_in":27591,"tokens_out":5287,"would_cite":true,"duration_ms":51172,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Contrastive pre-training lets a transformer learn money-laundering patterns from unlabeled transaction time series, and a two-threshold Benjamini-Hochberg rule converts its scores into fraudster detections at a controlled false-discovery…","keywords":["money laundering detection","contrastive learning","transformer","time series representation learning","Benjamini-Hochberg procedure","false discovery rate control","self-supervised pre-training","imbalanced classification"],"falsifier":"Compute the empirical p-values for the test-set scores using labels from a held-out calibration set drawn from the same period as the test set, apply the BH procedure at the levels reported in Tables 1 and 2, and record the realized FDR on the test set; if the realized $\\mathrm{FDR}_h$ or $\\mathrm{FDR}_l$ for the transformer exceeds the nominal level while the reported detection counts hold, the cross-model comparison at 'controlled' FDR is not valid. A cheaper check is seed stability: the reported counts carry standard deviations as large as the counts themselves (e.g., 23.6 ± 12.8 fraudsters at $\\mathrm{FDR}_h=0.4$), so rerunning the pre-training with more seeds would show whether the transformer's advantage over fine-tuning is an ordering or seed noise.","tokens_in":2054,"feed_emoji":"🚩","tokens_out":2074,"duration_ms":141924,"temperature":0.7,"pith_summary":"The paper claims that a transformer pre-trained with contrastive learning on raw transaction time series can produce general, label-free representations of bank accounts that make money laundering detectable with very little expert input. The pipeline encodes each account's full three-month transaction stream, learns a similarity structure from complementary tabular aggregates, and then scores every account with a logistic regression on the learned embedding. Detection uses two thresholds calibrated by the Benjamini-Hochberg procedure: a high threshold flags likely fraudsters and a low threshold clears likely non-fraudsters, both with false-discovery-rate control. On an anonymized real-life dataset, the transformer scores separate the two populations markedly better than logistic regression or XGBoost on tabular aggregates and better than an LSTM autoencoder on the same raw streams, yielding roughly two to three times more fraudster detections at a fixed FDR level and far more confidently cleared non-fraudsters. The paper positions this as a step beyond rule-based systems, which typically suffer 95-98% false-positive rates.","feed_headline":"No labels needed: contrastive transformer flags laundering","feed_subtitle":"Two FDR-controlled thresholds turn its embeddings into twice as many fraudster catches at the same false-positive rate.","key_machinery":"The load-bearing object is the contrastively pre-trained transformer encoder $f_\\theta$ composed with a projection head $h_\\gamma$, trained to minimize the InfoNCE loss on raw transaction time series, with the similarity criterion defined on the auxiliary tabular dataset $D'$: positive examples are the $\\kappa$ closest accounts in Euclidean distance in $D'$, and negative examples are drawn from the other K-means clusters. A memory bank stores recent projected embeddings to allow diverse positive and negative sampling at low computational cost, and Gaussian noise in the latent space plays the role of data augmentation that respects the temporal dependence structure. On the detection side, the machinery is the two-threshold Benjamini-Hochberg calibration: for each account an empirical p-value ranks its score against the labelled scores of the opposite class, the BH step-up rule yields the high threshold $T_h$ at FDR $\\alpha$, and the low threshold $T_l$ applies the same rule at the rescaled level $\\alpha' = (N/\\sum_i y_i)\\alpha$ so that clearing non-fraudsters also controls FDR at $\\alpha$.","core_discovery":"The central discovery is that contrastive pre-training of a transformer, without any label, yields embeddings from which a plain logistic regression produces scores whose conditional distributions for fraudsters and non-fraudsters are far better separated than those obtained from tabular aggregates or from an LSTM autoencoder. The procedure maximizes the InfoNCE contrastive loss, pulling the projection of an account toward the projections of its $\\kappa$ nearest neighbours in a tabular descriptor space ($\\kappa=50$) and pushing it away from accounts in other K-means clusters, with a memory bank for efficient sampling and Gaussian noise injected in the latent space as augmentation. A logistic regressor then scores each account, and two thresholds are calibrated by the Benjamini-Hochberg multiple-testing procedure: the high threshold $T_h$ declares fraudsters at requested FDR level $\\alpha$, and the low threshold $T_l$ declares non-fraudsters with the level rescaled by the estimated fraud proportion. In the experiments the transformer detects about twice as many fraudsters as the LSTM at $\\mathrm{FDR}_h=0.4$ (23.6 vs 8.5 of roughly 116 test fraudsters), and at $\\mathrm{FDR}_l=0.01$ it clears 54% of non-fraudsters compared with 3.4% for the LSTM, gains the authors attribute to the transformer's capacity to capture long-range dependencies in the transaction stream.","pith_inferences":["The realistic operating point of the procedure is the low threshold: clearing roughly 97% of non-fraudsters at 3% FDR is what actually removes alert volume, while the high-threshold counts at FDR levels of 0.2 or below are near zero, so an operational deployment would likely lean on $T_l$ for triage and use $T_h$ only to escalate a handful of accounts.","The design couples raw-sequence encoding with a tabular similarity signal in a way that is not laundering-specific, so the same recipe of a contrastive transformer over a sequence modality with positives and negatives drawn from a cheap tabular side view could be tested on other rare-event financial tasks such as insider trading or credit-card fraud.","Because the training and test periods have different fraud prevalences (27% vs 5%), the honest reading of 'FDR control' is a claim about transfer across time; a natural extension is to re-estimate the fraud proportion on the test period, as the paper's own rescaling formula requires, and report the realized test FDR for each model.","The seed-to-seed variation in detected counts (standard deviations comparable to the counts themselves) suggests ensembling several contrastively pre-trained encoders would be a cheap way to stabilize the two thresholds, an option the paper does not explore."],"forward_implications":["Because the pre-training uses no labels, the representation step can consume large volumes of unlabeled transaction streams, with only a small labelled set needed to fit the final logistic scorer and calibrate the two thresholds.","At a fixed requested FDR, the transformer-based pipeline detects roughly two to three times more true fraudsters than the LSTM autoencoder baseline, while the tabular baselines detect essentially none at FDR levels at or below 0.3.","On the low threshold, the transformer clears 54-97% of non-fraudster accounts at FDR levels between 0.01 and 0.03, versus at most 3.4-40% for the LSTM, meaning most routine alerts could be automatically dismissed without an analyst.","The t-SNE visualizations show one or two compact fraudster clusters that reappear across random seeds for the transformer but not for the LSTM, which the authors read as evidence that the representations capture laundering patterns rather than memorizing the training set.","The conclusion projects that, because the experiments used only a small anonymized subset of the full data, performance should improve further on larger datasets."],"supporting_citations":[{"why":"Supplies the Benjamini-Hochberg step-up procedure and the FDR guarantee that the two-threshold detection rule is built on.","marker":"[BH95]"},{"why":"Defines the InfoNCE loss that the contrastive pre-training optimizes.","marker":"[vdOL V19]"},{"why":"Provides the projection-head design and SimCLR-style contrastive framework that the encoder pipeline adapts.","marker":"[CKNH20]"},{"why":"Introduces the memory bank mechanism used for efficient positive and negative sampling during pre-training.","marker":"[HFW+20]"},{"why":"The LSTM autoencoder baseline whose embeddings are compared against the transformer's.","marker":"[SMS16]"},{"why":"The prior transformer-based money-laundering model (HAMLET) that this work contrasts with by using a single encoder and label-free pre-training.","marker":"[TPD+23]"},{"why":"Documents the 95-98% false-positive rate of rule-based systems that motivates the FDR-controlled alternative.","marker":"[LP21]"}],"fun_headline_variants":["Contrastive transformer doubles fraud catches without labels","No labels, just contrastive: transformer beats LSTM on laundering","Unsupervised transformer representations flag twice the laundering","Two-threshold contrastive transformer controls FPR and catches more","Self-supervised transformer spots fraudsters with FDR control"],"cache_read_input_tokens":29824,"weakest_assumption_plain":"The reported detection counts assume the Benjamini-Hochberg thresholds keep the false-discovery rate at the requested level on the test set, but in Sections 3.3 and 4.4 (Tables 1 and 2) the procedure is applied on the test data, the thresholds are calibrated on training data with 27% fraudsters while the test set has only 5%, and the FDR actually achieved on the test set is never reported.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive transformer doubles fraud catches without labels","No labels, just contrastive: transformer beats LSTM on laundering","Unsupervised transformer representations flag twice the laundering","Two-threshold contrastive transformer controls FPR and catches more","Self-supervised transformer spots fraudsters with FDR control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000481,"raw_usage":{"total_tokens":2395,"prompt_tokens":981,"completion_tokens":1414,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":1334}},"tokens_in":597,"tokens_out":1414,"duration_ms":10471,"temperature":1.0,"reasoning_tokens":1334,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:40:33.853299+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the empirical p-values for the test-set scores using labels from a held-out calibration set drawn from the same period as the test set, apply the BH procedure at the levels reported in Tables 1 and 2, and record the realized FDR on the test set; if the realized $\\mathrm{FDR}_h$ or $\\mathrm{FDR}_l$ for the transformer exceeds the nominal level while the reported detection counts hold, the cross-model comparison at 'controlled' FDR is not valid. A cheaper check is seed stability: the reported counts carry standard deviations as large as the counts themselves (e.g., 23.6 ± 12.8 fraudsters at $\\mathrm{FDR}_h=0.4$), so rerunning the pre-training with more seeds would show whether the transformer's advantage over fine-tuning is an ordering or seed noise.","supporting_citations":[],"review_version":1}