{"id":"71e94aff-21f2-4bfb-b99b-5af5f1c65e37","arxiv_id":"2411.15621","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A GNN-infused set transformer with farthest-point-sampled inducing points achieves the best reported F1 for automated MRD detection in flow cytometry data.","lead":"This paper tests deep learning architectures for detecting residual leukemia cells in flow cytometry data and proposes a hybrid transformer-graph model that sets a new reported benchmark. It combines sample-derived attention summaries with local neighborhood learning, improving accuracy and cross-laboratory generalization.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper splits samples, not patients, so the same ALL patient's follow-up samples can appear in both training and test; the reported F1 gains may reflect memorization of patient-specific leukemic phenotypes rather than generalizable architecture improvements.","rationale":"I agree with the reader's weakest assumption. I considered alternative concerns: lack of significance testing is likely not decisive because the reported standard deviations over at least five runs yield a significant difference for Table 5; the 50k-event subsampling affects all methods similarly; and the use of full marker panels in graph construction is an intended property that Table 7 explicitly investigates. The patient-level split is the one assumption that, if violated, invalidates the evaluation as a measure of generalization. The manuscript does not discuss it, and the longitudinal structure of ALL follow-up data makes it a realistic, concrete risk. A patient-stratified re-split is a standard, feasible check and would settle the issue. The reader's conditional verdict already encodes this concern, so I recommend no change to the verdict.","tokens_in":11436,"tokens_out":7931,"duration_ms":74451,"concrete_test":"Obtain the patient identifiers for Vie (and ideally Bln/Bue). Re-run the full pipeline with a patient-stratified split: group all samples by patient, randomly assign patients to train/validation/test in 50/25/25 proportions, and verify no patient appears in more than one partition. Report avg/median F1 for ST, ST-FPS, GAT-ST-FPS, and GIN-ST-FPS under this split, with the same number of runs. If GIN-ST-FPS retains a comparable margin over ST (e.g., >0.02 F1) and the cross-lab ordering in Table 6 is unchanged, the leakage concern is resolved; if the margin shrinks substantially, the SOTA claim depends on patient leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 specifies a random 50/25/25 split but never states that the split is stratified by patient. The Vie dataset is 519 bone-marrow samples from pediatric b-ALL patients collected over 11 years (Section 3.3); ALL patients under therapy contribute multiple follow-up samples, so sample-level splitting will with high probability put the same patient's samples on both sides of the split. Since leukemic clones are patient-specific, a model can inflate test F1 by recognizing a patient's phenotype from training samples, and the proposed model is precisely the one that attends to sample-specific structure (FPS inducing points) and local graph context, so it may benefit disproportionately relative to the ST baseline. This threatens the headline comparison in Table 5 (0.8665 vs 0.8284) and the cross-laboratory generalization claims in Table 6. The paper reports no patient count, no patient-ID split, and no overlap analysis, so the central claim is currently not independently verifiable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether local and global feature learning improves automated measurable residual disease (MRD) detection in flow cytometry data. It compares no-context models (MLP), global-context models (MLP-mean, MLP-max, MLP-pma, PointNet, Set Transformer and variants including reluFormer and the proposed ST-FPS), local-context GNN models (GCN, GAT, GIN, and ASAP variants), and two proposed hybrid models (GAT-ST-FPS and GIN-ST-FPS) that combine one GNN layer with a Set Transformer using FPS-sampled inducing points. On the Vie dataset, GIN-ST-FPS achieves an average F1 of 0.8665 ± 0.0083 versus 0.8284 ± 0.0117 for the previous ST baseline, and the paper reports improved inter-laboratory F1 on Bln and Bue. The code is made publicly available.","tokens_in":11673,"tokens_out":4139,"duration_ms":37684,"significance":"The study is potentially valuable for the flow cytometry deep learning community: it provides a broad, controlled comparison of architectures, reports means and standard deviations over multiple runs, uses public datasets, and includes useful ablations (removing attention, increasing inducing points, changing k-NN size). The central architectural insight, that sample-derived inducing points and explicit local graph context can complement global attention, is plausible and worth testing further. If the reported gains survive patient-level evaluation and significance testing, the paper would constitute a useful new state of the art for automated MRD detection. The public code release is a concrete strength.","major_comments":[{"comment":"The train/validation/test split is described only as a random 50/25/25 split by sample, not by patient. Vie consists of 519 bone-marrow samples from pediatric b-ALL patients, and such patients typically contribute multiple follow-up samples; the paper does not report the number of patients or state that same-patient samples are kept within one split. Since leukemic clones are patient-specific, a sample-level split can put the same patient's samples in both training and test, and the proposed model's reliance on sample-derived inducing points and local graph context may benefit disproportionately from this leakage. Please report patient counts, quantify the overlap across splits, and rerun the key comparisons in Tables 5 and 6 with a patient-stratified split.","section":"3.3, Tables 5 and 6"},{"comment":"The headline comparisons are not accompanied by significance tests, and several margins are small relative to run-to-run variability. For example, ST-FPS versus ST is 0.8369 ± 0.0076 versus 0.8284 ± 0.0117, and GAT-ST-FPS versus GIN-ST-FPS is 0.8465 ± 0.0094 versus 0.8665 ± 0.0083 over at least five runs. If the same split indices are used across models, a paired test or confidence intervals on per-run differences should be reported; otherwise the claimed ordering of methods, and especially the conclusion that GIN is the right local layer, is not established.","section":"4.2 and 4.4, Tables 3 and 5"},{"comment":"The proposed GIN-ST-FPS architecture is not specified sufficiently for reproduction. The text states that one GNN layer is used and its output is concatenated with the input feature vectors before three ST-FPS layers, but no equations or tensor-dimension description are given for the concatenation, for how the k-NN graph is constructed in the hybrid model, or for how the FPS-selected inducing points relate to the GNN output. Please provide a precise layer-by-layer specification.","section":"3.2, Figure 2"},{"comment":"The cross-laboratory experiments are reported without stating how the feature dimensions are aligned across Vie, Bln, and Bue, whose marker panels differ. The text argues that graph construction can incorporate sample-specific panels, but ST-FPS and the combined models expect a fixed input feature dimension for the attention layers. It is therefore unclear which features were used as node features and how the models were evaluated on laboratories with different panels; this should be stated explicitly, including any marker matching, feature harmonization, or imputation.","section":"3.3, 4.4, Table 6"}],"minor_comments":[{"comment":"There are several typos: 'Preleminaries' should be 'Preliminaries', 'Tabel 2' should be 'Table 2', and 'cyotmetry' should be 'cytometry'.","section":"3.1, 4.1, Introduction"},{"comment":"The sentence 'the performance increase of ST-FPS does stain from sampling feature vectors' should read 'does stem from'.","section":"4.2"},{"comment":"The phrase 'spacial proximity' should be 'spatial proximity'.","section":"4.3 and 4.4"},{"comment":"The paper reports 'med F1' throughout but never defines it; please state explicitly whether it is the median over samples or the median over runs.","section":"3.3"},{"comment":"Table 1 is referenced before it appears; consider moving the table earlier or adjusting the reference.","section":"3.3"}],"recommendation":"major_revision","confidential_remarks":"The patient-level split issue is the main risk to the paper's central claim; if the authors can show that same-patient samples do not cross the split, or that patient-stratified evaluation preserves the reported gains, the paper would be substantially stronger. The authors may also wish to clarify the relationship to their earlier transformer-based FCM work, since several architectural components overlap with prior publications from the same group."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this paper. It is a straightforward, well-run empirical study that proposes two small architectural changes to the Set Transformer for flow cytometry MRD detection: replacing learned inducing points with farthest-point-sampled event vectors, and adding one GNN layer for local features. The result is a new SOTA on the Vie dataset (avg F1 0.8665 vs 0.8284 for ST) and better cross-laboratory numbers on Bln and Bue. The paper also has a genuinely useful experiment where CD10, CD19, CD45 are removed from node features but kept in the graph, showing GNN-infused models can partly recover from missing markers.\n\nWhat is actually new: FPS-based inducing points are, as far as I know, not in the Set Transformer literature, and the FCM-specific combination with a GNN layer is novel. The ablation with ST-150I (more inducing points does not help) is a good control. The study answers its three questions cleanly. Code is promised.\n\nSoft spots, in order. First, no significance tests anywhere. The ST vs ST-FPS gap (0.8369 vs 0.8284) is within one standard deviation; the GIN-ST-FPS gap looks real, but with five runs a paired t-test would settle it. Second, the dataset split is by sample, not by patient. The paper says all data were collected on day 15 after induction, which might mean one sample per patient, but it never says so and it does not report patient counts. If the same patient appears in train and test, the headline numbers could be inflated. This is a request for clarification, not a proven flaw; the day-15 detail suggests the stress-test's 'high probability' claim may be too strong. Third, the 50k-event subsampling could hurt rare-cell detection when MRD is 0.01%, and the paper does not report sensitivity to that. This matters for the clinical framing but not for the relative comparison.\n\nOverall, the central claim holds up as far as I can tell. The paper is honest, the experiments are reproducible in structure, and the improvements are plausible. I would send it to review. A careful referee should ask for significance tests and a patient-stratified split, or an explicit statement that each patient contributes exactly one sample. If the authors add those, this becomes a solid contribution.","headline":"A solid empirical paper on MRD detection with a genuinely sensible architectural tweak (FPS-selected inducing points plus a GNN layer), held back by missing significance tests and an unclarified patient/sample split.","tokens_in":12163,"tokens_out":3823,"would_cite":true,"duration_ms":32419,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Combining farthest-point-sampled attention anchors with a GNN layer improves automated MRD detection in flow cytometry data.","keywords":["flow cytometry","measurable residual disease","acute lymphoblastic leukemia","deep learning","set transformer","graph neural networks","farthest point sampling","single-cell classification"],"falsifier":"Train GIN-ST-FPS and the ST baseline on a patient-stratified split of the Vie dataset, placing every sample from a given patient in the same fold, and compare average $F_1$ on the held-out patients; if the advantage of GIN-ST-FPS shrinks or reverses, the reported gains are partly patient-specific memorization rather than generalization.","tokens_in":11238,"feed_emoji":"🩸","tokens_out":8430,"duration_ms":69481,"temperature":0.7,"pith_summary":"This paper asks whether automated detection of measurable residual disease (MRD) in flow cytometry benefits from modelling long-range dependencies, from how the global summary is obtained, and from explicit local feature learning. It answers yes to all three, with a caveat on 'how': replacing the learned query vectors of a Set Transformer with event feature vectors selected by farthest-point sampling improves results, and inserting one graph neural network layer for local feature aggregation helps further. The resulting model, GIN-ST-FPS, reaches an average $F_1$ of $0.8665 \\pm 0.0083$ on the public Vie dataset, compared with $0.8284 \\pm 0.0117$ for the previous transformer baseline, and it generalizes better to two other laboratories' datasets. The practical point is a concrete architectural recipe for applying deep learning to unstructured single-cell data where marker panels vary between samples.","feed_headline":"Adding local context to transformers lifts MRD-detection F1 to 0.8665","feed_subtitle":"Using sampled cell events as attention anchors plus a GNN layer improves cross-laboratory leukemia MRD results.","key_machinery":"The load-bearing component is the induced set attention block (ISAB) of the Set Transformer, modified so that the inducing points are not learned query vectors but event feature vectors picked from the input by farthest-point sampling, combined with a single graph neural network layer (GIN or GAT) on a $k$-NN graph with $k=10$. The FPS-sampled inducing points give attention a sample-specific global summary while keeping complexity linear in the number of events, and the GNN layer injects explicit local structure that the otherwise permutation-equivariant transformer lacks. The paper's experiments isolate the contribution of each part: removing self-attention hurts, using FPS beats learned inducing points even when the transformer is given more learned points, and capping the GNN receptive field at $k=3$ hurts performance.","core_discovery":"The paper's central claim is that the current state of the art for automated MRD detection can be pushed further by two targeted changes to the Set Transformer. First, instead of using a fixed set of learned inducing points, the model should sample roughly 150 event feature vectors from the input sample using farthest-point sampling, so the self-attention mechanism is anchored in the sample's own cell-population structure. Second, one GNN layer (GIN or GAT) should be placed before the attention blocks to explicitly aggregate local neighborhoods via a $k$-NN graph with $k=10$, giving the model an inductive bias toward spatial locality while the attention layers still capture long-range dependencies. With both changes, GIN-ST-FPS reaches average $F_1 = 0.8665 \\pm 0.0083$ on Vie, outperforming the ST baseline at $0.8284 \\pm 0.0117$, and improves average $F_1$ on the cross-laboratory Bln and Bue sets from about $0.61$ to $0.70$ and from $0.73$ to $0.84$. The paper also reports that when three diagnostically important markers are removed from node features but kept for graph construction, the GNN-infused models degrade less, supporting the claim that graph structure itself carries usable sample-specific information.","pith_inferences":["An implication the paper leaves implicit is that a patient-level split of the Vie dataset would be a stronger generalization test, because repeated samples from the same patient could inflate the reported gains.","The FPS-and-GNN recipe could transfer to other rare-cell-detection tasks in flow or mass cytometry, where event counts are high and labeled data are scarce.","Making the local graph adaptive (e.g., choosing $k$ per cell-population density) is a natural extension that might sharpen detection of leukemic cells sitting on the edge of healthy clusters."],"forward_implications":["On the public Vie dataset, GIN-ST-FPS achieves average $F_1$ of $0.8665 \\pm 0.0083$, a gain over the previous transformer baseline's $0.8284 \\pm 0.0117$.","On the cross-laboratory Bln and Bue datasets, the same model improves average $F_1$ substantially over the ST baseline, indicating better generalization to unseen laboratories' data.","How the global summary is constructed matters: sample-derived FPS anchors outperform learned inducing points, and merely increasing the number of learned inducing points does not reproduce the gain.","Explicit local feature learning is beneficial on its own: GNN-only models match or slightly exceed the transformer baseline when the $k$-NN graph is connected enough, and a restricted $k=3$ graph degrades results.","Graph structure carries usable information beyond the marker values: when CD10, CD19, and CD45 are removed from node features but kept for graph construction, GNN-infused models retain noticeably more $F_1$ than transformer-only models."],"supporting_citations":[{"why":"Provides the Set Transformer baseline for automated MRD detection in b-ALL that this paper improves on, including the inducing-point count and comparison results.","marker":"[33]"},{"why":"Introduces the Set Transformer and the induced set attention block with learned inducing points that the paper modifies.","marker":"[14]"},{"why":"Defines the GIN layer used as the local-feature-learning component of GIN-ST-FPS.","marker":"[35]"},{"why":"Defines the GAT layer used in the alternative GAT-ST-FPS variant.","marker":"[28]"},{"why":"Supplies the idea of combining graph neural networks with global attention that the proposed architecture builds on.","marker":"[34]"},{"why":"PointNet serves as a global-context baseline and shows that high-dimensional global aggregation can already outperform the transformer baseline.","marker":"[20]"},{"why":"Underlies the reluFormer baseline used to test full-range linear self-attention without the softmax nonlinearity.","marker":"[21]"},{"why":"Provides the ASAP pooling used in the GAT-ASAP and GIN-ASAP local-context baselines.","marker":"[22]"}],"fun_headline_variants":["GNN + sampled anchors boost MRD detection to F1 0.8665","Flow cytometry MRD: local context lifts F1 by 0.038","Sampled cell events and GNN layer sharpen MRD detection","Cross-lab MRD gains with GNN and attention anchors","Farthest-point sampling plus GNN lift MRD F1 to 0.87"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a random 50/25/25 split by sample keeps training and test sets independent, even though the same patient can contribute several follow-up samples in the Vie dataset.","fun_headline_variants_meta":{"raw":{"variants":["GNN + sampled anchors boost MRD detection to F1 0.8665","Flow cytometry MRD: local context lifts F1 by 0.038","Sampled cell events and GNN layer sharpen MRD detection","Cross-lab MRD gains with GNN and attention anchors","Farthest-point sampling plus GNN lift MRD F1 to 0.87"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1562,"prompt_tokens":940,"completion_tokens":622,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":523}},"tokens_in":556,"tokens_out":622,"duration_ms":5294,"temperature":1.0,"reasoning_tokens":523,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:05:32.785889+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train GIN-ST-FPS and the ST baseline on a patient-stratified split of the Vie dataset, placing every sample from a given patient in the same fold, and compare average $F_1$ on the held-out patients; if the advantage of GIN-ST-FPS shrinks or reverses, the reported gains are partly patient-specific memorization rather than generalization.","supporting_citations":[{"cited_title":"Computers in Biology and Medicine p","cited_arxiv_id":null,"evidence_quote":"Provides the Set Transformer baseline for automated MRD detection in b-ALL that this paper improves on, including the inducing-point count and comparison results."},{"cited_title":"In: Interna- tional Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Introduces the Set Transformer and the induced set attention block with learned inducing points that the paper modifies."},{"cited_title":"Advances in Neural Information Processing Systems34, 13266–13279 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the idea of combining graph neural networks with global attention that the proposed architecture builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ASAP pooling used in the GAT-ASAP and GIN-ASAP local-context baselines."}],"review_version":1}