{"id":"743cf7aa-6f2b-4c64-b0ca-c9d2d3f06c13","arxiv_id":"2508.14087","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A 188M-parameter Mamba model pretrained on 11M+ simulated sPHENIX events with a new serialization and neighbor-prediction task beats task-specific baselines on three downstream detector tasks when frozen and paired with small adapters.","lead":"A team at Brookhaven trains a self-supervised model on over 11 million simulated collision events from the sPHENIX detector, then shows the frozen model can adapt to find particle tracks, identify particle types, and tag noise. If it holds up, one shared model could replace many separate specialized ML systems in collider physics.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never states that the 11M-event pretraining set is disjoint from the downstream evaluation events; if the same events appear in both, the frozen FM may be exploiting test-event exposure rather than transferable physics.","rationale":"The reader's weakest assumption focuses on serialization and pretext not being ablated. That is a legitimate methodological concern, but it is secondary to the evaluation-split issue: even a well-designed serialization and pretraining objective would not establish transfer if the downstream test events were seen during pretraining. The paper's own text never rules out overlap, and the absence of any stated split is a concrete, checkable gap. The proposed test directly addresses whether the frozen FM's downstream advantage reflects genuine generalization or memorization of event geometry. Until that test is run, the appropriate verdict remains conditional: the method is plausible, the scaling experiments are suggestive, and the downstream results are internally strong, but the central 'consistent outperformance' claim cannot be fully accepted without a clean pretraining/evaluation separation. I therefore do not change the reader's conditional verdict, though I identify a different load-bearing concern than the reader did.","tokens_in":27704,"tokens_out":4229,"duration_ms":49876,"concrete_test":"Check the dataset manifest/event IDs: verify that the events used for downstream adapter training and evaluation are disjoint from the 11M-event pretraining corpus. If overlap exists, rerun the downstream evaluation with an FM pretrained only on events excluded from all downstream train/test sets, and compare the Table 2 metrics. If ARI/accuracy drop materially, the reported gains are contaminated by test-set exposure; if unchanged, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that a frozen FM pretrained on 11M events learns task-agnostic representations that transfer to downstream track finding, PID, and noise tagging. For that claim to hold, downstream evaluation must measure generalization to events not used in pretraining. The manuscript provides no such guarantee: the dataset section describes one 11M-event simulated corpus used for pretraining, and the downstream-task section defines labeled tasks without stating that the labeled/evaluation events are a disjoint holdout from the pretraining corpus. No explicit train/test split between pretraining and downstream evaluation appears in the main text or appendices. Since the pretraining objective (k-Next-Nearest-Neighbor prediction) is computed over the geometry of each event, if any downstream test event was among the 11M pretraining events, the frozen FM has already been optimized to reconstruct the spatial structure of that exact event. The downstream adapters could then exploit event-specific features, inflating ARI/accuracy relative to baselines (EggNet, Exa.TrkX, SAGEConv, OneFormer3D) that only see labeled training data. This is not an internal inconsistency but a missing experimental control; the paper's strongest claim—consistent downstream outperformance from frozen representations—depends on resolving it.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FM4NPP, a Mamba2-based foundation model for sparse particle-detector data. Spacepoints from sPHENIX TPC simulations are serialized with a proposed Hierarchical Raster Scan, and the model is pretrained self-supervised with a k-Next-Nearest-Neighbor objective on 11M+ simulated p+p events. With frozen FM weights and lightweight adapters, the authors report performance on track finding, particle identification, and noise tagging, claim consistent gains over GNN baselines and OneFormer3D, and report scaling-law behavior in model size, data size, and compute. The paper also introduces labeled downstream tasks and argues that the learned representations are task-agnostic and can be specialized by a single linear projection.","tokens_in":28105,"tokens_out":5853,"duration_ms":64568,"significance":"If the empirical claims hold, this would be a valuable benchmark contribution: it provides a large simulated dataset, a practical frozen-FM transfer recipe, and the first demonstration of 188M-parameter pretraining for detector data, substantially larger than prior specialized track-reconstruction networks. The scaling-law and data-efficiency results are also of interest to the ML-for-physics community. However, the significance is currently undercut by unresolved data-leakage control and by internally inconsistent quantitative reporting in the main text versus appendices. The strengths are the scale of the pretraining corpus, the realistic simulation provenance, and the evaluation across three complementary downstream tasks; the weaknesses are experimental-control omissions and overstatement of the 'consistently outperforms' claim.","major_comments":[{"comment":"The paper never states that the downstream evaluation events are disjoint from the 11M-event pretraining corpus. The pretraining objective predicts k-next-nearest-neighbor coordinates for every spacepoint in an event, so if any test event appeared during pretraining, the frozen FM has already been optimized on the exact geometry of that event. This would inflate downstream metrics relative to baselines such as EggNet, Exa.TrkX, SAGEConv, and OneFormer3D, which see only labeled training events. Please specify the event-level train/test split, state whether the labeled downstream events are a subset of the pretraining corpus, and confirm that no pretraining event is used for downstream evaluation. This is load-bearing for the central transfer-learning claim.","section":"Particle Detector Dataset / Downstream Tasks"},{"comment":"The same model is reported with different numbers in the main text and appendices. For FM4NPP(m6), PID accuracy is 0.9039 in Table 2 but 0.8547 in Table 4; noise-tagging accuracy is 0.9713 in Table 2 but 0.9662 in Table 3, with recall 0.9367 vs. 0.9122 and precision 0.9190 vs. 0.9114. Track-finding efficiency/purity in Table 2 (96.08%/93.08%) also differ from the 'overall spacepoint efficiency/purity' in Table 5 (97.56%/98.34%). The authors need to reconcile these numbers and precisely define each metric; as printed, the quantitative claims are not reliable.","section":"Table 2 vs. Tables 3-5"},{"comment":"The claim that the FM 'consistently outperforms baseline models across all downstream tasks' is contradicted by the authors' own Table 2 on noise tagging: FM4NPP(m6) recall is 0.9367, lower than OneFormer3D's 0.9404. The main text later correctly says performance is 'similar' to OneFormer3D. The abstract and introduction should be revised to report per-task, per-metric superiority, or the comparison should be accompanied by statistical error bars/ tests.","section":"Abstract / Introduction / Table 2"},{"comment":"The novelty of the method rests on the Hierarchical Raster Scan serialization and the k-Next-Nearest-Neighbor objective, but no ablation is provided for either component. For example, replacing the HRS ordering with a radial sort or a space-filling curve, or replacing the k-next-neighbor objective with a simpler next-token objective, could yield similar downstream performance. Without such ablations, the specific claim that the proposed serialization and pretext are what make the representations task-agnostic is unsupported. The AdapterOnly comparison shows pretraining helps, but not that these particular design choices are necessary or beneficial.","section":"Methodology (Self-supervised Pretraining Objectives)"},{"comment":"The task-agnostic claim is supported only by qualitative PCA/t-SNE visualizations (Figures 8, 16-18). The linear projection used in the right panels is trained on downstream labels, so the emergence of track-separated clusters after projection is expected and does not by itself demonstrate that the raw FM features are task-agnostic. Please add a quantitative probe, e.g., linear-probe accuracy or a clustering metric, and compare with a randomly initialized encoder or a non-pretrained baseline to substantiate the 'single linear mapping' conclusion.","section":"Insights about FM Adaptation (Q3, Figure 8)"}],"minor_comments":[{"comment":"The event-difficulty bins and weights w_g(i) are not specified. For reproducibility, please provide the number of bins, bin boundaries, and the reweighting formula. Also specify the number of Mamba layers L and the depth of the adapters, which are not given in Table 1.","section":"Appendix B (Loss Re-scaling by Event Difficulty)"},{"comment":"The scaling-law figures show no fitted exponents, no confidence intervals, and no goodness-of-fit values. The statement that performance 'plateaus at m6' appears based on one or two points; please add fit parameters and uncertainties.","section":"Figure 5"},{"comment":"There are broken internal references: 'the same data pre-processing protocol as discussed in .' and similar dangling references. The Pyzer-Knapp reference is also duplicated (2025a and 2025b appear to be the same paper with different journal capitalization).","section":"Appendix C (Adapt GNNs)"},{"comment":"No test-set size, number of runs, or seed information is provided, so it is unclear whether the reported differences are statistically significant. Please report the number of test events and, ideally, error bars or multiple-seed results.","section":"Experiments and Results"},{"comment":"In Table 5 the FM4NPP parameter entry is listed as '188M + 2.39M', while the main text reports 2.39M trainable parameters. Please clarify whether the table reports total frozen parameters or trainable parameters, and keep the convention consistent.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the dataset release, if properly controlled, could be a useful community benchmark. The missing pretraining/evaluation split is the most serious issue; it is a missing experimental control rather than a flaw that invalidates the method. The inconsistent tables and overstated 'consistently outperforms' claim also need correction. I would support reconsideration after these are fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is the most serious attempt yet at a scaling foundation model for sparse detector data, and the central idea is genuinely new. But the paper has a load-bearing hole that needs closing before the phrase \"consistently outperforms\" can be taken at face value: it never states that the 11M-event pretraining corpus is disjoint from the events used to evaluate downstream tasks. If any downstream test event is in the pretraining set, the frozen FM has already been optimized to reconstruct that event's geometry, and the adapters can exploit familiarity rather than transferable physics. This is a missing experimental control, not a contradiction, but it sits right on the paper's central claim.\n\nWhat is actually good: the hierarchical raster scan serialization (6×8×8 boxes, radial intra-box order, box centers ordered by r/η/ϕ) and the k-Next-Nearest-Neighbor pretext are new to me; the 11M-event sPHENIX pp simulation dataset is a real resource; and the scaling study up to a 188M-parameter Mamba2 is the first of its kind in HEP. The appendices are unusually thorough—per-class recall/precision, visualizations, and details on adapting Exa.TrkX/EggNet/OneFormer3D. The data-efficiency comparison (2.9× vs 1.3× relative ARI gain) is a nice result.\n\nSoft spots, in order of seriousness. First, the disjointness issue above—please check if test events are held out. Second, the numbers don't line up: main-text Table 2 gives PID accuracy 0.9039 and noise accuracy 0.9713 for FM4NPP(m6), while appendix Tables 3–5 report 0.8547 and 0.9662 for the same model. One table must be wrong. Third, the abstract's \"consistently outperforms all baseline models\" overstates the main text's \"similar performance compared to OneFormer3D\" on noise tagging; appendix per-class metrics actually show OneFormer3D with higher macro precision/recall. Fourth, no ablations on the serialization or pretext choices; if a different ordering gives similar downstream numbers, the \"task-agnostic physics\" story weakens. Fifth, no error bars, and code/data release is not stated in the text.\n\nNone of this is fatal. The scaling laws and the representation-separation analysis are credible and don't depend on the downstream split. The paper deserves a serious referee, but the referee should ask for the split to be made explicit, the tables reconciled, and the \"consistently outperforms\" claim qualified. I'd pass it to peer review, expecting major revision.","headline":"A serious scaling-FM attempt for sparse detector data with fresh serialization and pretext, but the missing pretrain/test disjointness control and internal numeric inconsistencies undercut the headline claim.","tokens_in":28554,"tokens_out":3092,"would_cite":false,"duration_ms":34685,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A self-supervised foundation model, pretrained on 11 million simulated particle collisions and frozen, beats task-specific baselines in track finding, particle identification, and noise tagging with small adapters.","keywords":["foundation model","self-supervised learning","particle track finding","particle identification","noise tagging","Mamba2","scaling laws","sPHENIX TPC"],"falsifier":"Train the same model and data with the $k$-Next-Nearest-Neighbor target retained but the Hierarchical Raster Scan replaced first by a random permutation and then by a simple radius-sorted ordering, keeping all hyperparameters fixed. If downstream track-finding ARI stays within about 1-2 points of the reported 0.9448 in either case, the serialization is not load-bearing. Then separately replace the target with $k$ random outer-radius spacepoints instead of the $k$ nearest ones; if downstream performance is again comparable, the nearest-neighbor geometry is not load-bearing.","tokens_in":27654,"feed_emoji":"⚛️","tokens_out":11297,"duration_ms":114883,"temperature":0.7,"pith_summary":"The paper sets out to show that a foundation model for experimental particle physics is possible: sparse, unordered 3D detector data can be serialized, pretrained self-supervised at scale, and then adapted to multiple physics tasks without retraining the backbone. It introduces a dataset of more than 11 million simulated sPHENIX proton-proton collisions and defines three downstream tasks: track finding, particle identification, and noise tagging. A frozen 188-million-parameter Mamba2 model, paired with adapters of under 3 million trainable parameters, outperforms GNN track finders, the 45M-parameter OneFormer3D segmentation model, and the sPHENIX Kalman-based reconstruction pipeline on the metrics reported. The paper further claims the learned representations are task-agnostic: raw embeddings show no track structure, but a single linear projection produces separable track clusters. A sympathetic reader would care because this suggests one pretrained model could serve many collider-reconstruction tasks and remain effective with little labeled data.","feed_headline":"Frozen 188M model beats tuned GNNs on three detector tasks","feed_subtitle":"Self-supervised pretraining on 11M simulated collisions yields embeddings a single linear layer can specialize","key_machinery":"The load-bearing machinery is the pair of a Hierarchical Raster Scan and a $k$-Next-Nearest-Neighbor prediction objective, running on a Mamba2 state-space backbone. The Hierarchical Raster Scan first partitions the TPC volume into 6 radial by 8 $\\eta$ by 8 $\\phi$ boxes, sorts spacepoints within each box by radius, then orders the boxes by the $(r,\\phi,\\eta)$ coordinates of their centers; this balances local track continuity with the global outward flow of particles. The self-supervised target predicts the coordinates of the $k$ nearest spacepoints at larger radius, making the objective geometric and independent of the serialization artifacts. Mamba2 provides linear-time sequence processing,","core_discovery":"The central claim is that a sequence-model foundation model can learn generalizable detector physics from a purely geometric self-supervised objective. Spacepoints are mapped to cylindrical-polar coordinates $(r,\\phi,\\eta)$, partitioned into a $6\\times 8\\times 8$ grid, and serialized by a two-level Hierarchical Raster Scan before being processed by a Mamba2 backbone. The pretraining target is $k$-Next-Nearest-Neighbor prediction: for each spacepoint, predict the coordinates of the $k$ spatially nearest spacepoints that lie at larger radius, so the task tracks outward particle propagation and is not tied to the arbitrary 1D sequence order. After pretraining the backbone weights are frozen. Tr","pith_inferences":["The paper does not ablate the serialization scheme or the pretraining objective. A direct test is whether replacing the Hierarchical Raster Scan with a random permutation or a space-filling curve, keeping all else fixed, preserves downstream performance; if so, the specific physical ordering is less central than the scale of pretraining itself.","The claim that a single linear mapping specializes the representations is demonstrated mainly through visual dimensionality-reduction plots for track finding. A quantitative linear-probe benchmark across all three tasks would be a sharper test of the task-agnostic claim.","The dataset covers proton-proton collisions at 200 GeV with a mean of about 856 spacepoints per event. The most consequential stress test is high-occupancy heavy-ion data, where sequence lengths and track densities are far higher and the raster-scan ordering may behave very differently.","If the representation is truly task-agnostic, the same frozen backbone may extend beyond TPC tracking to calorimeter or silicon-tracker data, as the authors hint in their future-work discussion, but that extension is not yet demonstrated."],"forward_implications":["One frozen pretrained backbone can serve multiple downstream tasks: track finding, particle identification, and noise tagging are all handled by training only small adapters, so the expensive pretraining is amortized over many applications.","Pretraining on unlabeled simulated detector data scales predictably: power-law improvements in model size, data, and compute mean the same recipe can be projected to larger budgets, with the observed plateau at 188M parameters marking a saturation point worth studying.","The FM matters most where labels are scarce: relative to adapter-only training it gives a 2.9x gain in track-finding ARI at low label counts versus 1.3x at high label counts, suggesting pretraining sharply reduces the need for labeled tracking data.","Because a single linear projection on frozen embeddings creates separable track clusters, simple and cheap adapter heads are sufficient in deployment, which is why the method can beat much larger task-specific models with a fraction of the trainable parameters.","The order-independent, geometry-defined pretraining objective is a template that could be transferred to other detectors or collider geometries without redesigning the self-supervised task for each one."],"supporting_citations":[{"why":"Supplies the Mamba2 backbone and structured state-space duality architecture that the FM scales to 188M parameters.","marker":"Dao and Gu 2024"},{"why":"Exa.TrkX is the GNN track-finding pipeline adapted and used as the primary track-finding baseline.","marker":"Ju et al. 2021"},{"why":"EggNet is the evolving-graph GNN track finder used as a second baseline for track finding.","marker":"Calafiura et al. 2024"},{"why":"OneFormer3D, a 45M-parameter point-cloud segmentation model, is the strong baseline for particle identification and noise tagging.","marker":"Kolodiazhnyi et al. 2024"},{"why":"Provides the sPHENIX Cellular Automaton plus Kalman filter reconstruction pipeline against which FM4NPP is compared on high-transverse-momentum tracks.","marker":"Osborn et al. 2021"},{"why":"Defines the TrackML double-majority rule used to determine whether a predicted track matches a truth track.","marker":"Amrouche et al. 2020"},{"why":"Supplies the NeRF-style high-frequency positional encoding used to embed the spatial coordinates of each spacepoint.","marker":"Mildenhall et al. 2021"},{"why":"Provides the core simulation and reconstruction software stack used to generate the 11M-event training dataset.","marker":"sPHENIX Collaboration 2025c"}],"fun_headline_variants":["Frozen 188M FM beats tuned GNNs on three detector tasks","Self-supervised pretraining on 11M collisions yields task-agnostic embeddings","188M-parameter FM: data-efficient, beats tuned GNNs","One frozen model, three tasks, linear adapters beat GNNs"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the Hierarchical Raster Scan ordering and the $k$-Next-Nearest-Neighbor objective preserve enough track structure for a Mamba2 sequence model to learn generalizable physics; the paper does not ablate either choice, so if a different serialization or pretext gives the same downstream results, the claim that the FM specifically learns task-agnostic physics is unsupported.","fun_headline_variants_meta":{"raw":{"variants":["Frozen 188M FM beats tuned GNNs on three detector tasks","Self-supervised pretraining on 11M collisions yields task-agnostic embeddings","188M-parameter FM: data-efficient, beats tuned GNNs","One frozen model, three tasks, linear adapters beat GNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001555,"raw_usage":{"total_tokens":6031,"prompt_tokens":701,"completion_tokens":5330,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":445,"completion_tokens_details":{"reasoning_tokens":5248}},"tokens_in":445,"tokens_out":5330,"duration_ms":43329,"temperature":1.0,"reasoning_tokens":5248,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:45:01.159099+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same model and data with the $k$-Next-Nearest-Neighbor target retained but the Hierarchical Raster Scan replaced first by a random permutation and then by a simple radius-sorted ordering, keeping all hyperparameters fixed. If downstream track-finding ARI stays within about 1-2 points of the reported 0.9448 in either case, the serialization is not load-bearing. Then separately replace the target with $k$ random outer-radius spacepoints instead of the $k$ nearest ones; if downstream performance is again comparable, the nearest-neighbor geometry is not load-bearing.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Mamba2 backbone and structured state-space duality architecture that the FM scales to 188M parameters."},{"cited_title":"EggNet: An Evolving Graph-based Graph Attention Network for Particle Track Reconstruction","cited_arxiv_id":"2407.13925","evidence_quote":"EggNet is the evolving-graph GNN track finder used as a second baseline for track finding."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OneFormer3D, a 45M-parameter point-cloud segmentation model, is the strong baseline for particle identification and noise tagging."},{"cited_title":"D.; Frawley, A","cited_arxiv_id":null,"evidence_quote":"Provides the sPHENIX Cellular Automaton plus Kalman filter reconstruction pipeline against which FM4NPP is compared on high-transverse-momentum tracks."},{"cited_title":"R.; Finnie, L.; Finnie, N.; Germain, C.; Gligorov, V","cited_arxiv_id":null,"evidence_quote":"Defines the TrackML double-majority rule used to determine whether a predicted track matches a truth track."}],"review_version":1}