{"id":"3c12cd6b-f29e-4f79-9ae0-d9ce2294b77f","arxiv_id":"2502.01474","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A latent-variable label refinement scheme trains a U-Net to pick seismic first breaks while correcting noisy manual labels, improving accuracy over standard baselines.","lead":"This paper introduces a training method that automatically finds the first arrival time of seismic waves while also correcting unreliable manual labels used during training. It is relevant because noisy labels are a real obstacle in seismic monitoring, and the method reports better accuracy and robustness on two public datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SPR's refinement claim relies on P(y|ey,x)=P(y|ey) in Eq. (2); if mislabeling correlates with trace quality, the Laplace prior biases refined labels toward the network's own errors, and no experiment tests this.","rationale":"The central claim is conditional on the labeling-noise model in Eq. (2). The paper itself flags the independence assumption as a limitation, which confirms this is the right weak point rather than a manufactured one. The strongest experimental evidence, Table V, uses synthetic Gaussian label shifts that are symmetric and input-independent, so it directly follows the model's assumption instead of testing it. A synthetic experiment with input-correlated noise is the minimal check that would determine whether the method can refine realistic manual errors. Other issues, such as test-set-based selection of gamma and the underspecified ey update, affect reproducibility and the magnitude of reported gains, but they do not threaten the validity of the mechanism as directly as the independence assumption does. The reader's weakest_assumption matches this concern, so agreement is 'agree'. Since the concern is a limitation rather than a demonstrated failure, the verdict remains CONDITIONAL/UNCHANGED; if the proposed test shows a large failure under input-correlated noise, the verdict should move toward REJECT, or at least the claim should be restricted to input-independent label noise.","tokens_in":15989,"tokens_out":12031,"duration_ms":122328,"concrete_test":"Generate a synthetic first-break dataset with known true arrival times t*. Create two noisy-label versions: (A) shifts drawn from a Laplace/Gaussian distribution independent of x, as Eq. (3) assumes; (B) shifts of the same magnitude but with probability and sign correlated with trace SNR or amplitude, e.g., low-SNR traces shifted to a later cycle to mimic real annotator error. Train SPR following Algorithm 1 with gamma=5 on each version, and evaluate refined labels on held-out traces against true t* using HR and MAE. If SPR recovers t* in (A) but fails in (B), e.g., HR0 drops by more than 10 points, the independence assumption is load-bearing and the paper's real-data refinement claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (2) factors P(y,ey|x;W) as P(y|ey)P(ey|x;W), i.e., manual-label noise is independent of x given the latent first-break. This factorization is what lets Eq. (9) combine a fixed Laplace prior with the network output to refine labels. In real first-break datasets, mislabeling is likely input-dependent: low-SNR or ambiguous traces are exactly where annotators make systematic errors such as choosing a later cycle or picking noise. Under such dependence, the Laplace prior centered at the manual pick t_k in Eq. (3) pulls the refined label toward the erroneous region, and the network cannot fully correct it because its own training target ey has been regularized toward the same biased t_k. Section IV-A concedes this assumption and lists x-dependent mislabeling as future work. The synthetic noise experiment in Section III-F uses Gaussian shifts with variance 3, which are input-independent and symmetric, so they instantiate the model's assumption rather than stress-testing it. Thus the empirical support for the central claim is established only under a favorable noise model.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes SPR, a probabilistic latent-variable model for first-break picking. SPR treats the true first-break as a latent variable ey, models manual labels y as generated from ey via a Laplace prior, and trains a UNet to predict ey by alternating between cross-entropy updates of the network weights and an update of ey that balances the prior with the network output. At inference, the network output can be used directly (Eq. (8)), or combined with the prior to refine available manual labels (Eq. (9)). Experiments on the Sudbury and Lalor datasets compare SPR with five deep-learning baselines on picking accuracy, cross-site generalization, noisy signals, and noisy labels, and report that SPR generally achieves higher hit rates and lower MAE.","tokens_in":16257,"tokens_out":8856,"duration_ms":72210,"significance":"If the results hold, SPR would be a useful contribution to robust first-break picking, particularly for training with imperfect manual labels. The probabilistic framing is principled and the use of a latent variable to decouple manual labels from the learning target is a clean idea. The derivation of the objective is transparent, and the paper makes available all experimental details on public datasets. However, the validation has several gaps—an unspecified ey update, test-set hyperparameter selection, a single-direction generalization test, and a synthetic noise model that matches the model's assumptions—so the strength of the empirical claims is currently not commensurate with the conclusions.","major_comments":[{"comment":"The paper does not specify how the update of ey is computed in Eq. (7). Since ey is a binary matrix with exactly one 1 per trace, the minimization involves a discrete search (or a closed-form per-trace selection), but Algorithm 1 simply states 'Update ey by Eq. (7)' without giving the procedure, any approximation, or initialization beyond ey=y. This is load-bearing because the alternating update is the core of SPR; without this detail the algorithm is not reproducible. Please provide the exact per-trace update rule (e.g., evaluating the objective for each candidate first-break position) and discuss any issues of local optima.","section":"II-B, Eq. (7)"},{"comment":"The hyperparameter γ is selected by sweeping over γ on the Sudbury dataset, and the same dataset (or its test split) is then used for the reported results in Tables II, IV, and V. This is effectively test-set tuning: the best γ is chosen on the basis of the very metrics used to evaluate SPR in the main comparisons. The authors should instead use a separate validation split (or report results for all γ with a clear selection protocol) so that the reported numbers are not optimistically biased. This is important because Table VI shows that performance varies strongly with γ (e.g., HR0 from 55.11 to 74.20).","section":"III-G, Table VI"},{"comment":"The synthetic label-noise experiment uses Gaussian shifts with variance 3, i.e., input-independent, symmetric noise. This exactly instantiates the model's assumption in Eq. (2) that P(y|ey,x)=P(y|ey). The paper concedes in Section IV-A that real annotation errors may depend on x (low-SNR traces, different annotation standards, etc.). Without an experiment where label noise is correlated with trace characteristics, the claim that SPR 'refines misaligned manual annotations' is only validated under a favorable noise model. Adding an experiment with, e.g., larger label shifts on low-SNR traces would directly test the robustness of the refinement procedure.","section":"III-F and IV-A"},{"comment":"The cross-site generalization experiment is one-directional (train on Sudbury, test on Lalor), and on the MAE metric SPR is not the best: ResUNet achieves 5.1749 while SPR achieves 6.2842. The paper's claim that SPR 'maintained a high degree of consistency with manual picking' and shows superior generalization is therefore only partially supported. Please report the reverse direction (Lalor→Sudbury) or otherwise justify why the one-directional test is sufficient, and address the HR/MAE trade-off in the discussion.","section":"Table III"},{"comment":"All experimental results appear to be based on a single training run per configuration. Several improvements over the baselines are small (e.g., Lalor HR0: 92.60 vs 91.91 for MSNet), and without error bars or multiple seeds it is not possible to assess whether these differences are statistically meaningful. The authors should report mean and standard deviation over at least three runs (or provide an equivalent stability analysis) for the main comparisons.","section":"Tables II–V"}],"minor_comments":[{"comment":"In Section III-C, the text reads 'SPR exhibits a higher HR and a lower MSE', but the metric used throughout is MAE; please correct the terminology.","section":"III-C"},{"comment":"In Table I, the Lalor dataset is listed with 1001 sample points but an input shape of 192×1504; please reconcile this discrepancy (the text also states 1001 sampling points, while the input width 1504 suggests a different sample count).","section":"Table I"},{"comment":"The instruction 'Calculate the number of steps L based on the dataset size' is vague; please specify how L is determined (e.g., number of batches per epoch).","section":"Algorithm 1"},{"comment":"The phrase 'logging it' should be 'taking the logarithm' for clarity.","section":"II-A, Eq. (5)"},{"comment":"The two sums in Eq. (7) use different indices (k over traces, i,j over sample points); please make explicit that s_k is the index of the non-zero entry in ey_k, and that the Laplace term couples the two sums through this index.","section":"II-B, Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The core idea has merit, but the experimental validation and reproducibility need substantial improvement. The paper is well written overall, but I urge the editor to require the authors to supply the missing ey-update details, use a proper validation split for hyperparameter selection, run multiple seeds, and add an x-dependent noise experiment. The scope is more aligned with applied geophysics/remote sensing than with general computer vision, so the editor should consider the journal's readership."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: this is a competent application of noisy-label learning to first-break picking. The model is standard—latent true pick, Laplace prior on manual labels, alternating updates—but the translation to seismic picking is clean and the experiments are thorough enough to be useful. Not a breakthrough, but a solid, honest contribution.\n\nWhat is actually new: treating manual first-break picks as noisy observations of a latent true pick, with a Laplace prior that encourages most traces to keep their manual label, plus two inference modes: one for automatic picking from data and one for refining given manual picks. The reported gains on Sudbury (HR0 from 69.03 to 74.20 over MSNet) are real. The generalization test from Sudbury to Lalor is in the right direction and SPR is best on it, though the sampling-rate mismatch muddies the comparison. The noisy-label experiment shows SPR recovers a decent picker when training labels are shifted by Gaussian noise, and it also refines those shifted labels back toward the originals. That is a genuine result.\n\nSoft spots, in proportion. First, the update of the latent variable (Eq. 7) is never actually solved; the paper says it aligns with network output and manual label but gives no closed form, no approximation, no algorithm. That is a reproducibility gap. Second, gamma is selected by a sweep on the Sudbury test set (Tab. VI) with no validation split, so the headline numbers are the ones that produced the hyperparameter. This is a mild overfitting to the benchmark. Third, no error bars or repeated runs anywhere, so I cannot tell whether the differences are stable. The stress-test point about Eq. (2) is legitimate but not fatal: the paper explicitly assumes P(y|ey,x)=P(y|ey) and defends it in Section IV-A. If real mislabeling correlates with trace quality, the refinement will inherit the network's bias. The synthetic noise experiment uses Gaussian shifts that are input-independent, so it instantiates the assumption rather than stress-testing it. Still, the main application—training a picker on imperfect manual labels—does not collapse under that concern; it just means the method is safest when label errors are random rather than systematic.\n\nWho this is for: anyone in exploration geophysics working with noisy manual picks. A serious referee should engage; my recommendation is conditional acceptance after the authors actually specify the ey update, report validation-based gamma selection, and add at least one repeated-run or cross-validation sanity check.","headline":"A solid, honest application of latent-variable noisy-label learning to first-break picking, worth a serious referee but with a few reproducibility gaps.","tokens_in":16725,"tokens_out":2739,"would_cite":true,"duration_ms":25753,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A latent-variable training loop learns first-break picking from noisy manual labels.","keywords":["first-break picking","microseismic","noisy labels","latent variable","label refinement","deep learning","seismic data","UNet"],"falsifier":"Create a synthetic or semi-synthetic seismic dataset with known true first-breaks, then inject label errors whose probability increases as trace signal-to-noise ratio decreases; if SPR's refined labels are noticeably less accurate on the low-SNR subset than on the clean subset, the independence assumption in Eq. (2) is violated and a conditional noise model would be needed.","tokens_in":15839,"feed_emoji":"📡","tokens_out":6587,"duration_ms":56413,"temperature":0.7,"pith_summary":"First-break picking locates the arrival time of seismic energy, and deep-learning systems usually train on manual picks. This paper introduces SPR, which treats the true first-break as a latent variable that is updated alongside network weights, so the model learns a cleaner labeling rather than memorizing noisy manual picks. In experiments on two hard-rock seismic datasets, SPR improves exact-match hit rate and mean absolute error over five supervised baselines, and it outperforms them when labels are randomly shifted by Gaussian noise. The same latent variable can also refine manual picks at inference time. If the method holds up, noisy or partially mislabeled seismic labels no longer need a separate cleaning step before training.","feed_headline":"Latent-variable loop learns first breaks from noisy labels","feed_subtitle":"SPR trains networks on a hidden true pick, beating five supervised baselines and fixing manual mislabels.","key_machinery":"The central object is the latent first-break label $\\tilde{y}$ combined with the alternating maximization of the log-likelihood in Eq. (5). That objective couples a Laplace labeling prior $P(y|\\tilde{y})\\propto\\prod_k \\exp(-\\|t_k-s_k\\|/\\gamma)$ with the network's binary cross-entropy on $\\tilde{y}$. Fixing $\\tilde{y}$ makes the $W$-update ordinary supervised training; fixing $W$ makes the $\\tilde{y}$-update pull latent picks toward a $\\gamma$-weighted compromise between the manual pick and the network's current prediction. This single mechanism serves both automatic picking (Eq. (8)) and manual-pick refinement (Eq. (9)).","core_discovery":"The paper's central claim is that maximizing the joint likelihood of manual labels and a latent first-break, rather than the likelihood of manual labels alone, produces a first-break picker that is more accurate and more robust to label noise. The model writes $P(y,\\tilde{y}|x;W)=P(y|\\tilde{y})P(\\tilde{y}|x;W)$, with a Laplace prior on the gap between manual and latent picks and the network predicting the latent picks. Alternating updates—cross-entropy training on current latent picks for $W$, and a prior-plus-prediction compromise for $\\tilde{y}$—are what allow the method to see through outliers and mislabels. On the Sudbury and Lalor datasets the trained model achieves higher hit rates and lower mean absolute error than five comparison networks, transfers better across sites, and, when trained on labels of which only 13.30% are correct, still recovers most true first-breaks.","pith_inferences":["If mislabeling is more common on low-signal-to-noise traces, the independence assumption $P(y|\\tilde{y},x)=P(y|\\tilde{y})$ will bias refined labels toward the network's own errors, so conditioning the noise prior on $x$ is a natural extension.","The alternating procedure is a form of generalized expectation-maximization, so convergence may depend on initializing $\\tilde{y}=y$ and on the $\\gamma$ schedule; annealing $\\gamma$ during training is a testable way to reduce sensitivity to the fixed $\\gamma=5$.","The same latent-label loop could be applied to other sparse, densely annotated geophysical labels, such as P- and S-wave arrival pairs or DAS event detections, wherever label noise is sparse rather than systematic."],"forward_implications":["Seismic datasets with outlier traces or partially wrong manual picks can be used for training without a prior cleaning stage, because the latent variable absorbs sparse errors.","Any segmentation-style network can be plugged into SPR, so the accuracy gain is available to existing picking architectures without redesigning them.","The trained model can double as a label-refinement tool, letting analysts correct or audit manual picks across large surveys.","Because the network learns a distribution over the true first-break rather than memorizing manual labels, cross-site generalization is improved, as the Sudbury-to-Lalor experiment shows."],"supporting_citations":[{"why":"Supplies the Sudbury and Lalor field datasets and the manual picks that define the picking task and evaluation.","marker":"[69]"},{"why":"The original UNet architecture used as the network inside SPR, demonstrating that the algorithm is not tied to a custom network.","marker":"[68]"},{"why":"The UNet-based first-break picking method that serves as the main supervised baseline; SPR is compared against it under identical training settings.","marker":"[40]"},{"why":"Defines the noise-level protocol (standard deviation scaled by trace maximum amplitude) used in the noisy-signal experiments.","marker":"[21]"}],"fun_headline_variants":["Latent variable loop beats noisy labels for first-break picks","SPR model refines manual picks using hidden first-break variable","First-break picking gets robust to mislabels via latent truth","Joint likelihood with latent picks surpasses five supervised baselines","Hidden first-break variable improves picking despite noisy updates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the chance a manual pick is wrong does not depend on the signal itself, only on the distance from the true first-break; if noisy traces are systematically mislabeled more often, the Laplace prior cannot represent that and refinement will inherit the network's own bias.","fun_headline_variants_meta":{"raw":{"variants":["Latent variable loop beats noisy labels for first-break picks","SPR model refines manual picks using hidden first-break variable","First-break picking gets robust to mislabels via latent truth","Joint likelihood with latent picks surpasses five supervised baselines","Hidden first-break variable improves picking despite noisy updates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1641,"prompt_tokens":1020,"completion_tokens":621,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":537}},"tokens_in":636,"tokens_out":621,"duration_ms":6608,"temperature":1.0,"reasoning_tokens":537,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:10:37.618737+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Create a synthetic or semi-synthetic seismic dataset with known true first-breaks, then inject label errors whose probability increases as trace signal-to-noise ratio decreases; if SPR's refined labels are noticeably less accurate on the low-SNR subset than on the clean subset, the independence assumption in Eq. (2) is violated and a conditional noise model would be needed.","supporting_citations":[{"cited_title":"A multi-survey dataset and benchmark for first break picking in hard rock seismic exploration,","cited_arxiv_id":null,"evidence_quote":"Supplies the Sudbury and Lalor field datasets and the manual picks that define the picking task and evaluation."},{"cited_title":"First-arrival picking with a u-net convolutional network,","cited_arxiv_id":null,"evidence_quote":"The UNet-based first-break picking method that serves as the main supervised baseline; SPR is compared against it under identical training settings."},{"cited_title":"First-break picking method based on the difference between multiwindow energy ratios,","cited_arxiv_id":null,"evidence_quote":"Defines the noise-level protocol (standard deviation scaled by trace maximum amplitude) used in the noisy-signal experiments."}],"review_version":1}