REVIEW 2 major objections 6 minor 28 references
A Start To End Machine Learning Approach To Maximize Scientific Throughput From The LCLS-II-HE
T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A trust-region Bayesian optimizer, run on a single scalarized objective of intensity and overlap, can align a twelve-axis X-ray split-and-delay system in minutes rather than the current one to four hours.
desk verdict A clear, honest strategy paper for facility-wide ML; its one quantitative claim (HXRSND TuRBO alignment) needs better statistics and metric calibration before being treated as a validated result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is trust-region Bayesian optimization over a scalarized objective that folds beam intensity and spatial overlap into one number. In this algorithm, local surrogate models are fit inside trust regions and samples are allocated across those regions to shrink the search manifold, which is exactly what is needed for a crystal-optics response with a sharp rise followed by a gently sloping top; the optimum sits on a flat-topped ridge in a much wider space. The scalarization matters because it lets a single optimizer chase both scientific requirements at once, while the trust regions keep the search from getting lost in the twelve-dimensional haystack.
What would settle it
Measure the photon energy of each branch of an optimizer-aligned hard X-ray split-and-delay system at the sample with an independent spectrometer; if the two branches differ by more than 0.1 eV while the pixel objective reports an aligned optimum, the central claim fails.
Extended reading notes
Core claim
The central discovery is that the hard X-ray split-and-delay alignment, previously a sequential expert task limited to two or three dimensions at a time and prone to ending far from optimal, is solvable by an off-the-shelf trust-region Bayesian optimization routine driven by a single scalarized objective of intensity and overlap. The paper reports that conventional single- and multi-objective Bayesian optimization fails to converge on this 'needle in a haystack' problem even after a few hundred iterations, whereas the trust-region variant consistently finds high-quality optima in both simulation and real beamtime: under one pixel of beam position error inside 130 samples and an intensity maximum better than the manual optimum within 100 samples. This is claimed to cut alignment time from one to four hours down to a few minutes, with the remaining time dominated by motor motion, and to remove the need for intermediate sensors by aligning all twelve degrees of freedom simultaneously.
Load-bearing premise
The whole speedup rests on assuming a screen-image metric, beam position error under one pixel on a ten-pixel-wide beam, faithfully tracks the experiment's real requirement of matching photon energy to within 0.1 eV with near-perfect spatial overlap at the sample, and the paper does not demonstrate that calibration.
Editorial extensions
If this is right
- Automated alignment of the hard X-ray split-and-delay system becomes a few-minute procedure rather than a one-to-four-hour expert task, freeing significant beamtime for user experiments.
- All twelve motor degrees of freedom can be optimized simultaneously without intermediate sensors, so the procedure no longer depends on sequential sensor placement or on an expert's ability to manage more than two or three dimensions.
- The same scalarized-objective-with-trust-regions recipe should transfer to other crystal-optics systems whose sharp, flat-topped optima defeat conventional Bayesian optimization.
- The pixel-level overlap reported at beamtime suggests the method is stable enough for real experimental conditions, not just simulation.
- Because the optimizer needs only a screen image and motor control, the optics module can be chained more easily into the paper's larger start-to-end pipeline, where upstream model outputs feed downstream objectives.
Reading between the lines
- Beyond the paper: the pixel-based objective is never calibrated against the 0.1 eV energy-matching requirement cited for X-ray photon correlation spectroscopy, so whether the optimized setting actually satisfies the scientific requirement is an open question the paper does not close.
- Beyond the paper: a direct test would be to add an independent energy-dispersive measurement of the two branches during optimization and check whether sub-pixel overlap implies sub-0.1 eV energy match.
- Beyond the paper: if this generalizes to other sharp, flat-topped crystal-optics systems, it would suggest that objective design and trust-region tuning, rather than bespoke facility-specific tooling, are the main ingredients needed for autonomous alignment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper describes a 'Start-2-End' machine-learning strategy being developed at SLAC for LCLS-II-HE, spanning the electron injector, X-ray optics, and experimental endstations. The authors propose a modular, co-designed pipeline built on common tools (Xopt, Badger, Lume, Bluesky) and organized in three phases: independent module development, chaining with uncertainty propagation, and integrated optimization. Three case studies are presented: accelerator tuning with digital twins, Bayesian optimization of the Hard X-Ray Split and Delay (HXRSND) using TuRBO, and the planned autoMFX endstation automation. The most concrete quantitative claim is that TuRBO aligns the HXRSND in all 12 dimensions in minutes rather than the current 1-4 hours, based on representative beamtime results shown in Figure 3.
Significance. If the HXRSND claim withstands scrutiny, it would be a useful demonstration that an off-the-shelf Bayesian optimization algorithm (TuRBO) can replace a slow, expert-dependent, sequential alignment procedure for a critical X-ray optics system, and it would lend credibility to the broader S2E vision. The paper's strengths are its modular architecture, its emphasis on uncertainty quantification and cascading-error awareness, and its explicit human-in-the-loop, risk-averse deployment philosophy. The authors are also transparent that much of the described work is at the planning or early-deployment stage, and they name the specific software artifacts involved. However, the paper currently functions more as a research-strategy overview than as a validated instrument-study result, and the only quantitative validation reported is not yet adequately documented.
major comments (2)
- [Section 3.2, Figure 3] The central quantitative claim that TuRBO reduces HXRSND alignment from 1-4 hours to minutes and meets the XPCS requirements lacks the required metrological link. The XPCS use case is stated to require energy matching to within 0.1 eV, near-perfect overlap, and matched intensities, but the optimization objective is reported only as beam position error in pixels (beam width 10 pixels, error under 1 pixel) and as intensity on an unspecified screen. The paper does not state where the diagnostic screen is located, whether pixel offsets at that screen are representative of the sample plane, how pixel error maps to photon-energy error or Bragg-angle error, or how the scalarized objective weights intensity against overlap. Without this calibration, the claimed optimum may satisfy the screen-based objective while failing the scientific tolerance. Please provide the calibration chain from pixels to the scientific metric and report the achieved energy and overlap values.
- [Section 3.2] The claims of 'consistent high-quality optima' and of a reduction in alignment time are not supported by the reported statistics. Figure 3 is described as representative, but no number of experimental runs, trials per run, error bars, confidence intervals, or statistical tests are given. The manual baseline ('1-4 hours', 'often far from optimal', 'better than the optimum manual setting') is not documented with respect to how the manual optimum was measured, over how many sessions the time range was observed, or what the operator-defined success criteria were. In addition, the paper does not specify the parameterization of the 12 degrees of freedom or provide evidence that all 12 were simultaneously varied during the reported runs. Please add repeated-run statistics, define the manual baseline quantitatively, and describe the 12-dimensional setup.
minor comments (6)
- [Section 2] There are doubled periods in two places: 'stand alone manner..' and 'facility..'.
- [Figure 3] Panels (b) and (c) lack axis labels and units; the text should state what is plotted on each axis and how many samples or trials are shown.
- [Section 3.2] The statement that conventional Bayesian optimization, both single- and multi-objective, 'was unable to converge to an acceptable optimum even after a few hundred iterations' is unsupported by any comparison data; please add a reference or a plot.
- [Section 3.3] Section 3.3 uses promotional wording such as 'revolutionary progression', 'immense potential', and 'ushering in an exciting future'; please replace these with measured descriptions that distinguish what has been demonstrated from what is planned.
- [Section 3.2] The HXRSND results should cite the prior work in reference [16] and clarify the relationship between the earlier LCLS-II-HE optics alignment study and the present TuRBO demonstration.
- [Section 2] The role of uncertainty propagation in chained models is asserted as a central motivation but is not illustrated with a concrete example or quantitative study; a short demonstration or a clear pointer to existing work would strengthen the section.
Circularity Check
No circular derivation: the HXRSND result is a direct experimental optimization demonstration using the externally published TuRBO algorithm.
full rationale
The paper does not derive a prediction from fitted parameters; its central claim is an empirical demonstration that the TuRBO algorithm, with a scalarized objective of intensity and overlap, aligns the HXRSND in 12 dimensions. The result is reported as direct beamtime measurements ('TuRBO can consistently achieve a minimum of under 1 pixel inside 130 samples'), not as a quantity that was constructed from the objective or from a self-citation. TuRBO is an externally published algorithm (Eriksson et al., 2019), and the paper's in-house tools (Xopt, Badger, Lume) are used as platforms, not as evidence that forces the optimization outcome. Self-citations appear in background discussions of uncertainty propagation and chained models, but none is load-bearing for the alignment claim. The uncalibrated relationship between the pixel-based objective and the 0.1 eV XPCS requirement is a validity or correctness concern, not a circularity: the paper does not claim to derive that calibration from the objective itself. No quoted equation or construction reduces the reported optimum to its input, so no circular step is present.
Assumptions & free parameters
free parameters (2)
- Objective scalarization weights (intensity vs. overlap)
- TuRBO hyperparameters (trust region sizes, initial points, candidate counts)
assumptions (3)
- domain assumption Crystal optics have a singular optimal characterized by a sharp Darwin curve with a flat top.
- ad hoc to paper Chained ML models can be made robust through uncertainty quantification and uncertainty propagation.
- domain assumption ML models trained on simulations will transfer to the real accelerator with operator supervision.
Cite this review
Pith. "Pith review of A Start To End Machine Learning Approach To Maximize Scientific Throughput From The LCLS-II-HE." pith.science (2026). https://pith.science/paper/PQOG37GC
@misc{pith2026250523858,
author = {Pith},
title = {Pith review of: A Start To End Machine Learning Approach To Maximize Scientific Throughput From The LCLS-II-HE},
year = {2026},
howpublished = {\url{https://pith.science/paper/PQOG37GC}},
note = {Machine review of arXiv:2505.23858}
}
read the original abstract
With the increasing brightness of Light sources, including the Diffraction-Limited brightness upgrade of APS and the high-repetition-rate upgrade of LCLS, the proposed experiments therein are becoming increasingly complex. For instance, experiments at LCLS-II-HE will require the X-ray beam to be within a fraction of a micron in diameter, with pointing stability of a few nanoradians, at the end of a kilometer-long electron accelerator, a hundred-meter-long undulator section, and tens of meters long X-ray optics. This enhancement of brightness will increase the data production rate to rival the largest data generators in the world. Without real-time active feedback control and an optimized pipeline to transform measurements to scientific information and insights, researchers will drown in a deluge of mostly useless data, and fail to extract the highly sophisticated insights that the recent brightness upgrades promise. In this article, we outline the strategy we are developing at SLAC to implement Machine Learning driven optimization, automation and real-time knowledge extraction from the electron-injector at the start of the electron accelerator, to the multidimensional X-ray optical systems, and till the experimental endstations and the high readout rate, multi-megapixel detectors at LCLS to deliver the design performance to the users. This is illustrated via examples from Accelerator, Optics and End User applications.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
R. Roussel, A. Edelen, A. Bartnik, and C. Mayes, Xopt: A simplified framework for opti- mization of accelerator problems using advanced algorithms, in Proc. IPAC’23, 05. JACoW Publishing, Geneva, Switzerland, no. 14 in IPAC’23 - 14th International Particle Acceler- ator Conference, 2023, pp. 4796–4799, Available at https://indico.jacow.org/event/ 41/contr...
work page 2023
-
[3]
Z. Zhang, M. B¨ ose, A. Edelen, J. Garrahan, Y. Hidaka, C. Mayes, S. Miskovich, D. Ratner, R. Roussel, J. Shtalenkova, S. Tomin, and G. Wang, Badger: The Missing Optimizer in ACR , in Proc. IPAC’22, 07. JACoW Publishing, Geneva, Switzerland, no. 13 in International Particle Accelerator Conference, 2022, pp. 999–1002, Available at https://jacow.org/ipac202...
work page 2022
- [4]
-
[5]
A. Le Borgne, X. Marjou, F. Parzysz, and T. Lemlouma, Comparing a composite model versus chained models to locate a nearest visual object , in 2023 IEEE/ACIS 8th Interna- tional Conference on Big Data, Cloud Computing, and Data Science (BCD) . IEEE, 2023, pp. 231–236
work page 2023
-
[6]
T. Shafighfard, F. Kazemi, F. Bagherzadeh, M. Mieloszyk, and D.Y. Yoo, Chained ma- chine learning model for predicting load capacity and ductility of steel fiber–reinforced concrete beams, Computer-Aided Civil and Infrastructure Engineering 39 (2024), pp. 3573–3594. 11
work page 2024
- [7]
-
[8]
D. Douglas, A. Mishra, D. Ratner, F. Petersen, and K. Terao, Uncertainty propagation within chained models for machine learning reconstruction of neutrino-lar interactions , arXiv preprint arXiv:2411.09864 (2024)
work page Pith review arXiv 2024
Show all 28 references
-
[9]
Carreras and L
X. Carreras and L. M` arquez,Introduction to the CoNLL-2005 shared task: Semantic role labeling, in Proceedings of the ninth conference on computational natural language learning (CoNLL-2005). 2005, pp. 152–164
2005
-
[10]
Attardi, V
G. Attardi, V. Basile, C. Bosco, T. Caselli, F. Dell’Orletta, S. Montemagni, V. Patti, M. Simi, and R. Sprugnoli, State of the art language technologies for italian: The evalita 2014 perspective, Intelligenza Artificiale 9 (2015), pp. 43–61
2015
-
[11]
Li and M
Y. Li and M. van der Schaar, On error propagation of diffusion models , arXiv preprint arXiv:2308.05021 (2023)
2023 arXiv
-
[12]
Gow, Uncertainty quantication and propagation through complex chains of computa- tional models, Ph.D
S. Gow, Uncertainty quantication and propagation through complex chains of computa- tional models, Ph.D. diss., University of Southampton, 2021
2021
-
[13]
Eriksson, M
D. Eriksson, M. Pearce, J. Gardner, R.D. Turner, and M. Poloczek, Scalable global op- timization via local bayesian optimization , Advances in neural information processing systems 32 (2019)
2019
-
[14]
Gupta, A
L. Gupta, A. Edelen, N. Neveu, A. Mishra, C. Mayes, and Y.K. Kim, Improving surrogate model accuracy for the lcls-ii injector frontend using convolutional neural networks and transfer learning, Machine Learning: Science and Technology 2 (2021), p. 045025
2021
-
[15]
Edelen, N
A. Edelen, N. Neveu, C. Mayes, C. Emma, and D. Ratner, Machine learning models for optimization and control of x-ray free electron lasers , in NeurIPS Machine Learning for the Physical Sciences Workshop . 2019
2019
-
[16]
Mishra, N
A. Mishra, N. Brennan, T. Huang, J. Jaquith, H. Yavas, and M. Seaberg, Machine learning-based alignment for LCLS-II-HE optics , in Advances in Computational Meth- ods for X-Ray Optics VI , Vol. 12697. SPIE, 2023, pp. 43–48
2023
-
[17]
Psaros, X
A.F. Psaros, X. Meng, Z. Zou, L. Guo, and G.E. Karniadakis, Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons , Journal of Computa- tional Physics 477 (2023), p. 111902
2023
-
[18]
K.D.H. Koh, A. Mishra, and K. Terao, Evaluating deep learning uncertainty quantification methods for neutrino physics applications , in 35th Conference on Neural Information Processing Systems. 2021
2021
-
[19]
D. Koh, A. Mishra, and K. Terao, Deep neural network uncertainty quantification for lartpc reconstruction, Journal of Instrumentation 18 (2023), p. P12013
2023
-
[20]
Mishra, A
A.A. Mishra, A. Edelen, A. Hanuka, and C. Mayes, Uncertainty quantification for deep learning in particle accelerator applications , Physical Review Accelerators and Beams 24 (2021), p. 114601
2021
-
[21]
Smith, Uncertainty quantification: theory, implementation, and applications , SIAM, 2024
R.C. Smith, Uncertainty quantification: theory, implementation, and applications , SIAM, 2024
2024
-
[22]
Mishra, J
A.A. Mishra, J. Mukhopadhaya, G. Iaccarino, and J. Alonso, Uncertainty estimation module for turbulence model predictions in su2 , AIAA Journal 57 (2019), pp. 1066–1077
2019
-
[23]
Petersen, A
F. Petersen, A. Mishra, H. Kuehne, C. Borgelt, O. Deussen, and M. Yurochkin, Uncer- tainty quantification via stable distribution propagation , arXiv preprint arXiv:2402.08324 (2024)
2024 arXiv
-
[24]
Petersen, C
F. Petersen, C. Borgelt, A. Mishra, and S. Ermon, Generalizing stochastic smoothing for differentiation and gradient estimation , arXiv preprint arXiv:2410.08125 (2024)
2024 arXiv
-
[25]
Hoidn, A.A
O. Hoidn, A.A. Mishra, and A. Mehta, Physics constrained unsupervised deep learning for rapid, high resolution scanning coherent diffraction reconstruction , Scientific Reports 13 (2023), p. 22789
2023
-
[26]
Vong, S.F
A. Vong, S.F. Henke, O. Hoidn, H. Parraga, T. Zhou, A. Mehta, and N. Schwarz, Portable Cross-Facility Workflows for X-ray Ptychography, in SC24-W: Workshops of the Interna- tional Conference for High Performance Computing, Networking, Storage and Analysis . 12 IEEE, 2024, pp. ...
2024
-
[27]
Sierra, A
R.G. Sierra, A. Batyuk, Z. Sun, A. Aquila, M.S. Hunter, T.J. Lane, M. Liang, C.H. Yoon, R. Alonso-Mori, R. Armenta, et al. , The macromolecular femtosecond crystallography instrument at the linac coherent light source , Synchrotron Radiation 26 (2019), pp. 346– 357
2019
-
[28]
Boutet, A.E
S. Boutet, A.E. Cohen, and S. Wakatsuki, The new macromolecular femtosecond crystal- lography (mfx) instrument at lcls , Synchrotron radiation news 29 (2016), pp. 23–28. 13
2016
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.