{"id":"a6fadb50-81d5-4842-a857-b01fce8c4503","arxiv_id":"2501.18523","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A CNN trained on RHEED images from 31 PLD-grown SrxTi1-xO3 films predicts the XPS-measured Sr fraction with validation R2 = 0.867, and saliency maps point to (01)/(02) streak intensity and spacing as empirical stoichiometry indicators.","lead":"Researchers trained a deep neural network on just 31 electron diffraction images from growing strontium titanate films and found it could predict the strontium/titanium ratio measured later by X-ray spectroscopy. The work suggests a routine, mostly qualitative growth monitor could become a quantitative sensor for autonomous materials synthesis.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation set likely leaks growth-recipe duplicates: 31 films come from ~16 N/thickness recipes, and a random 70/30 split can put the paired film of the same recipe in train and test, so r²=0.867 may measure recipe recognition, not stoichiometry prediction.","rationale":"The paper is a plausible proof-of-concept, and the authors are transparent about dataset size, XPS limitations, and the absence of an independent composition standard. The GitHub data/code availability is a plus and enables the proposed test. I do not see internal inconsistency in the CNN architecture or the saliency analysis; the main risk is external validity of the performance number. The reader's XPS-monotonicity concern is legitimate but, as the authors argue, monotonicity is enough for a surrogate; the deeper problem is whether the validation split can certify generalization at all. Because each N/thickness recipe contributes two near-identical films, a random split makes the validation partly a duplicate test. This is a standard leakage mechanism in small materials datasets. The proposed leave-one-recipe-out check directly addresses it: if the model still achieves high r² when entire recipes are held out, the central claim is strongly supported; if not, the paper should be revised to claim only interpolation within known recipes and to treat the (01)/(02) correlations as preliminary. Since the requested additions (group-aware validation, baselines, independent confirmation) are conditions for the current claim, the verdict remains CONDITIONAL.","tokens_in":12052,"tokens_out":6298,"duration_ms":79410,"concrete_test":"Re-run the entire training pipeline with leave-one-recipe-out cross-validation: define one group per nominal N (and thickness if desired), train on all groups except one, validate only on held-out group(s), and report mean r² and RMSE with per-fold values. If the group-excluded r² falls markedly below the reported 0.867 (e.g. below ~0.5) or has high variance across folds, the central claim must be weakened to 'predicts x for recipes resembling training recipes' and the saliency correlations require independent confirmation on newly grown films.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that a gated CNN can predict the Sr atomic fraction from final RHEED images with accuracy comparable to XPS—rests on a validation procedure that does not rule out the model recognizing the deposition recipe rather than stoichiometry-dependent diffraction physics. The dataset consists of 31 films grown from only ~16 distinct (N, thickness) recipes: N ranges from -5 to 10 and two films (15 and 20 nm) were grown for each N (Methods/Fig. 1). Because x is controlled by N through the number of TiO2 pulses, any image feature that encodes N, total thickness, or growth history predicts the label almost as well as a true stoichiometry feature. A random 70/30 split of 31 samples makes it highly likely that the two films of a given recipe are separated, so the 10-sample validation contains near-duplicates of training samples. With r2_val=0.867 and no bootstrapped or group-aware error bars, the reported generalization is not established. The saliency-based (01)/(02) correlations are derived from the same non-independent samples (Fig. 4 has a visible outlier), so they inherit the same risk. The XPS-label monotonicity issue identified by the reader is real but secondary: even a perfect label would not fix a validation that can be solved by recipe memorization.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains a gated convolutional neural network to regress the Sr atomic fraction x of SrxTi1-xO3 films from final RHEED images, using a dataset of 31 films grown by sequential PLD from SrO and TiO2 targets. The model achieves r2=0.956 on training and r2=0.867 on a random 70/30 validation split. Saliency analysis is used to identify the (01) and (02) diffraction streaks as important, and the authors report manual correlations between the (01)/(02) intensity ratio and spacing ratio and the XPS-derived Sr fraction. The central claims are that the model can predict stoichiometry from RHEED images with accuracy comparable to post-growth XPS, and that explainable AI reveals previously unknown empirical correlations between diffraction features and cation ratio.","tokens_in":12353,"tokens_out":3866,"duration_ms":46356,"significance":"If the claims hold, the work would be a valuable demonstration of using deep learning to turn RHEED from a qualitative monitor into a quantitative in situ stoichiometry probe, with potential impact on autonomous synthesis and real-time control. The paper is transparent about its small dataset and provides open code and data, which are strengths. The saliency-guided discovery of specific diffraction-feature correlations is scientifically interesting, provided the correlations are robust and not an artifact of the training set. However, the current evidence does not conclusively establish the predictive claim because of potential data leakage in the validation split, and the manual correlations are not validated on independent data.","major_comments":[{"comment":"The validation procedure does not account for the non-independence of the samples. The dataset contains two films (15 nm and 20 nm) grown under each N value, i.e., roughly 16 distinct recipes for 31 films. A random 70/30 split can place both members of a recipe pair into training and validation, allowing the model to achieve high r2 by recognizing the recipe (which is essentially a proxy for x, since x is calibrated to the Sr flux fraction) rather than by learning a generalizable mapping from RHEED patterns to stoichiometry. The reported r2val=0.867 is therefore not strong evidence for the central predictive claim. The authors should perform a group-wise split (e.g., by N value or by recipe) and report validation performance on held-out recipes, as well as the performance on the paired film of the same N when the other is in training.","section":"Methods: Machine learning model training and data preprocessing; Figure 1b"},{"comment":"The claim that the model predicts Sr atomic fraction 'with an accuracy comparable to post-growth XPS measurements' is not supported by the reported metrics. No XPS measurement uncertainty or repeated-measurement precision is provided, and the validation set has only 10 samples, so r2=0.867 has a wide confidence interval. The model was also selected via hyperparameter tuning on the same 31-sample dataset (3-fold CV with Ray Tune/Optuna), so the final validation split is not a fully independent test of the selected configuration. The authors should report cross-validated performance with error bars, and ideally a group-wise CV, to substantiate the generalization claim.","section":"Results, Figure 3c and Discussion"},{"comment":"The saliency-guided empirical correlations between (01)/(02) intensity ratio, spacing ratio, and Sr atomic fraction are extracted from the same 31 samples used to train the model, and the plots show one visible outlier. No correlation coefficients, confidence intervals, or leave-one-out analyses are provided. Without an independent dataset or a formal statistical test, the assertion that these are 'previously unknown empirical correlations' is not established. The authors should quantify the correlations (e.g., Pearson r with p-values, with and without the outlier) and, if possible, validate on held-out samples.","section":"Results, Figure 4 and Conclusions"},{"comment":"The training target x is derived from XPS peak areas with no independent composition standard (e.g., RBS). The authors acknowledge that XPS values are surface-sensitive and 'not necessarily a precise estimate of film composition', but assume a monotonic relationship with true stoichiometry. If surface segregation or matrix effects break that monotonicity for some compositions, the model is predicting a distorted target, and the comparison to XPS as ground truth becomes circular. This is a limitation that should be explicitly addressed, for instance by comparing XPS and RBS on at least a few representative samples.","section":"Methods: XPS; Figure 2; Discussion"}],"minor_comments":[{"comment":"The formula in the title and abstract, Sr2xTi2(1-x)O3, is inconsistent with the notation used in the main text, which is SrxTi1-xO3. Please unify the notation.","section":"Title and Abstract"},{"comment":"The target augmentation is described as adding 'normally distributed noise with 0 mean and 0.015 variance', which implies a standard deviation of about 0.122, contradicting the stated 'typical ± 2.5 atomic percent error in XPS measurements' (which would suggest a standard deviation around 0.025). Please clarify whether 0.015 is the variance or the standard deviation.","section":"Methods: data augmentation"},{"comment":"With only 10 validation points, r2=0.867 is very sensitive to individual points. Please show the validation points distinctly and consider reporting the mean absolute error or root-mean-square error in addition to r2, along with bootstrap confidence intervals.","section":"Figure 3c"},{"comment":"The outlier point in Figures 4b and 4c is not identified or discussed. Since the correlations are central to the claim, the authors should state whether the correlations remain significant when this point is excluded, and explain its origin if possible.","section":"Figure 4"},{"comment":"The GitHub repository link is provided but the manuscript does not specify which scripts and data files it contains. Please verify that the repository includes the raw RHEED images, XPS-derived labels, and the exact model training and evaluation code to ensure reproducibility.","section":"Data and code availability"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely topic and the authors are transparent about their dataset size. The main concern is that the validation scheme is not statistically sound for the stated claims, and the saliency-driven correlations are not yet rigorously established. These issues are fixable with re-analysis and additional experiments or analyses, so I recommend major revision rather than rejection. The paper would also benefit from closer attention to notation consistency and to the XPS-label limitation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is the first demonstration I know of deep regression from raw RHEED images to a continuous stoichiometry value, and the saliency-guided (01)/(02) streak correlation is a genuinely interesting finding. But the r² = 0.867 headline should be read with caution: the validation split is too weak to support the claim that the model predicts stoichiometry with accuracy comparable to XPS.\n\nWhat is new and good: the paper does something not in the cited literature, which covers classification, clustering, and one manual-feature linear model. The gated CNN is small, the data and code are on GitHub, and the authors are candid about XPS surface sensitivity and the monotonicity assumption. The saliency maps pointing at the (01)/(02) streaks, followed by manual Gaussian fitting, is a nice use of explainability to generate a testable physical hypothesis.\n\nWhere it is soft: the stress-test concern is fair. Thirty-one films come from roughly sixteen N recipes, with two thicknesses per N. A random 70/30 split can put the paired film from the same recipe in both training and validation, so the model can score high by recognizing N-specific or thickness-specific image features rather than by learning a general stoichiometry-to-diffraction mapping. The reported validation number is a single split of ten images; the 3-fold CV mean (0.83) was used for hyperparameter tuning, not for the final claim. There is no baseline comparison to the manual-feature model or to a model that simply receives the recipe. The empirical (01)/(02) correlations are extracted and fit on the same dataset, with one visible outlier and no fit statistics, so they need confirmation on newly grown films. The XPS-label monotonicity issue is real but secondary: even a perfect label would not fix the split problem.\n\nBottom line: for anyone working on ML for thin-film synthesis, this is worth reading and worth sending to peer review. The revision needs group-aware cross-validation, an independent composition measurement, and a baseline comparison. I would not yet cite the quantitative claim as established.","headline":"A plausible and genuinely novel proof-of-concept that deserves referees, but the reported r² rests on a split that likely shares growth recipes between train and test, so the generalization claim is not yet established.","tokens_in":12898,"tokens_out":2614,"would_cite":false,"duration_ms":34042,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a gated convolutional network can regress the strontium atomic fraction of SrxTi1-xO3 films from the final RHEED image alone, with validation accuracy comparable to post-growth XPS measurements.","keywords":["pulsed laser deposition","RHEED","deep learning","gated convolutional neural network","stoichiometry prediction","saliency analysis","strontium titanate","in situ diagnostics"],"falsifier":"Measure the true cation ratio of the same 31 films with an absolute technique such as Rutherford backscattering spectrometry; if the XPS-derived strontium fraction is not monotonically ordered against the absolute ratio across the sample series, the reported $r^2$ values quantify prediction of the XPS proxy rather than of film stoichiometry.","tokens_in":11870,"feed_emoji":"🔬","tokens_out":10860,"duration_ms":108394,"temperature":0.7,"pith_summary":"This paper tries to establish that deep learning can turn reflection high-energy electron diffraction (RHEED) images, a routine and mostly qualitative in-situ diagnostic in thin-film growth, into a quantitative estimate of the cation ratio in strontium titanate films. If the claim holds, growth chambers could monitor stoichiometry in real time instead of waiting for post-growth measurements, and the same approach could be aimed at other continuously valued film properties. The authors train a gated convolutional network on just 31 films, using XPS-derived strontium fractions as training labels, and report a validation coefficient of determination of $r^2 = 0.867$, an accuracy they describe as comparable to post-growth XPS. They additionally use saliency analysis to uncover two previously unreported empirical markers: the (01)/(02) streak intensity ratio and the (01)/(02)-to-central streak spacing ratio both track the strontium fraction.","feed_headline":"Neural net predicts film chemistry from electron diffraction images","feed_subtitle":"A 31-film training set suffices; the model matches XPS accuracy and flags (01)/(02) streak ratios.","key_machinery":"The load-bearing mechanism is the gated convolution layer: a standard 2D convolution output is split along the channel dimension, one half is passed through a nonlinear activation, the other half through a sigmoid gate, and the two are multiplied so the gate acts as a learned spatial attention that rescales feature activations. Three such gated convolutions with max pooling, followed by two gated linear units with dropout, output a single scalar regression for the strontium fraction; the optimized model has 906k parameters. For interpretation, the authors adapt Grad-CAM++ to regression to generate saliency maps, and then use Gaussian fits to integrated line profiles of the RHEED images to quantify the (01)/(02) intensity ratio and the spacing ratio relative to the central streak. These two ratios are the empirical observables that carry the discovered correlation with stoichiometry.","core_discovery":"The paper's central claim is that a supervised gated convolutional neural network can convert a single post-growth RHEED image into a quantitative estimate of cation stoichiometry for homoepitaxial $\\mathrm{Sr}_x\\mathrm{Ti}_{1-x}\\mathrm{O}_3$ films grown by pulsed laser deposition. Trained on 31 films whose strontium atomic fraction $x$ was assigned from XPS Sr 3d and Ti 2p peak areas, the model reaches $r^2 = 0.956$ on the training split and $r^2 = 0.867$ on the validation split; the authors state that this is comparable to post-growth XPS measurements. Saliency analysis of the trained model concentrates on the (01) and (02) diffraction streaks, and guided by those maps the authors identify two empirical correlations they say have not been noted before: the (01)/(02) intensity ratio and the ratio of the (02) and (01) streak spacings from the central spot both increase with the strontium fraction. The paper speculates that these variations arise from distortions from the ideal $\\mathrm{SrTiO}_3$ lattice that alter the scattering intensity of the diffraction planes. The broader conclusion is that RHEED, normally limited to qualitative assessment, can serve as a quantitative surrogate measurement for continuously valued film properties.","pith_inferences":["Because the training labels come from XPS, which the paper acknowledges is affected by surface segregation, the model may be learning a surface-weighted composition rather than true bulk stoichiometry; for real-time process control a monotonic proxy may still suffice, but transferring predictions to absolute composition would require calibration against an independent standard.","The newly reported streak-intensity and spacing correlations suggest a concrete physical mechanism that could be tested by dynamical RHEED simulations or by correlating the metrics with the measured c-axis expansion from X-ray diffraction.","The authors use only the final RHEED frame; feeding the full time-resolved RHEED sequence during deposition is a natural extension that may capture growth-mode information and make predictions more robust across different growth conditions.","The saliency-guided strategy for finding empirical indicators could be transferred to other in-situ diagnostics, such as optical emission or reflectance monitoring, to mine for new quantitative signatures of film properties."],"forward_implications":["Final RHEED images alone can serve as an in-situ stoichiometry sensor during pulsed laser deposition, reducing the need for post-growth ex-situ composition measurements in growth campaigns.","The (01)/(02) streak intensity ratio and the spacing ratio relative to the central streak become real-time observable indicators of the Sr/Ti ratio in strontium titanate films.","A labeled dataset of roughly 31 samples can be sufficient for accurate regression when image and target augmentation are combined with a compact gated CNN, lowering the data barrier for ML-guided synthesis.","The same regression approach can in principle be turned on other continuously valued film properties in other materials, although the paper demonstrates it only for cation stoichiometry in this system.","Because the trained model is small and fast, it can be embedded in closed-loop or autonomous deposition workflows for on-the-fly stoichiometry control."],"supporting_citations":[{"why":"Supplies the sequential SrO/TiO2 binary-target deposition method used to vary the cation stoichiometry across the 31 films.","marker":"29"},{"why":"Established RHEED intensity oscillation behaviour for stoichiometric SrTiO3 growth, supporting the choice of the [110] azimuth and the layer-by-layer interpretation.","marker":"14"},{"why":"Documents surface segregation effects in XPS of complex oxide films, the basis for the paper's caveat that the training labels are only monotonic proxies for composition.","marker":"32"},{"why":"Demonstrates that manually engineered RHEED features can predict stoichiometry in W1-xVxSe2, the regression precedent this work extends by removing manual feature engineering.","marker":"28"},{"why":"Supplies the Grad-CAM++ method adapted for regression to produce the saliency maps that identify the (01)/(02) streaks.","marker":"35"},{"why":"Introduced the gated activation used in the network's attention mechanism.","marker":"33"},{"why":"Source of the gated convolutional network design that the model architecture is based on.","marker":"34"}],"fun_headline_variants":["AI turns RHEED images into quantitative film chemistry sensor","Deep learning reads electron diffraction to predict cation ratio","31 films train AI to predict oxide stoichiometry from RHEED","Explainable AI links diffraction streak ratios to film composition","CNN predicts Sr fraction in titanate films from single RHEED image"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the XPS-derived strontium fraction, computed from Sr 3d and Ti 2p peak areas without an independent composition standard and acknowledged to be affected by surface segregation, is a monotonic and sufficiently accurate proxy for true film stoichiometry.","fun_headline_variants_meta":{"raw":{"variants":["AI turns RHEED images into quantitative film chemistry sensor","Deep learning reads electron diffraction to predict cation ratio","31 films train AI to predict oxide stoichiometry from RHEED","Explainable AI links diffraction streak ratios to film composition","CNN predicts Sr fraction in titanate films from single RHEED image"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000275,"raw_usage":{"total_tokens":1679,"prompt_tokens":1020,"completion_tokens":659,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":575}},"tokens_in":636,"tokens_out":659,"duration_ms":7182,"temperature":1.0,"reasoning_tokens":575,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T23:08:44.716565+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the true cation ratio of the same 31 films with an absolute technique such as Rutherford backscattering spectrometry; if the XPS-derived strontium fraction is not monotonically ordered against the absolute ratio across the sample series, the reported $r^2$ values quantify prediction of the XPS proxy rather than of film stoichiometry.","supporting_citations":[],"review_version":1}