{"id":"64d16a74-42f7-4e42-97ee-e4a30cd79fdf","arxiv_id":"2504.20011","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Curiosity-driven sampling, which measures where a neural network predicts it will make the largest spectral errors, learns structure-property maps faster than random sampling in ferroelectric microscopy.","lead":"The paper describes an active learning algorithm that decides where on a sample to take spectroscopic measurements by predicting where a deep learning model will make its biggest errors. It reports that this \"curiosity driven\" sampling reduces prediction error faster than random sampling in ferroelectric microscopy, including on a live atomic force microscope.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Fig. 6 benchmark may conflate the curiosity acquisition rule with its k-means initialization; matched-baseline and fixed-budget loss reporting are needed before the outperformance claim is verified.","rationale":"The reader's weakest assumption concerns whether high predicted error is a valid acquisition signal. I agree that is a real risk, but the more immediately load-bearing issue is the comparison protocol: the paper's own algorithm description introduces a non-random initialization mechanism, and the random baseline's initialization is not specified. If the comparison is not matched, Fig. 6 does not demonstrate that the acquisition strategy, rather than the initialization strategy, is what outperforms random sampling. The ambiguous 'minimum loss achieved' metric and the 100-vs-30 trial inconsistency further weaken the evidence. I am not rejecting the paper: the code and data are released, the real-time deployment is a useful demonstration, and the authors candidly discuss failure modes. But the central claim should be conditional on a matched-baseline rerun with fixed-budget metrics, which is a cheap and decisive check.","tokens_in":10278,"tokens_out":6837,"duration_ms":75560,"concrete_test":"Re-run the Section III.B benchmark under a matched protocol: identical k-means-seeded 30-point initialization for both curiosity and random, identical Im2spec architecture, identical retraining and error-predictor update schedule. Report held-out test loss at measurement counts 50/100/150/200 for all trials, with full distributions and a two-sided significance test. If curiosity still beats random at fixed budgets, the concern is resolved; if the advantage disappears, the headline claim must be qualified as initialization-dependent rather than a general property of the curiosity acquisition rule.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Fig. 6 is the only quantitative support for the headline claim, but it may not compare like with like. Section III.B states that the curiosity algorithm initializes with k-means-selected points in the autoencoder latent space precisely because the algorithm is sensitive to initialization. The random baseline is not reported to receive the same initialization, the same Im2spec retraining schedule, or the same error-predictor training budget. If random starts from uniform random points, the lower loss in Fig. 6 could be attributable to better initial coverage rather than to curiosity-driven acquisition. The evidence is also hard to audit: Fig. 6 reports 'minimum loss achieved' rather than loss at a fixed measurement budget, the main text says 100 trials while the caption says 30, and no per-iteration curves or p-values are shown. The paper's own acknowledgment that error-predictor targets are non-stationary and that plateau trapping can occur makes a matched-protocol evaluation especially important.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents two \"curiosity-driven\" active-learning workflows for learning structure-property relationships in scanning probe microscopy. In the first workflow (Sec. III.A), an ensemble of Im2spec models predicts spectra from image patches, and a frozen-encoder auxiliary network is trained to predict the L1 spectral mismatch; acquisitions are selected by an empirical function with an exploration/exploitation parameter beta. In the second workflow (Sec. III.B), an autoencoder latent representation of image patches feeds an error predictor of Im2spec MSE, with k-means initialization and a harmonic-mean exploration reward combined with predicted error. On a pre-acquired PFM dataset the authors report that the curiosity algorithm achieves lower Im2spec loss than random sampling (Fig. 6), and they deploy a related sampler on a real AFM for both Im2spec and Spec2im tasks (Sec. III.C). The paper argues that the approach is a lightweight alternative to deep kernel active learning for multidimensional, correlated spectral outputs, and it provides public code and data.","tokens_in":10482,"tokens_out":6787,"duration_ms":66133,"significance":"If the central claim is established, the contribution is useful: a lightweight, curiosity-style acquisition rule that can reduce the number of spectroscopic measurements needed to learn structure-property maps, with a real-time microscope demonstration and open source code and data. The paper honestly discusses known failure modes (initialization sensitivity, non-stationary error targets, plateau trapping), and the second pre-acquired experiment uses repeated trials, both of which are strengths. The main weakness is that the quantitative head-to-head evidence in Fig. 6 is under-specified: the baseline protocol is not stated to be matched, the trial count is inconsistent between text and caption, and no test statistic is reported despite the claim of statistical significance. These issues are fixable and do not invalidate the approach, but they must be addressed before the outperformance claim can be verified.","major_comments":[{"comment":"The benchmark against random sampling does not yet establish that the gain comes from curiosity-driven acquisition. The text states that the algorithm is sensitive to initialization and therefore initializes via k-means in the autoencoder latent space, but it does not state whether the random baseline also received the same k-means initialization, the same Im2spec retraining schedule, and the same error-predictor training budget. If the random baseline uses uniform random starts, the lower loss in Fig. 6 could be caused by better initial coverage rather than by the acquisition rule. Please run a matched-protocol baseline (same initialization, same retraining schedule, same per-iteration compute) and report loss at a fixed measurement budget, not only the 'minimum loss achieved.'","section":"§III.B, Fig. 6"},{"comment":"The statistical evidence for the headline claim is internally inconsistent and incomplete. The main text says the overall loss metrics were obtained 'after running 100 trials,' whereas the Fig. 6 caption says '30 trials,' and the claim that the difference between the means is statistically significant is made without reporting the test, the effect size, or a p-value. Please reconcile the trial count, report the statistical procedure and its result, and show per-iteration mean and confidence intervals so the reader can see the loss trajectories rather than only the minimum.","section":"§III.B, Fig. 6 caption"},{"comment":"The central mechanism assumes that the error predictor supplies a reliable acquisition signal, yet the paper acknowledges that the error targets are non-stationary and that training the predictor is 'very challenging,' requiring a large learning rate and multiple epochs per measurement. Because the acquisition function also contains an explicit exploration reward, the reported improvement could in principle come primarily from that reward rather than from predicted-error-based curiosity. Please add a diagnostic of error-predictor quality over iterations (e.g., correlation between predicted and actual MSE on a held-out set) or an ablation that removes the error term from the acquisition function, so the contribution of the error model to the Fig. 6 result is separable.","section":"§III.B"}],"minor_comments":[{"comment":"The data availability statement misspells 'Zenodo' as 'Zenedo'; please correct it.","section":"§V"},{"comment":"The real-time deployment section describes the results qualitatively ('decent predictions'); adding a quantitative metric, such as final test loss or a comparison with a random run on the same sample, would make the demonstration easier to evaluate.","section":"§III.C"},{"comment":"The acquisition function in Eq. (1) would be clearer if the symbols L_j, lambda, and beta were typeset with subscripts and their normalization explained immediately after the equation; the current text leaves the reader to infer that L_j is the normalized L1 error.","section":"§III.A, Eq. (1)"},{"comment":"Panels (d)-(f) of Fig. 5 would benefit from color bars and a sentence quantifying the spatial correlation between the predicted error map and the sampled exploration path, since the text relies on qualitative visual inspection.","section":"§III.B, Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a methods-demonstration paper with open code and data, and the second pre-acquired experiment uses repeated trials, which is a positive sign. My main concern is the under-specified benchmark in Fig. 6: matching of the baseline protocol, trial-count consistency, and statistical reporting. These are fixable with additional experiments and reporting changes; I do not see a fundamental flaw in the approach. The paper's reliance on the authors' own prior frameworks (im2spec, AEcroscopy) is appropriate given the implementation details."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read of arXiv:2504.20011. The paper does something useful: it takes the familiar idea of active learning by sampling where a surrogate predicts high error, and adapts it to structure-to-spectra mapping in scanning probe microscopy, with two workflows (Im2spec ensemble and autoencoder-latent error prediction), an exploration reward based on distances in latent space, and a real-time AFM deployment. The code and data are on Zenodo/GitHub, which makes it independently checkable. The authors also acknowledge failure modes (non-stationary error targets, plateau trapping, no structure-spectrum bijection) and discuss them honestly. That deserves credit.\n\nThe main quantitative claim is in Fig. 6: curiosity sampling reaches lower minimum Im2spec loss than random sampling over repeated trials. The claim is plausible, and the live deployment supports it. But the benchmark has a confound. The curiosity algorithm initializes with k-means-selected points in the autoencoder's latent space, because the algorithm is sensitive to initialization. The random baseline is never described as receiving the same initialization. If random starts from uniform random points, you can't separate the benefit of better initial coverage from the benefit of curiosity-driven acquisition. That's a load-bearing issue for the headline. There are also smaller reporting problems: Fig. 6's caption says 30 trials, the text says 100; no p-values are given, just \"statistically significant\"; the first pre-acquired experiment (Section III.A) appears to be a single run with no variance. And the introduction motivates against DKL but no DKL baseline is included; that's not fatal, but it would sharpen the contribution.\n\nThe novelty is moderate—uncertainty/expected-error sampling is old—but the specific integration with Im2spec/Spec2im, the latent-space exploration reward, and the real-time deployment make it a new result. The math and code look sound; there's no hidden circularity because the benchmark is held-out test data.\n\nWho should read it: people building self-driving labs for SPM, and anyone working on active learning for high-dimensional spectroscopic outputs. It deserves a serious referee, not a desk reject. I'd send it out, with the request that the authors rerun the random baseline with the same k-means initialization, and report loss at fixed budgets with variance.\n\nThe paper is worth engaging, but the central comparison needs to be cleaned up before I'd trust the headline.","headline":"Useful integration of error-prediction active learning with Im2spec/Spec2im, but the headline 'beats random' claim may conflate the acquisition rule with k-means initialization; fix the baseline and statistics before trusting it.","tokens_in":11028,"tokens_out":3304,"would_cite":false,"duration_ms":29663,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that a curiosity-driven active-learning algorithm, which samples where a surrogate model predicts the strongest imaging-to-spectroscopy errors, lowers prediction error compared with random sampling and runs in real time…","keywords":["active learning","curiosity-driven exploration","structure-property relationships","piezoresponse force microscopy","Im2spec","Spec2im","deep learning","autonomous microscopy"],"falsifier":"Run the 200-iteration autoencoder workflow many times with identical initialization but with the error predictor's training labels shuffled relative to the image patches, so predicted error carries no information about true Im2spec error. If the shuffled-curiosity acquisition still matches or beats random sampling, the reported gain is not caused by error prediction; conversely, a dataset in which the highest predicted-error regions are pure noise would exhibit the plateau failure mode the authors acknowledge, showing the claim depends on informative regions coinciding with high error.","tokens_in":10053,"feed_emoji":"🔬","tokens_out":8988,"duration_ms":87662,"temperature":0.7,"pith_summary":"The paper claims that a lightweight 'curiosity' algorithm can decide where a microscope should take its next spectroscopic measurement, and that this rule beats random sampling for learning structure-property mappings. The algorithm trains an auxiliary deep network to predict where the main image-to-spectrum (Im2spec) or spectrum-to-image (Spec2im) model is currently wrong, then measures at the spot with the highest predicted error. On pre-acquired ferroelectric data, this lowers prediction loss for a fixed measurement budget; the authors also run the loop in real time on an atomic force microscope. If true, the method gives materials scientists a cheap, deployable way to find structure-property relationships without the computational burden of Bayesian active learning.","feed_headline":"Curiosity-driven sampling beats random scans in microscopy","feed_subtitle":"An error-predicting AI picks which spectra to measure, cutting structure-property prediction error in ferroelectrics.","key_machinery":"The central object is an auxiliary error-prediction network: a surrogate that learns to forecast where the main structure-property model is failing. In the first workflow, the frozen encoder of the best Im2spec model is joined to a new decoder trained on the L1 spectral-mismatch error, and the acquisition function $A_j = 1 - e^{-\\lambda |L_j - (1-\\beta)|}$ tunes between exploitation ($\\beta=0$) and curiosity-driven exploration ($\\beta=1$). In the second workflow, an autoencoder maps every image patch to a latent embedding, an error predictor is trained on normalized mean-squared-error targets, and the acquisition function combines predicted error $C_j$ with an exploration reward $E_j$—the harmonic mean of distances to already measured points in latent space—under an epsilon-decreasing schedule. The error model converts 'where is the model ignorant?' into a measurable score that chooses the next spectrum.","core_discovery":"On the paper's own terms, the central discovery is that a neural-network surrogate trained to predict where the structure-property model currently fails provides a usable acquisition signal for active learning in microscopy. Against random sampling on pre-acquired piezoresponse force microscopy data, the curiosity algorithm reaches a lower minimum loss with the same number of measurements, and the difference is statistically significant. The authors further find that adding Monte Carlo dropout uncertainty to the exploration reward does not lower average Im2spec loss but does reduce error on the ten hardest points, and that the loop can be deployed on a live microscope in both forward and inverse directions. They draw the corollary that a single curiosity policy cannot optimize both directions at once, because several structures can produce the same hysteresis loop.","pith_inferences":["An implication the authors leave implicit is that persistent high predicted error at the same latent locations across iterations could serve as a stopping rule or a flag for regions where no learned mapping will suffice, directing the user to a different probe, a theoretical model, or a human decision.","A clean way to separate the contribution of the error signal from the exploration reward would be to benchmark curiosity against pure dropout-uncertainty sampling and pure latent-distance sampling; the paper's combined acquisition function makes the individual contributions hard to isolate.","Because the authors show the forward and inverse acquisition paths differ on the same sample, an autonomous experiment that needs both models could alternate between the two acquisition functions or weight them by the user's downstream task rather than committing to one policy.","The same machinery should transfer to other microscopy modalities in which an image patch predicts a spectrum, such as electron-energy-loss or cathodoluminescence mapping, provided the patch size is chosen on physical grounds as done here for ferroelectric domains."],"forward_implications":["With the same budget of spectroscopic measurements, the curiosity algorithm reaches a lower test-set prediction loss than random sampling; in the autoencoder benchmark over 100 trials the difference is statistically significant.","Acquisitions concentrate on domain walls and defective regions where structure-to-spectrum correlation is hardest, so the error signal identifies physically informative sites instead of filling a regular grid.","The loop runs in real time on a microscope, so both Im2spec and Spec2im versions are deployable during an experiment.","Because the structure–spectrum correspondence is not one-to-one, a single curiosity policy cannot simultaneously optimize the forward and inverse problems; the appropriate direction must be chosen for the application.","The method can stall on regions where high predicted error reflects noise rather than learnable structure, so practical use needs a plateau-escape mechanism such as human intervention or injected stochasticity."],"supporting_citations":[{"why":"introduces the deep-kernel active-learning approach this paper positions itself against as too expensive for high-dimensional outputs","marker":"15"},{"why":"provides the Im2spec/Spec2im encoder-decoder architecture that curiosity-driven sampling is built around","marker":"10"},{"why":"supplies the pre-acquired BE-PFM dataset used for the quantitative benchmark against random sampling","marker":"23"},{"why":"underlies the Monte Carlo dropout uncertainty estimate tested in the exploration variant","marker":"5"},{"why":"provides the microscope-control software used for real-time deployment of the algorithm","marker":"16"},{"why":"characterizes the hierarchical-domain ferroelectric sample on which the live-microscope test was run","marker":"14"},{"why":"motivates the epsilon-decreasing schedule in the acquisition function as in multi-armed bandits","marker":"24"}],"fun_headline_variants":["Curiosity-driven sampling outperforms random scans in microscopy","Error-predicting AI guides microscopy sampling to beat random","Smart sampling: AI predicts failure to map ferroelectrics faster","Neural network learns which spectra to measure next in microscopy","Curiosity algorithm cuts structure-property error in live microscopy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the spot where the model currently expects to be wrong is the most valuable place to measure next, and that the auxiliary error predictor can keep learning a target that changes as the main model improves.","fun_headline_variants_meta":{"raw":{"variants":["Curiosity-driven sampling outperforms random scans in microscopy","Error-predicting AI guides microscopy sampling to beat random","Smart sampling: AI predicts failure to map ferroelectrics faster","Neural network learns which spectra to measure next in microscopy","Curiosity algorithm cuts structure-property error in live microscopy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2795,"prompt_tokens":806,"completion_tokens":1989,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":422,"completion_tokens_details":{"reasoning_tokens":1908}},"tokens_in":422,"tokens_out":1989,"duration_ms":13445,"temperature":1.0,"reasoning_tokens":1908,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:37:26.941306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the 200-iteration autoencoder workflow many times with identical initialization but with the error predictor's training labels shuffled relative to the image patches, so predicted error carries no information about true Im2spec error. If the shuffled-curiosity acquisition still matches or beats random sampling, the reported gain is not caused by error prediction; conversely, a dataset in which the highest predicted-error regions are pure noise would exhibit the plateau failure mode the authors acknowledge, showing the claim depends on informative regions coinciding with high error.","supporting_citations":[{"cited_title":"Experimental discovery of structure--property relationships in ferroelectric materials via active learning","cited_arxiv_id":null,"evidence_quote":"introduces the deep-kernel active-learning approach this paper positions itself against as too expensive for high-dimensional outputs"},{"cited_title":"Toward decoding the relationship between domain structure and functionality in ferroelectrics via hidden latent variables","cited_arxiv_id":null,"evidence_quote":"provides the Im2spec/Spec2im encoder-decoder architecture that curiosity-driven sampling is built around"},{"cited_title":"Bayesian conavigation: Dynamic designing of the material digital twins via active learning","cited_arxiv_id":null,"evidence_quote":"supplies the pre-acquired BE-PFM dataset used for the quantitative benchmark against random sampling"},{"cited_title":"Dropout as a bayesian approximation: Representing model uncertainty in deep learning","cited_arxiv_id":null,"evidence_quote":"underlies the Monte Carlo dropout uncertainty estimate tested in the exploration variant"},{"cited_title":"Aecroscopy: a software--hardware framework empowering microscopy toward automated and autonomous experimentation","cited_arxiv_id":null,"evidence_quote":"provides the microscope-control software used for real-time deployment of the algorithm"},{"cited_title":"Exploring physics of ferroelectric domain walls in real time: deep learning enabled scanning probe microscopy","cited_arxiv_id":null,"evidence_quote":"characterizes the hierarchical-domain ferroelectric sample on which the live-microscope test was run"},{"cited_title":"Introduction to multi-armed bandits","cited_arxiv_id":null,"evidence_quote":"motivates the epsilon-decreasing schedule in the acquisition function as in multi-armed bandits"}],"review_version":1}