{"id":"d74428ae-e8e6-46ca-bdd1-75a3c57e807a","arxiv_id":"2508.06735","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper studies how lag time and sampling measure choices affect errors in MSM estimates of first-passage times and committors.","lead":"This preprint analyzes how hyperparameter and sampling choices generate errors in Markov state model estimates of mean first-passage times and committors. It evaluates a recently proposed stopped-process estimator and uses the condition number of the transition matrix to explain sensitivity to statistical noise.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified: abstract provides insufficient detail to locate a technical flaw, but the condition-number proxy claim remains unverified.","rationale":"The reader's weakest_assumption—that the condition number is a faithful proxy for error in committors and first-passage times—is indeed the most load-bearing aspect of the abstract's claim. However, this is not an identified flaw in the paper's reasoning; it is an unverified empirical assertion. Since only the abstract is available, I cannot locate a concrete technical error, a missing derivation, or an internally inconsistent step. The appropriate verdict remains UNVERDICTED, as the reader stated. My agreement is partial because I share the same concern but do not find it to be a decisive objection—only a reason to seek the full text. The concrete test I propose is a minimal verification that would either support or undermine the diagnostic claim once the full paper is examined.","tokens_in":643,"tokens_out":1496,"duration_ms":17892,"concrete_test":"Obtain the full manuscript and locate the numerical experiments. Check whether the authors report a quantitative comparison between the computed transition-matrix condition number and the actual error in mean first-passage times/committors (e.g., a scatter plot or a correlation coefficient across all test systems). If this comparison is absent or the correlation is weak, the claim that condition number is a useful sensitivity diagnostic is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"This is an abstract-only review. The central claim—that sampling-measure choice dominates MSM error for mean first-passage times and committors, and that the transition-matrix condition number is a useful sensitivity diagnostic—is plausible but not derivable from the abstract alone. The key unverified assumption, as the reader noted, is that the condition number of the estimated transition matrix faithfully proxies the error in these specific dynamical quantities. This requires the quantities to be sufficiently smooth in the transition matrix and the perturbation to be within the linear-sensitivity regime. The abstract does not state whether such a relationship was empirically validated, whether the systems tested cover a range of metastability, or whether the sample-measure effects were isolated from lag-time and basis-set effects. Without the full text, no internal inconsistency or missing derivation can be identified; the concern is one of evidence, not of a definite logical gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper (arXiv:2508.06735) analyzes error sources in Markov state model (MSM) estimates of mean first-passage times and committors. It evaluates a recently introduced 'stopped-process estimator' intended to mitigate errors caused by choosing too large a lag time, then studies statistical-sensitivity via the condition number of the estimated transition matrix. The authors highlight the sampling measure — the distribution from which initial points are drawn — as an important factor governing MSM sensitivity, with implications for variational approaches to committor evaluation.","tokens_in":848,"tokens_out":1131,"duration_ms":15244,"significance":"If the claims are borne out, the paper would provide practitioners with a concrete, inexpensive diagnostic (transition-matrix condition number) for when MSM estimates of first-passage times and committors are reliable, and it would sharpen guidance on sampling-measure choice. The condition-number approach is standard in numerical analysis, and applying it to MSM dynamical quantities is a sensible idea. However, the abstract alone provides no equations, datasets, or numerical evidence; the central relationship between condition number and errors in the specific dynamical quantities remains unverified.","major_comments":[{"comment":"The central claim that the condition number of the estimated transition matrix is a useful measure of statistical sensitivity for mean first-passage times and committors is not substantiated. The abstract provides no definition of the condition number, no statement of the perturbation model, and no numerical evidence that condition number correlates with errors in these specific quantities. Since MFPTs and committors are nonlinear functions of the transition matrix, the paper should demonstrate, with equations or experiments, that the linear-sensitivity regime applies and that the proxy holds across systems with different metastability.","section":"Abstract"},{"comment":"The evaluation of the 'stopped-process estimator' is described only in vague terms. The abstract does not state which systems, lag times, state-space discretizations, or baselines are used, nor how the estimator's error is measured. Without this, the relative contribution of lag-time error versus statistical error cannot be assessed, and the subsequent claim that sampling measure is a dominant factor cannot be separated from confounding hyperparameter choices.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract says MSMs are prone to 'systematic or statistical errors' but later focuses on 'statistical errors' for the condition-number analysis. Clarify whether the stopped-process estimator addresses systematic lag-time bias, statistical noise, or both.","section":"Abstract"},{"comment":"The phrase 'has implications for recent work applying a variational principle for evaluating the committor' is too brief to evaluate. A sentence specifying the nature of these implications would help readers judge the scope of the claim.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":"This review is based on the abstract only, as the full text was not available. The topic is appropriate and the condition-number diagnostic is plausible, but there is insufficient evidence to assess soundness. I would recommend sending the manuscript out for full review, or requesting the full text before making a decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Since we only have the abstract, everything below is provisional. The paper does something sensible: it takes two practical problems in MSM practice — choosing lag time and drawing initial points — and looks at how they affect mean first-passage times and committors. Evaluating the stopped-process estimator and using the transition-matrix condition number as a sensitivity diagnostic are legitimate extensions of existing work. The abstract is honest about the goal and doesn't oversell a new framework. If the numerical experiments actually isolate sampling measure from lag time and state-space discretization, this could give practitioners a useful rule of thumb for when MSM error is dominated by the sampling scheme rather than by noise in the transition matrix. The main soft spot is exactly what the reader flagged: the abstract asserts that the condition number of the estimated transition matrix is a useful proxy for error in specific dynamical quantities, but it doesn't show how that claim was validated. The condition number is a linear-sensitivity concept, and it's not guaranteed that committor or MFPT errors stay within that regime across a range of metastability. The abstract also doesn't say how many systems were tested or whether the results are empirical or partly theoretical. None of this is a fatal flaw — it's just not checkable from the abstract. The stress-test note is right that there's no identifiable internal contradiction, only an evidence gap. I'd send this to peer review rather than desk reject. The problem is real, the approach is standard, and the conclusions are concrete enough that a referee can verify them with the full text. My own verdict would depend on whether the condition-number proxy survives contact with the actual numerical errors, and whether the sampling-measure effect is demonstrated across a reasonable range of systems rather than a single example. If the full text delivers that, it's a solid contribution to the MSM literature. If not, the abstract has overpromised a diagnostic that the paper doesn't establish. Either way, it's worth a serious look.","headline":"A plausible, useful sensitivity analysis of MSM error that we can only judge from the abstract; worth a serious referee if the full paper backs up the condition-number proxy.","tokens_in":607,"tokens_out":1176,"would_cite":false,"duration_ms":25111,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the sampling measure used to generate starting configurations, not the lag time, is the dominant source of statistical error in Markov state model estimates of first-passage times and committors, and that the transiti","keywords":["Markov state models","mean first-passage time","committor","condition number","sampling measure","statistical error","lag time","stopped-process estimator"],"falsifier":"Construct a two-state Markov state model from equilibrium simulation data, then deliberately perturb the transition matrix in a way that keeps the condition number fixed but changes the mean first-passage time substantially. If a large change in the kinetic estimate occurs without a corresponding change in the condition number, the claimed sensitivity measure fails. More directly, run the same system under two different sampling measures, compute the condition numbers, and check whether the condition number predicts which measure produces lower error in the committor relative to a long referen","tokens_in":599,"feed_emoji":"📊","tokens_out":3667,"duration_ms":42037,"temperature":0.7,"pith_summary":"Markov state models estimate kinetic quantities like mean first-passage times and committors from simulation data. The paper tries to pin down where the errors in these estimates come from, separating systematic lag-time bias from statistical noise. It argues that a poor choice of sampling measure—the distribution from which starting points are drawn—is a dominant factor in statistical error, and that the condition number of the estimated transition matrix provides a practical measure of sensitivity to perturbation. It also evaluates a recently proposed stopped-process estimator designed to reduce error caused by a lag time that is too large. If correct, the work gives practitioners a concrete diagnostic for when their MSM kinetic estimates are trustworthy.","feed_headline":"Sampling measure drives MSM error; condition number spots it","feed_subtitle":"A transition-matrix condition number can flag when first-passage time estimates are unreliable.","key_machinery":"The central objects are the condition number of the Markov state model transition matrix and the stopped-process estimator. The condition number, typically denoted $\\kappa(T)$ for transition matrix $T$, quantifies how much the matrix's eigenvalues and derived quantities change under small perturbations; the paper proposes it as a proxy for statistical error in kinetic estimates. The stopped-process estimator is a way of computing first-passage times that avoids the fixed-lag-time waiting introduced by ordinary MSM construction, thereby reducing lag-time bias.","core_discovery":"The paper's central claim is that for estimates of mean first-passage times and committors from Markov state models, the dominant error contribution is set by the sampling measure through its effect on the estimated transition matrix's sensitivity to perturbations. The condition number of that matrix—a standard measure of how much the matrix changes under small input changes—is proposed as a useful, computable diagnostic for statistical sensitivity. This connects the error analysis to an object practitioners already build, and it underlines the importance of the distribution from which initial configurations are drawn. The paper also evaluates a stopped-process estimator, an approach that co","pith_inferences":["If the condition number is the right diagnostic, then model comparison for MSMs should control for the sampling measure; otherwise apparent improvements in a committor or rate estimate may be artifacts of sensitivity rather than genuine model quality.","The condition-number approach could be extended to other derived quantities such as mean passage times between arbitrary sets, not just the pair tested, provided the smoothness assumption holds.","A testable prediction follows: for a fixed molecular system, the ranking of statistical errors across different lag times or different sampling measures should track the ranking of the corresponding transition-matrix condition numbers.","The work suggests a practical heuristic—compute the condition number before running long validation simulations—that could be built into MSM construction tools."],"forward_implications":["Practitioners can compute a condition number of their estimated transition matrix before trusting any mean first-passage time or committor, turning sensitivity analysis into a routine diagnostic.","The sampling measure should be reported and tuned explicitly, since it can affect statistical error more strongly than the lag time itself.","Adopting the stopped-process estimator can reduce systematic error when the lag time is chosen too large.","Evaluations of committors via a variational principle need to account for the sampling measure when comparing models, since the measure influences the sensitivity of the estimates."],"supporting_citations":[],"fun_headline_variants":["Sampling measure dominates MSM error; condition number detects it","MSM error: sampling measure drives; condition number gauges","Transition-matrix condition number flags unreliable first-passage times","Sampling measure, not lag time, is the main MSM error source","For MSM first-passage times, sampling measure is the hidden lever"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the condition number of the estimated transition matrix faithfully tracks the statistical error in the mean first-passage time and committor, meaning these quantities are smooth enough functions of the matrix and the perturbation is small enough for linear sensitivity analysis to apply.","fun_headline_variants_meta":{"raw":{"variants":["Sampling measure dominates MSM error; condition number detects it","MSM error: sampling measure drives; condition number gauges","Transition-matrix condition number flags unreliable first-passage times","Sampling measure, not lag time, is the main MSM error source","For MSM first-passage times, sampling measure is the hidden lever"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002267,"raw_usage":{"total_tokens":8559,"prompt_tokens":675,"completion_tokens":7884,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":419,"completion_tokens_details":{"reasoning_tokens":7794}},"tokens_in":419,"tokens_out":7884,"duration_ms":62642,"temperature":1.0,"reasoning_tokens":7794,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:33:19.293462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a two-state Markov state model from equilibrium simulation data, then deliberately perturb the transition matrix in a way that keeps the condition number fixed but changes the mean first-passage time substantially. If a large change in the kinetic estimate occurs without a corresponding change in the condition number, the claimed sensitivity measure fails. More directly, run the same system under two different sampling measures, compute the condition numbers, and check whether the condition number predicts which measure produces lower error in the committor relative to a long referen","supporting_citations":[],"review_version":1}