{"id":"1535a356-d6b9-4310-8895-23f8cdcf20f8","arxiv_id":"2508.18853","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A review arguing that identifiability analysis is a prerequisite for reliable parameter estimation and prediction in systems biology, and summarizing available computational tools.","lead":"This paper reviews how parameter identifiability, sensitivity analysis, and uncertainty quantification work in systems biology models, and why they must be checked before fitting. It surveys methods from the Fisher information matrix to profile likelihoods and synthetic-data checks, with practical advice for making models more identifiable.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FIM rank and local structural identifiability are not equivalent in nonlinear models; the stated 'if and only if' is false and can misdiagnose identifiable parameters.","rationale":"The reader's verdict is CONDITIONAL, and I agree with that overall assessment. However, the single most load-bearing technical concern is not the model-misspecification limitation that the reader nominated as weakest, but the paper's unqualified claim that det I(θ) ≠ 0 is equivalent to local structural identifiability. This is a concrete, easily demonstrated mathematical falsehood in the main methodological discussion, and it affects the FIM-based workflow that the paper recommends. The model-discrepancy caveat in the Conclusions is explicitly acknowledged and partly addressed by urging predictive validation, so it is a scope limitation rather than an internal inconsistency. The FIM equivalence, by contrast, is stated without qualification and can generate false structural-unidentifiability conclusions in nonlinear models. Since the reader already noted this overstatement in the rationale and assigned CONDITIONAL on that basis, my independent analysis does not change the recommended verdict. The paper should correct the equivalence before acceptance, ideally by stating that full-rank FIM is sufficient, not necessary, for local structural identifiability, and by adding a caveat about zero-sensitivity points.","tokens_in":11944,"tokens_out":7085,"duration_ms":75412,"concrete_test":"Use the scalar model f(θ)=θ^3 with a single observation y=f(θ)+ε, ε~N(0,σ²), and θ*=0. Compute V(θ*)=3θ*²=0, so det I(θ*)=0. Verify from the definitions in the 'Is it mission impossible?' section that the model is structurally locally identifiable at θ*: for any θ≠0 in a neighborhood of 0, f(θ)≠f(0)=0. This directly falsifies the stated 'if and only if'. As a control, replace the equivalence with the one-way implication 'V(θ*) full column rank ⇒ local structural identifiability', which holds by the inverse function theorem, and check that the counterexample does not violate the corrected statement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"At the end of the 'From linear to nonlinear' section, the paper states: 'A model is locally structurally identifiable at θ if and only if det I(θ) ≠ 0.' This equivalence is false for nonlinear models. Local structural identifiability at θ* is local injectivity of f at θ*. By the inverse function theorem, full column rank of V(θ*) = df/dθ|θ* is sufficient, but it is not necessary. A scalar counterexample is f(θ) = θ^3 at θ* = 0: the map is injective on R, hence locally (indeed globally) structurally identifiable, but V(0)=0 and det I(0)=0. Consequently, a zero eigenvalue of the FIM does not demonstrate structural unidentifiability in nonlinear models; the review's diagnostic can reject identifiable parameters whose sensitivity vanishes at a point. This is the most load-bearing technical weakness because the FIM is the primary local check recommended in the paper, and the 'if and only if' converts a sufficient condition into a criterion with false negatives. The acknowledged model-discrepancy limitation in the Conclusions is real but explicitly scoped and partially mitigated by the recommendation of predictive validation; the FIM statement, by contrast, is an unqualified mathematical overstatement in the main line of the argument.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a review/tutorial, not a new methods paper. It argues that structural and practical identifiability should be checked before fitting systems biology models, and that Fisher information matrix (FIM), profile likelihood, and synthetic-data re-inference checks can reveal when weakly identifiable parameters may undermine out-of-sample predictions. It defines identifiability concepts, develops linear-model intuition, then treats nonlinear models, global sensitivity analysis, emulators, and strategies for improving identifiability (experimental design, model reduction, priors). The Conclusions acknowledge that model discrepancy/misspecification is outside scope and recommend predictive validation on unseen data.","tokens_in":12298,"tokens_out":7805,"duration_ms":79104,"significance":"As a review, the paper's value is pedagogical synthesis: clear diagrammatic classification of identifiability concepts, an accessible linear-model derivation of the FIM, practical guidance in choosing methods by model complexity, and honest treatment of limitations. It cites recent software and examples from the authors' own cardiac modeling work. It contains no new theorems or code; its central recommendation aligns with the broad identifiability literature. If the technical overstatement described below is fixed, the paper would be a useful introduction for applied researchers.","major_comments":[{"comment":"The sentence 'A model is locally structurally identifiable at θ if and only if det I(θ) ≠ 0' is not true for nonlinear maps. Counterexample: scalar model f(θ)=θ^3 at θ*=0. The map is globally injective, hence locally structurally identifiable, yet V(0)=0 and det I(0)=0. Therefore a zero eigenvalue of the FIM is not proof of structural unidentifiability; it can occur at identifiable parameters where the local sensitivity vanishes, with estimation then converging at a nonstandard rate. Since this criterion feeds directly into the recommended local FIM check, please replace the iff by a one-way implication (full rank of V(θ) is sufficient for local structural identifiability, not necessary), or restrict the iff to the linear case.","section":"From linear to nonlinear"},{"comment":"Eq. (5) is used to justify the FIM as a measure of practical identifiability and to state that no zero eigenvalues implies local structural identifiability. The asymptotic distribution in Eq. (5) requires, among other conditions, that V(θ*) has full column rank and that the model is correctly specified. The θ^3 counterexample shows that when V is rank-deficient at θ*, the least-squares estimator is not asymptotically normal with covariance I(θ*)^(-1); it has a cube-root-rate limit. The text should state these regularity conditions and should not claim that a full-rank FIM means parameters 'can hence be learned from y' without finite-sample caveats about the correctly-specified, large-n regime.","section":"Practical identifiability"}],"minor_comments":[{"comment":"The figure contains what appear to be embedded author/editor annotation notes ('Gary: are emulators commonly differentiable for FIM?', 'Right click objects and then Edit points...', 'Spare shape!'). This is not suitable for publication and must be removed or replaced with a proper caption.","section":"Figure 4"},{"comment":"The caveat 'Beyond the scope of this review is model discrepancy/misspecification...' is appropriately explicit, but it is introduced only at the end. Since all the reviewed diagnostics (FIM, profile likelihood, synthetic-data checks) assume correct model specification, it would help readers to signal this limitation earlier, e.g. when practical identifiability is first defined. As written, the caveat is sensible and does not weaken the main recommendation provided the FIM issue above is corrected.","section":"Conclusions"},{"comment":"Small presentation issues: 'anemulator' should be 'an emulator'; 'Strike-goldd' appears with inconsistent capitalization; the label 'Practical GlobalUnidentifiability(everywhere)' in Figure 1 lacks spacing. These are cosmetic but should be cleaned.","section":"General"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good review, readable synthesis. The central claim — check identifiability before fitting — is standard but well argued. The paper earns its keep with the linear-model intuition, the implication diagram, and the practical rundown of methods (FIM, profile likelihood, Sobol indices, synthetic-data checks). Fig 4's cost ranking is a nice addition, though it needs tidying.\n\nThe soft spot the stress test flagged is real: \"A model is locally structurally identifiable at θ if and only if det I(θ) ≠ 0\" is false for nonlinear models. f(θ)=θ^3 at θ=0 is injective but has zero derivative, so det I=0. Full rank is sufficient, not necessary. The paper should say so and warn that a singular FIM does not prove structural unidentifiability in the nonlinear case. This is a local fix, but important because the FIM is the primary check recommended.\n\nI agree with the reader's overall assessment: no new theorem or data, but a solid consolidation. The model-discrepancy limitation is acknowledged in the conclusions; the reader's worry about it is mitigated by that discussion and by the recommendation of predictive validation.\n\nRecommendation: Yes, send to peer review. The mathematical overstatement will be caught and fixed; the rest is useful for practitioners.","headline":"Useful review of identifiability methods for systems biology, with one false mathematical equivalence (FIM rank vs local structural identifiability) that needs correcting.","tokens_in":12706,"tokens_out":1783,"would_cite":true,"duration_ms":17935,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["92C42","62-02","62F12"],"pacs":[],"model":"deepseek-v4-flash","headline":"Before fitting a systems biology model, check whether its parameters can be learned from the data at all—this review argues identifiability, not just fit quality, sets the limits of prediction.","keywords":["identifiability","practical identifiability","structural identifiability","Fisher information matrix","profile likelihood","sensitivity analysis","systems biology","parameter estimation"],"falsifier":"Take a set of systems-biology ODE models, fit each to synthetic data generated from the same model, and record out-of-sample prediction error on unseen experimental conditions. If models with near-zero Fisher information eigenvalues or flat profile likelihoods regularly deliver predictions as accurate as those of identifiable models, the central warning would be contradicted. Conversely, a single well-documented case where an apparently well-identified model fails dramatically on unseen data due to parameter combinations that were unidentifiable in the training design would support it.","tokens_in":11911,"feed_emoji":"🧬","tokens_out":4354,"duration_ms":41122,"temperature":0.7,"pith_summary":"This review argues that before fitting a systems biology model you must ask whether its parameters can be learned from the data at all, and with what precision. It distinguishes structural identifiability—learnable given infinite perfect data—from practical identifiability—learnable with acceptable precision from finite noisy data—and shows how the Fisher information matrix, profile likelihood, and synthetic-data re-inference expose both. The central warning is that weakly identifiable parameters can quietly produce good fits yet break predictions outside the calibration setting. A sympathetic reader is meant to leave with a workflow: check identifiability, then decide whether to improve the experiment, simplify the model, or add prior information. The paper makes no claim about model misspecification, which it explicitly leaves out of scope.","feed_headline":"Identifiability checks decide which biology models you can trust","feed_subtitle":"Structural and practical identifiability gate whether a fitted model will predict beyond its training data.","key_machinery":"The carrying object is the Fisher information matrix I(θ) = σ⁻²V(θ)ᵀV(θ), built from the sensitivity matrix V = dE[y]/dθ. In the linear case I = σ⁻²XᵀX is constant; its rank decides structural identifiability and its eigenvalues and eigenvectors give the directions and sizes of parameter uncertainty through the asymptotic distribution θ̂ ≈ N(θ*, I(θ*)⁻¹). The same object transfers to nonlinear models as a local approximation, and profile likelihood plus synthetic-data re-inference extend the check beyond the local quadratic picture.","core_discovery":"The paper's central claim is that identifiability is not a technical afterthought but the property that determines the limits of inference and prediction. For a parametric model f(t, θ), identifiability is injectivity of the map θ → Pθ: different parameter values must give different distributions for the observed data. Structural identifiability asks whether θ could be recovered from infinite, noise-free data; practical identifiability asks whether finite, noisy data constrain θ enough for the intended use. In linear models the Fisher information matrix I = σ⁻²XᵀX completely answers this: rank deficiency means structural unidentifiability, and small eigenvalues mean poor practical identifiab","pith_inferences":["The same logic points toward automated 'identifiability gates' in model-fitting pipelines: before any parameter estimate is reported, a synthetic-data re-inference or profile-likelihood check could be run, and models failing it flagged rather than released.","Because the paper excludes model misspecification, its recommendations are most safely read as applying to models whose form is trusted; for real biological systems, predictive validation against unseen data remains the only complete check.","A testable extension: for a class of ODE models, rank models by minimum Fisher information eigenvalue and measure out-of-sample prediction error; if the ranking holds, identifiability checks double as model-selection scores.","For expensive simulators, emulator-based Fisher information matrices could screen parameters before committing to full Markov chain or profile-likelihood analysis."],"forward_implications":["A full-rank Fisher information matrix is necessary but not sufficient for trustworthy prediction; the experimental design—time points, measured outputs, repeated measurements—determines how much information the data carry.","Weakly identifiable parameters can be addressed either by measuring different outputs or refining the model structure, or by adding prior knowledge and regularization.","Nonlinearity means identifiability must be assessed across parameter space, not once at a single estimate.","Models that pass identifiability checks in the training context can still fail at prediction time, so the quantity of interest should be uncertainty in the predicted output, not just uncertainty in parameters."],"supporting_citations":[{"why":"Supplies the nonlinear regression geometry and Fisher-information-based confidence regions used throughout the review.","marker":"Bates and Watts (1988)"},{"why":"Provides the asymptotic distribution of the least-squares estimator that justifies the key approximation θ̂ ≈ N(θ*, I(θ*)⁻¹).","marker":"Seber and Wild (1989)"},{"why":"Gives the Taylor series approach to structural identifiability, one of the foundational methods the review surveys.","marker":"Pohjanpalo (1978)"},{"why":"Establishes differential algebra techniques for structural identifiability, used for nonlinear systems.","marker":"Ljung and Glad (1994)"},{"why":"Introduces the sloppiness concept—eigenvalues of the Fisher information approximately linearly spaced on a log scale—that motivates practical identifiability checks.","marker":"Gutenkunst et al. (2007)"},{"why":"Provides profile likelihood methodology for systems biology, the review's main global practical-identifiability tool.","marker":"Kreutz et al. (2013)"},{"why":"Compares approaches for parameter identifiability analysis of biological systems, supporting the review's method recommendations.","marker":"Raue et al. (2014)"},{"why":"Shows how parameters unidentifiable and unimportant during calibration can become critical at prediction time in cardiac electrophysiology models.","marker":"Whittaker et al. (2020)"},{"why":"Provides the method for computing profile-likelihood-based confidence intervals that underlies profile identifiability checks.","marker":"Venzon and Moolgavkar (1988)"}],"fun_headline_variants":["Identifiability: the real test of model trust","Can you trust that model? Check identifiability","Before fitting, ask: are parameters identifiable?","Weak identifiability breaks predictions beyond data","Identifiability limits what models can predict"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The entire toolkit assumes the model is correctly specified: that some true parameter value within the model class can generate the data up to noise. If the model is misspecified, identifiability checks overstate the fidelity of calibration, and parameter estimates are biased by compromise fits.","fun_headline_variants_meta":{"raw":{"variants":["Identifiability: the real test of model trust","Can you trust that model? Check identifiability","Before fitting, ask: are parameters identifiable?","Weak identifiability breaks predictions beyond data","Identifiability limits what models can predict"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1319,"prompt_tokens":682,"completion_tokens":637,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":426,"completion_tokens_details":{"reasoning_tokens":580}},"tokens_in":426,"tokens_out":637,"duration_ms":6397,"temperature":1.0,"reasoning_tokens":580,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:08:17.192199+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of systems-biology ODE models, fit each to synthetic data generated from the same model, and record out-of-sample prediction error on unseen experimental conditions. If models with near-zero Fisher information eigenvalues or flat profile likelihoods regularly deliver predictions as accurate as those of identifiable models, the central warning would be contradicted. Conversely, a single well-documented case where an apparently well-identified model fails dramatically on unseen data due to parameter combinations that were unidentifiable in the training design would support it.","supporting_citations":[],"review_version":1}