{"id":"37e1928b-1e2f-4ec8-ad9c-220d372d338e","arxiv_id":"2611.17015","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"This review frames the reliability limits of biological foundation models as a knowledge boundary set by five design elements, and proposes machine-readable model-limit statements.","lead":"This review proposes that biological AI models have a knowledge boundary, the range of inputs and tasks where their outputs can be trusted, and that five design choices set that boundary. It assembles published evidence of where protein, DNA, and single-cell models succeed and fail, and argues for machine-readable model-limit statements so researchers and AI agents can decide when to rely on a model.","discovery_kind":"review","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Boundary statements record where a model was tested, not whether a new query is reliable; the paper's own Section 4.3 failures (AlphaMissense, RNA, SORT1) fall inside the declared domain, so the proposed five-element mechanism is asserted rather than shown to work.","rationale":"The reader's CONDITIONAL verdict is the right one, and my analysis does not move it. My concern is close to, but narrower than, the reader's weakest_assumption. The reader emphasized that reliability may be task- and context-dependent in ways a fixed boundary statement cannot encode. I sharpen this: even with the task held fixed, the paper's own Section 4.3 supplies documented within-domain reliability failures (AlphaMissense on curated clinical variants; RNA models on homology-reduced tests; AlphaFold2 on destabilizing point mutations), and the SORT1 case shows a confident within-domain wrong answer that a boundary statement recording training/evaluation domain would not flag. So the unverified premise is not only task-dependence but the very sufficiency of a static statement organized by the five elements to support a prospective reliability decision. This is a genuine soft spot in the central claim that boundary statements let agents and researchers rely on models, but the paper is a perspective proposing new infrastructure, the authors explicitly acknowledge the difficulty of knowing a model's boundaries (b83, b253, b326), and the gap is addressable by adding reliability-calibrating content to the statement template. That supports keeping the verdict CONDITIONAL rather than ACCEPT, and does not justify REJECT. I do not regard the inaccessible companion resource or the absence of delivered standardized descriptions as load-bearing for the scientific claim; those are secondary. I agree with the reader's overall posture and verdict, differing only in the precise location of the weak premise.","tokens_in":24403,"tokens_out":4886,"duration_ms":35976,"concrete_test":"Construct, from Section 4.3, the five-element boundary statement that Section 6.2's template would generate for each failing model, AlphaMissense on ref 118, RNA models on ref 120, AlphaFold2 on ref 100, and AlphaGenome on the SORT1 case (b230-b233). For each, determine whether the query lies inside the declared boundary, training corpus, tokenization, context length, objective, benchmark list, post-analysis. If any query falls fully inside the declared boundary yet the model output is unreliable, then a boundary statement alone cannot deliver the promised reliability decision and Section 6.2 requires an additional reliability-calibrating component (e.g., per-task error rates or calibrated confidence). If every failure query falls outside its statement's declared boundary, the framework as specified is sufficient and this concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central deliverable (Section 6.2) is a machine-readable boundary statement, organized by the five design elements, that would let an agent \"confirm that a query lies within a model's training and evaluation domain before relying on its output.\" For this to work, the statement must support a prospective reliability judgment. The paper asserts this (b369) but does not test it, and its own evidence cuts against the sufficiency claim in two ways. First, within-domain failures: Section 4.3 documents cases where models fail precisely on queries their design elements cover, AlphaMissense discordance across 5,845 clinically curated variants in 59 disease genes (ref 118), RNA language models no better than a one-hot encoder once homologous sequences are excluded (ref 120), and AlphaFold2 largely missing destabilizing point mutations (ref 100). A statement listing training corpus, tokenization, context length, objective, benchmarks, and post-analysis would not flag any of these as out-of-boundary: they sit inside the data distribution, within architecture limits, and on the models' own tasks. Second, the Paper2Agent/SORT1 example (b230-b233), the paper's own motivating case, is a within-boundary confident failure: AlphaGenome was used for its intended task (regulatory variant scoring) and produced a wrong-gene, wrong-direction output; no boundary statement would have intercepted it. Two consequences follow. (a) A statement that records the evaluation domain cannot distinguish reliable from unreliable queries inside that domain, yet that distinction is exactly the decision the agent must make. (b) The framework risks circularity: evaluation scheme is both one of the five defining elements (Section 4.1) and the instrument that reveals the boundary, so listing benchmarks \"used\" (Section 6.2) is retrospective, not predictive. The leap from \"design elements shape capabilities\" to \"a statement of them supports prospective trust\" is the unverified premise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This review-perspective proposes a framework for when biological foundation models can be trusted. It defines a model's 'knowledge boundary' as the range of inputs, tasks, and conditions over which its outputs are reliable, and proposes that five design elements—data, architecture, training scheme, evaluation scheme, and post-analysis—define this boundary (abstract; Section 2.1.2). Sections 2 and 3 introduce the design elements and the four routes by which agentic systems can incorporate foundation models. Section 4 reviews evidence, drawing heavily on independent evaluations that compare foundation models with simple baselines, to show both what models capture (residue chemistry, contact recovery, zero-shot mutational-effect likelihoods, and successful generative-design cases) and where they fail (AlphaMissense discordance with curated clinical variants, RNA language models at parity with one-hot encoders under homology filtering, perturbation and single-cell models matched by linear baselines, and evaluation artifacts). Section 5 assesses gaps in evaluation practice and the limits of the language-model scaling analogy; Section 6 proposes machine-readable boundary statements organized by the five design elements, a portfolio of specialized models coordinated by agents, and community infrastructure. The paper is explicitly a perspective (b369), and its empirical statements are attributed to cited published or preprint sources.","tokens_in":24931,"tokens_out":14296,"duration_ms":89100,"significance":"The review is timely and useful as an organizing map of a fast-moving field. Its principal strength is that it does not rest on developer-reported results: it foregrounds independent reassessments and simple-baseline comparisons (e.g., refs 100, 102–117, 118–120) that cut against model claims, and it is candid about the field's early stage and about circularity risks in LLM-based annotation (b288). The five-element taxonomy is a clear pedagogical device, and the proposal to attach knowledge-boundary statements to model releases (Section 6.2) addresses a real operational need for both human users and agents. If the sufficiency claim were validated, the paper would give the field a standard vocabulary for model limits. However, the central claim is asserted rather than demonstrated, and the paper's own Section 4.3 evidence includes failures inside the declared domain that a static five-element statement would not flag; the significance is therefore conditional on the authors reframing or defending the sufficiency claim.","major_comments":[{"comment":"The stress-test concern about the sufficiency of the five design elements lands. The paper's central claim—that the five elements define the knowledge boundary—is operationalized in Section 6.2 as a machine-readable statement that would let an agent 'confirm that a query lies within a model's training and evaluation domain before relying on its output.' But Section 4.3 documents failures on queries that are inside the declared domain: AlphaMissense discordance across 5,845 clinically curated variants in 59 disease genes (ref 118); RNA language models that fail to beat a one-hot encoder for secondary-structure prediction once homologous sequences are excluded (ref 120); and AlphaFold2's failure to detect structural disruption from destabilizing point mutations (ref 100). The paper's own motivating case, Paper2Agent using AlphaGenome for regulatory variant scoring at 1p13 (b230–b233), produced a wrong-gene ranking and a wrong-direction expression claim while operating on the model's intended task. A statement listing training corpus, tokenization, context length, training objective, benchmarks, and post-analysis methods would not have marked any of these queries as out-of-boundary. The paper partially acknowledges the gap ('Benchmark performance, confidence scores, transferable embeddings, and reliability in application must therefore be established separately,' end of Section 4.3; b326), but Section 6.2 does not incorporate these caveats into the proposed mechanism. The authors should either weaken the claim from 'define' to 'organize and partially determine,' explicitly distinguishing the declared domain from the verified reliable range, or specify the additional components (evaluation-coverage records, out-of-distribution or calibration signals, explicit failure-mode lists) that would make the statement support prospective reliability judgments, and demonstrate on at least one documented case what the statement would have flagged.","section":"Abstract; §2.1.2; §4.3; §6.2","authors_acknowledge":"Few models report whether they can signal that a query lies outside their boundary.","authors_acknowledge_p":0.63},{"comment":"The paper presents BioReason's results as evidence for the general principle that 'when [the answer] lies beyond [the LLM's textual knowledge], the encoder sets the ceiling of knowledge' (b227), citing the improvement from 49% to 80% accuracy after adding Evo 2 (b226). Section 5.1, however, cites ref 123, which found that BioReason and BioReason-Pro were as accurate with shuffled biological inputs as with intact ones. The paper should reconcile these two statements: either qualify the Section 3 claim as task-dependent, or explain why the shuffled-input control does not undermine the attribution of the gain to the encoder. As written, the paper's most concrete example of an agent inheriting a foundation model's boundary is in tension with a control study it cites.","section":"§3 (b225–b227) vs §5.1 (ref 123)","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The text 'runs locally, or at remove facilities without internet access' contains a typo: 'remove' should read 'remote.'","section":"§3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The online resource URL appears as the placeholder '[URL]' in both Section 6.2 and the Data and Code Availability section; this should be resolved before publication.","section":"§6.2; Data and Code Availability","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Several reference entries lack identifiers (e.g., ref 77, the Anthropic Engineering Blog post; ref 84, the Biomni-R0 technical report; ref 89, the OpenAI mathematics statement), which will complicate verification; the reference list should be brought to journal style.","section":"References (77, 84, 89)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The relationship between 'training and evaluation domain' and 'knowledge boundary' should be defined explicitly, since the abstract defines the boundary as the range over which outputs are reliable whereas Section 6.2 uses 'domain' to mean what the model was exposed to; these are not the same set, as the Section 4.3 evidence shows.","section":"Abstract; §6.2","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Worth reading, and worth sending to referees. The paper's real asset is its evidence: it collects the independent baseline studies that cut against biological foundation models, including ProteinGym's moderate unsupervised performance, linear models matching perturbation predictors, parameter-free single-cell representations, AlphaMissense discordance on clinical variants, RNA language models no better than one-hot once homologs are excluded, and AlphaFold2 missing destabilizing mutations. It also includes the SORT1/Paper2Agent case, where an agent used AlphaGenome for its intended task and reached the right gene only by contradicting both the tool's output and the experimental literature. Few reviews in this space are this honest. What is actually new is smaller. The five design elements are a reasonable organizing template, and knowledge boundary is a useful label, but the components are known and the authors explicitly build on model cards and datasheets. There is no new data, no systematic search, and no delivered specification. The central soft spot is the leap from design elements to prospective trust. Section 6.2 proposes machine-readable boundary statements that would let an agent confirm a query lies within a model's training and evaluation domain. But Section 4.3's failures are inside that domain. AlphaMissense failed on curated clinical variants in disease genes; the RNA models failed once homologous sequences were removed; AlphaFold2 largely missed destabilizing point mutations. A statement recording corpus, architecture, objective, benchmarks, and post-analysis would not flag any of those queries as out-of-boundary. The SORT1 example is the cleanest version: the model was used on its own task and gave a wrong-gene, wrong-direction answer. I checked the stress-test concern and it lands. A statement of where a model was tested is not a certificate of reliability for a new query, and listing evaluation benchmarks is retrospective. The paper frames this as a perspective and acknowledges that few models can signal out-of-boundary queries, but the proposed deliverable needs either a concrete schema plus empirical test, or a more modest claim. Minor issues: no actual boundary-statement template or worked example is given, the companion resource URL is a placeholder, and the literature selection is not systematic, though it looks balanced. This deserves a serious referee. Send it out, and ask referees to focus on whether boundary statements can support prospective decisions and to require at least one worked example. For anyone working in this area, it is worth citing for its landscape and its baseline evidence.","headline":"A genuinely useful, balanced review of where biological foundation models fail, wrapped around a five-element framework that is asserted rather than demonstrated, and whose proposed boundary statements would not have caught the paper's own failure examples.","tokens_in":26163,"tokens_out":3427,"duration_ms":55807},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Five design choices define when biological AI models can be trusted.","keywords":["foundation models","agentic systems","knowledge boundary","protein language models","single-cell foundation models","benchmarking","model cards","transfer learning"],"falsifier":"Run a routing experiment in which an agent receives machine-readable boundary statements for a portfolio of models and must choose which model to call on challenging queries, compared with an agent that receives only model names; if statement-aided routing is no more accurate at choosing the reliable model, the claim that knowledge boundaries can be made explicit for agents fails.","tokens_in":26001,"feed_emoji":"🧬","tokens_out":5781,"duration_ms":55472,"temperature":0.7,"pith_summary":"This review argues that a biological foundation model's reliability is not open-ended but is set by five choices made when the model is built: data, architecture, training scheme, evaluation scheme, and post-analysis. The authors call the resulting range of inputs, tasks, and conditions the model's knowledge boundary, and they propose that every model be released with a structured, machine-readable boundary statement organized by those five elements. The motivation is that agentic systems increasingly call foundation models as tools and inherit whatever boundary the foundation model has, so errors from outside that boundary can look as authoritative as well-supported results. The review supports the framework with published evidence on where models succeed and where simple baselines match or beat them, concluding that making boundaries explicit is what would let biologists and agents use these tools reliably.","feed_headline":"Five design choices define when biological AI models can be trusted","feed_subtitle":"A new review says data, architecture, training, evaluation, and post-analysis set each model's range of reliable use.","key_machinery":"The organizing object is the knowledge boundary, the range of inputs, tasks, and conditions over which a model's outputs are reliable, and the carrying mechanism is a five-element template used to decompose that boundary: data, architecture, training scheme, evaluation scheme, and post-analysis. Each element is tied to a specific kind of failure: data limits what can be known, architecture limits what relationships can be captured, the training objective determines what is rewarded, evaluation determines which weaknesses become visible, and post-analysis determines what knowledge becomes accessible. The template does the argument's work by turning an abstract trust question into a concrete checklist, by tracing observed failures back to design choices, and by providing the schema for the proposed machine-readable boundary statements that agents could consult before calling a model.","core_discovery":"The paper's central claim is that a biological foundation model's knowledge boundary, defined as the range of inputs, tasks, and conditions over which its outputs can be relied upon, is set by five design elements: the data and tokenization, the model architecture, the training scheme, the evaluation scheme, and the post-analysis methods such as probing, fine-tuning, zero-shot scoring, and interpretability. The review assembles two bodies of evidence for this claim: studies showing that self-supervised models capture real biological structure, from residue chemistry and contacts to evolutionary constraint and functional design, and independent studies showing that the same models stop being reliable where simple linear models, mean responses, or one-hot encodings match or beat them, especially for perturbation prediction, single-cell tasks, and mutation-dense design. From this, the authors conclude that integrating foundation models with agentic systems depends on making the knowledge boundary explicit through structured boundary statements that record each model's five elements in a machine-readable form, in the spirit of model cards and datasheets.","pith_inferences":["Editorial inference: the same five-element template could be applied to the large language models inside agentic systems themselves, since the planner has its own training data, architecture, objective, evaluation, and post-processing boundaries.","Editorial inference: if boundary statements became the norm, the unit of scientific reproducibility could shift from the model weights alone to the model-plus-boundary-statement package, making negative results and attribution of failures easier to record.","Editorial inference: a directly testable extension is a routing benchmark in which agents receive machine-readable boundary statements for a portfolio of models and are scored on choosing the right model for out-of-distribution queries; the paper argues for such routing but does not supply the experiment.","Editorial inference: the reviewed evidence implies that publishing only average benchmark scores hides most failures, and that reporting failure distributions conditioned on sequence similarity and task type would be a cheap near-term implementation of the proposal."],"forward_implications":["If boundary statements become standard, an agent can route each question to the model whose stated boundary covers it and fall back to a simple baseline when no model applies.","Benchmark results would be reported together with simple baselines and train-test similarity controls, making it harder for a high score to reflect recall of near-duplicate training sequences rather than learned rules.","The field would have a principled reason to develop specialized models for boundary regions where general models fail, rather than waiting for one ever-larger general-purpose model.","New data collection, including self-driving laboratory experiments, could be targeted directly at the regions a boundary statement shows are missing.","A correct final answer from an agent would no longer be treated as evidence that the underlying model reasoning was sound, because the boundary statement would say which post-analysis conditions the result depends on."],"supporting_citations":[{"why":"Supplies the multi-assay benchmark showing unsupervised fitness predictions track measured effects only moderately, with no single model best across proteins.","marker":"[44]"},{"why":"Provide the independent perturbation-evaluation results showing linear baselines and mean responses match or beat foundation models.","marker":"[105-107]"},{"why":"Supports the claim that protein-model likelihoods partly reflect how often sequences were sampled across the tree of life rather than fitness alone.","marker":"[32]"},{"why":"Provides the single-cell transfer-learning example in which an embedding-based in-silico perturbation was followed up experimentally, supporting the claim that captured knowledge can be read out and used.","marker":"[9]"},{"why":"Supplies the counterexample in which a structure-prediction approach that relies on related sequences fails to predict the effect of destabilizing point mutations.","marker":"[100]"},{"why":"Supply the model-card and datasheet tradition that the proposed structured boundary statements extend.","marker":"[128-129]"},{"why":"Provides the language-model scaling-law comparison used to argue that biology may not follow the same scaling trajectory.","marker":"[126]"},{"why":"Defines the foundation-model reuse premise whose limits the knowledge-boundary framework is built to address.","marker":"[22]"}],"fun_headline_variants":["Five design elements define when biological AI outputs are reliable","A bio AI model's knowledge boundary comes down to five choices","Five design elements mark where biological AI models can be relied on","Knowledge boundary: five design elements make bio AI reliable or not"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The framework assumes that a model's reliability can be captured by a fixed description of its five design elements that holds across tasks, users, and queries, and the paper asserts this rather than testing it.","fun_headline_variants_meta":{"raw":{"variants":["Five design elements define when biological AI outputs are reliable","A bio AI model's knowledge boundary comes down to five choices","Five design elements mark where biological AI models can be relied on","Knowledge boundary: five design elements make bio AI reliable or not"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"cost":0.0002137226,"is_byok":false,"cost_details":{"upstream_inference_cost":0.0002137226,"upstream_inference_prompt_cost":2.88106e-05,"upstream_inference_completions_cost":0.000184912},"total_tokens":2003,"prompt_tokens":987,"completion_tokens":1016,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":838,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":958}},"tokens_in":987,"tokens_out":1016,"cost_source":"openrouter_usage","duration_ms":8584,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":958,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T07:22:01.669811+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Run a routing experiment in which an agent receives machine-readable boundary statements for a portfolio of models and must choose which model to call on challenging queries, compared with an agent that receives only model names; if statement-aided routing is no more accurate at choosing the reliable model, the claim that knowledge boundaries can be made explicit for agents fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the multi-assay benchmark showing unsupervised fitness predictions track measured effects only moderately, with no single model best across proteins."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the claim that protein-model likelihoods partly reflect how often sequences were sampled across the tree of life rather than fitness alone."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the single-cell transfer-learning example in which an embedding-based in-silico perturbation was followed up experimentally, supporting the claim that captured knowledge can be read out and used."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the counterexample in which a structure-prediction approach that relies on related sequences fails to predict the effect of destabilizing point mutations."},{"cited_title":"ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models","cited_arxiv_id":"2001.08361","evidence_quote":"Provides the language-model scaling-law comparison used to argue that biology may not follow the same scaling trajectory."},{"cited_title":"Do ‘Dominant Frequencies’ explain the listener’s response to formant and spectrum shape variations?","cited_arxiv_id":"2108.07258","evidence_quote":"Defines the foundation-model reuse premise whose limits the knowledge-boundary framework is built to address."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}