BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models

Haoran Xu; Jiaxin Han; Sirui Ding; Xuan Wang; Yi Fang; Yizhi Wang; Yue Wang

REVIEW 3 major objections 6 minor 33 references

BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models

T0 review · 3 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that automated architecture search can discover biological foundation-model backbones that are 25x smaller and still outperform human-designed architectures on DNA tasks, with hybrid Hyena-Transformer-CNN topologies as the

desk verdict A useful NAS testbed and design principles for bio foundation models, but the headline SOTA gains need a clean validation/test split in the ranking protocol before they can be believed. read the letter →

arxiv 2512.00283 v3 pith:FL3BQX27 submitted 2025-11-29 cs.LG cs.AIq-bio.QM

Yi Fang , Haoran Xu , Jiaxin Han , Sirui Ding , Yizhi Wang , Yue Wang , Xuan Wang This is my paper

classification cs.LGcs.AIq-bio.QM

keywords neuralarchitecturesearchfoundationmodelsDNAlanguageproteinhybridarchitecturessupernetpredictiontokenization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BioArc is a neural-architecture-search framework for biological sequence models. The paper tries to establish that instead of importing NLP architectures like Transformers wholesale, one can automatically search over combinations of CNN, LSTM, Mamba, Hyena, and Transformer blocks and find models that outperform the largest pretrained DNA and protein foundation models at a fraction of the size. Its headline evidence is that a BioArc-found backbone, pretrained on 1/10 of the steps and with 1/20 of the parameters, beats DNABERT-2, Nucleotide Transformer, and VQDNA on the GUE benchmark. The paper also reports a recurring design pattern—Hyena blocks up front, Transformers in the middle, CNNs at the end—and argues that architecture, tokenization, and pretraining strategy are deeply entangled and must be co-optimized. The practical payoff, if right, is a principled way to generate task-specific and foundation backbones for biology instead of relying on human intuition.

What carries the argument

The central object is the supernet: a single weight-sharing network whose weights are trained by randomly sampling one 'path'—a sequence of block types and hidden dimensions—per forward pass. Ranking uses Eq. (8), a Z-score aggregate of task performances. The discovered load-bearing pattern is a hybrid topology of the form Hyena → Transformer → CNN, where Hyena reads long-range context, Transformers model contextual syntax, and CNNs extract local motif vocabulary. The architecture predictors (GNN regressor, LLM+RAG, multi-agent system) are secondary machinery for exploiting the search results.

What would settle it

Check whether the GUE and PEER test-set labels were accessible during the Eq. (8) ranking step. If they were, rerun the full search on a strictly disjoint validation split and measure whether the top-ranked BioArc architectures still beat DNABERT-2, VQDNA, and ESM-2 at 25x fewer parameters. A simpler proxy: compare BioArc's from-scratch accuracy on CPD-notata with a random architecture of the same size; if random topologies match it, the search is not the cause.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is empirical: a one-shot NAS search over 360 representative module paths—built from CNN, Hyena, LSTM, Mamba, and Transformer blocks—yields architectures that beat far larger pretrained models on DNA tasks, with the best foundation backbone reaching about 82-85% accuracy on transcription-factor and promoter tasks while DNABERT-2 and VQDNA sit in the 66-78% range. The same search on proteins shows a split: BioArc architectures win on sequence-level tasks (solubility, human PPI) but lose on structure-heavy tasks (fold classification, PPI affinity), attributed to limited pretraining data and missing rotary positional embeddings. The paper further c

Load-bearing premise

The load-bearing premise is that the architecture ranking used to select the top models was computed on a validation split that never touched the test labels reported in Tables 2 and 3; the paper does not state this split, and if test information leaked into selection the headline comparisons would be selection artifacts rather than intrinsic architecture advantages.

Editorial extensions

If this is right

If the BioArc-F result generalizes, future biological foundation models can be built at 1/20 the parameter count and 1/10 the training steps while matching or beating current large pretrained models on DNA benchmarks.
The reported pattern implies that hybrid topologies, not any single module class, are the right template for DNA sequence modeling: start with long-range context, then attention, then convolutional feature extraction.
The tokenizer finding implies that architecture and tokenization must be selected together; a tokenizer that is optimal for a Transformer is not optimal for a CNN or Mamba.
The training-strategy finding implies that large-scale pretraining is not automatically beneficial for biological tasks; from-scratch training wins on many tasks, so pretraining decisions need task-specific justification.
The architecture-prediction result implies that once an (architecture, task, performance) database exists, optimal architectures for new but similar biological tasks can be retrieved rather than searched from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the ranking-split issue is resolved cleanly, the strongest consequence is that the same Z-score ranking protocol could be applied to other data modalities, such as RNA and single-cell, which the paper names but does not demonstrate.
The Hyena-Transformer-CNN recipe may be read as an inductive-bias claim about biological sequences: long-range dependencies first, contextual syntax second, local motif extraction last. This is a testable design principle for future models even outside the NAS loop.
A practical extension implied by the predictor results: a small multi-agent LLM system could cut the cost of NAS for a new biological task by retrieving good architectures from a library of past searches, but the reported hit rate of about 0.5 at @5 suggests the method still needs substantial improvement before it can replace search.
Cross-modal transfer is the open frontier: the paper suggests the framework generalizes beyond DNA and protein, but its own results show that optimal architectures differ by modality, so a universal biological architecture is unlikely.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

BioArc is a substantive empirical contribution: a large NAS testbed for DNA and protein foundation models, 360 architectures evaluated, and a clear set of design principles (hybrid Hyena→Transformer→CNN topologies, tokenizer and training strategy interplay). The agent-based architecture prediction on held-out tasks is a nice extension. But the load-bearing headline—that these architectures outperform much larger pretrained models, including with zero pretraining—rests on an architecture-ranking protocol that never states a validation/test split. Section 3.4 ranks paths by the mean Z-score over 'downstream tasks T' and selects the top candidates. Nothing in the main text or appendix says that T is a validation set disjoint from the test labels reported in Tables 2 and 3. If the same test labels were used to select the architectures, the only-ft and mask-ft rows are selection artifacts, and the '25x smaller' claims are not intrinsic. That doesn't invalidate the search space or the empirical design principles, but it means the central RQ1/RQ3 comparisons are unsupported until the authors clarify the split.

What the paper does well: the scale and breadth of the evaluation, the layer-wise contribution analysis, the Spearman 0.817 correlation between supernet-initialized and from-scratch rankings (a good proxy check, though it doesn't address split hygiene), and the hybrid-vs-single-module comparison. The honest reporting that pretraining does not guarantee gains is a useful negative result. The protein structural-task gap is explained by pretraining scale and absolute positional encoding—that analysis seems fair.

Soft spots: besides the split issue, Tables 2-3 are single-point metrics with no error bars or significance tests, and the only-ft wins are surprising enough that they need seeds/CIs. The code is partial and anonymized, which limits reproducibility. The architecture prediction ground truth uses the same ranking, so it inherits the same concern, though the transfer setting is less affected.

This paper deserves a serious referee. If the authors can show the ranking used only validation labels, and add robustness checks, it becomes a genuinely useful resource for biological model design. I'd bring it to reading group and cite the search space and design principles, but I wouldn't cite the SOTA numbers.

Referee Report

3 major / 6 minor

Summary. The paper introduces BioArc, a Neural Architecture Search framework for biological sequence models. It defines a search space over CNN, LSTM, Mamba, Hyena, and Transformer blocks with varying depth and width, prunes the space from 67M combinations to 360 representative paths, trains a weight-sharing supernet under masked modeling, contrastive learning, and next-token prediction, and then evaluates each path either from scratch or from supernet-initialized weights on 12 DNA tasks (GUE) and 6 protein tasks (PEER). Architectures are ranked by a z-score-aggregated score (Eq. 8), and the top performers are reported as task-specific and foundation-model backbones. The paper additionally proposes three architecture-prediction methods — a GNN predictor, LLM+RAG, and a multi-agent LLM system — evaluated in supervised and transfer settings. The central claims are that BioArc-discovered architectures outperform much larger pretrained biological models, that optimal architectures show a Hyena-to-Transformer-to-CNN pattern, and that a BioArc-foundation model achieves state-of-the-art results with roughly 1/20 the model size and 1/10 the training steps of existing DNA foundation models.

Significance. If the central claims hold, BioArc would be a substantial contribution: it provides a systematic, automated alternative to intuition-driven architecture design in biology, yields concrete design principles (Hyena->Transformer->CNN), and proposes methods to predict architectures for new tasks. The paper has real strengths: a broad and carefully pruned search space, three pretraining objectives, a large set of downstream tasks, internal controls such as the Spearman rank correlation (rho=0.817, Appendix A.6.13) between supernet-initialized and from-scratch rankings, layer-wise interpretability analysis (Appendix A.7), and a held-out transfer setting for architecture prediction. However, the empirical claims of state-of-the-art performance are currently undermined by a load-bearing methodological gap: the architecture ranking protocol in Section 3.4 never states whether the downstream tasks used for selection are disjoint from the test labels reported in Tables 2 and 3. Without a validation/test split, the headline gains — especially the zero-pretraining BIOARC(only-ft) results — could be selection artifacts rather than intrinsic architecture advantages. The main experimental tables a

major comments (3)

[Section 3.4, Eq. (8)] The architecture ranking protocol is the load-bearing step for RQ1 and RQ3, but the paper never states that the set of downstream tasks T used in Eq. (8) is a validation split disjoint from the test sets reported in Tables 2 and 3. Since those tables report performance on exactly the tasks used for ranking, the top architectures may have been selected using test labels. This is especially acute for BIOARC(only-ft), which uses zero pretraining data: with test-label-based selection, a model with no pretraining can appear to beat heavily pretrained baselines merely because it was chosen for that test set. The paper must either state the validation-split protocol explicitly or rerun the selection and evaluation with a held-out split. Without this, the claim that BioArc 'discovers optimal architectures' is unsupported.
[Tables 2 and 3] The main SOTA tables report a single number per method and task, with no error bars, no number of seeds, and no significance tests. Many differences among BioArc variants and against baselines are smaller than 1 percentage point (e.g., Table 2, TFP-0: only-ft 84.70 vs mask-ft 84.80; Table 3, Solubility: BioARC(only-ft) 73.34 vs ESM2-8m 73.48). Without uncertainty estimates, the claimed outperformance of much larger models is not quantitatively established. Relatedly, the text in Section 4.2 states that BioARC 'outperform[s] larger counterparts' on sequence-level tasks like Solubility, but Table 3 shows ESM2-8m (8M parameters) scoring 73.48 vs BioARC-only-ft 73.34; the claim is contradicted by the paper's own table. Please report mean +/- std over at least 3 seeds and qualify the claim.
[Figure 3 / RQ3] The foundation-model experiment selects the BioArc-F architecture by 'top average performance across all tasks' and then evaluates it on the same downstream tasks. This repeats the selection-on-test-set concern: the architecture is not independent of the evaluation tasks. Additionally, the comparison '1/10 training steps' is against full-pretraining baselines; to attribute the gain to the architecture, the authors should compare against a baseline architecture trained for the same 1/10 steps (or matched compute). Without this control, the claim that BioArc-F 'outperforms human-designed architectures' at reduced budget is not cleanly supported.

minor comments (6)

[Appendix A.6.1] Train/test splits are provided for the architecture-prediction experiments, but no analogous split is given for the architecture-ranking protocol of Section 3.4. This is not merely a presentation issue; it directly relates to Major Comment 1.
[Tables 2 and 3] The naming 'BIOARC(only-ft)' is confusing: it denotes training from scratch with no pretraining, not fine-tuning. Consider renaming to 'BIOARC(scratch)' for clarity.
[Section 3.4 vs Appendix A.6.13] Section 3.4 states that each path is 'optimized independently' via fine-tuning or from-scratch training, but Appendix A.6.11/Figure 12 describes evaluating paths 'directly from the pretrained supernet by freezing the backbone weights.' Please clarify which evaluation mode is used for the ranking correlation in A.6.13; if the correlation is computed with frozen weights, it does not validate the ranking used for Tables 2-3.
[Appendix A.6.9] In Figure 10, the task index labels are small and some curves are hard to distinguish. Enlarge fonts or use distinct markers.
[Appendix A.6.1] The sentence 'We use one A100 80G for the pretraining for 10 epoch' should be corrected (grammar and subject-verb agreement).
[General] The framework is sometimes spelled 'BIOARC' and sometimes 'BioArc' (e.g., Figure 1 vs Figure 3). Use one consistent spelling throughout.

Circularity Check

2 steps flagged · score 6.0 of 10

Architecture selection (Eq. 8) and the BIOARC-F backbone are chosen on the same downstream tasks later reported as gains; with no stated held-out split, the headline 'discovery' results reduce to selection on the evaluation set.

self definitional [Section 3.4 (Eq. 8) and Section 4.2 / Tables 2-3]
"We rank the sampled architectures S based on their aggregated performance across the set of downstream tasks T. ... Score(a) = 1 |T | X t∈T st · Pt(a)−µ t σt , (8) ... Finally, the top candidates A top are identified by selecting the k architectures with the highest Score(a). ... As detailed in Table 2, our models achieve superior performance across all DNA-related tasks."

The 'downstream tasks T' in Eq. 8 are the same GUE/PEER tasks reported in Tables 2-3 (Section 3.1 defines downstream evaluation as the GUE benchmark), and no validation split is stated anywhere in Section 3.4 or the appendices. Therefore A_top is selected to maximize the exact metrics later reported as 'superior performance'; the only-ft row (0 pretraining tokens) beating heavily pretrained baselines is the signature of selection on the reported labels. The discovery claim reduces to the selection rule by construction.
fitted input called prediction [Section 4.2 / Figure 3 (RQ3)]
"The architecture is selected by the top average performance across all tasks and pretrained for 1/10 training steps of baselines. ... with only 1/20 model size and 1/10 training steps, BIOARC-based foundation model outperforms the human-designed architectures on downstream DNA tasks."

The caption states the backbone was chosen by 'the top average performance across all tasks', and the comparison is then made on those same downstream DNA tasks. The 'outperforms' statement is therefore a direct consequence of the selection criterion, not an independent test of the backbone. This is the RQ3 headline claim and is circular for the same reason as Eq. 8: the architecture was fitted to the very tasks on which it is declared superior.

full rationale

The main circularity is in the architecture-ranking protocol. Eq. 8 computes Score(a) over 'the set of downstream tasks T'; Section 3.1 defines downstream evaluation as the GUE benchmark (and PEER for protein); and Tables 2-3 report performance on exactly those datasets. The paper nowhere states that Score(a) was computed on a validation split disjoint from the reported test labels. Thus, selecting A_top by Eq. 8 and then reporting that BIOARC architectures 'achieve superior performance across all DNA-related tasks' makes the headline result a selection artifact rather than an independent prediction: the top architectures were chosen to maximize the very metrics being reported. The only-ft row (0 pretraining samples) beating heavily pretrained baselines is consistent with this reading. The same issue appears in Figure 3, whose caption says the backbone was 'selected by the top average performance across all tasks' and then compared on those tasks. The held-out architecture-prediction experiments (Table 4) are less circular because they use disjoint training/test tasks, although their ground truth inherits the Section 3.4 evaluation. No load-bearing self-citation or imported uniqueness theorem is present; the NAS machinery, ablations, and ranking-correlation analysis provide independent content, but they do not fix the selection-on-evaluation-set circularity. Overall, one central construction-based circularity affecting the paper's strongest claims warrants a score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on standard NAS machinery plus several hand-set design choices. The main free parameters shape the search space and prediction pipeline. The domain assumptions are plausible but not independently established. No new physical entities are introduced.

free parameters (4)

Search depth set D = {3,4,5,6}
Chosen by hand; defines the search space, and the depth analysis in A.6.10 depends on it.
Hidden dimension set H = {64,128,256,512}
Chosen by hand; controls width diversity in the search space.
Pruning threshold τ and cluster counts k_d
Rule/distance/k-means pruning reduces 67M to 360 paths; τ and k_d are hand-set but values are not reported, so coverage guarantees are unknown.
LLM/RAG hyperparameters n, k, m
Retrieval and recommendation counts for LLM+RAG and BioArcAgent; no sensitivity analysis is provided.

assumptions (5)

domain assumption One-shot supernet weight sharing gives unbiased estimates of standalone path performance.
Invoked in Section 3.4; validated only by Spearman ρ=0.817 in A.6.13, so ranking noise remains.
ad hoc to paper The pruned 360-architecture search space contains near-optimal architectures for the benchmark tasks.
Reduction from 67M to 360 via rule/distance/cluster pruning (Section 3.2, A.3) has no coverage guarantee; if the true optimum was pruned away, all optimality claims fail.
ad hoc to paper Functionally similar biological tasks share optimal architecture topology.
Stated in Section 3.5 and used by prediction methods; empirically supported by similarity matrix in A.6.5 but not a general law.
domain assumption GUE and PEER benchmark splits and labels are clean and leakage-free.
Relied on for all RQ1-RQ3 conclusions; no evidence in the paper about label noise or split leakage.
domain assumption Hybrid local+global modules can capture the 'grammar' of biological sequences.
Motivates the search space including CNN/LSTM/Mamba/Hyena/Transformer; not proven before search.

how reviews work

0 comments

Cite this review

Pith. "Pith review of BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models." pith.science (2026). https://pith.science/paper/FL3BQX27

@misc{pith2026251200283,
  author       = {Pith},
  title        = {Pith review of: BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FL3BQX27}},
  note         = {Machine review of arXiv:2512.00283}
}

read the original abstract

Foundation models have revolutionized various fields such as natural language processing (NLP) and computer vision (CV). While efforts have been made to transfer the success of the foundation models in general AI domains to biology, existing works focus on directly adopting the existing foundation model architectures from general machine learning domains without a systematic design considering the unique physicochemical and structural properties of each biological data modality. This leads to suboptimal performance, as these repurposed architectures struggle to capture the long-range dependencies, sparse information, and complex underlying ``grammars'' inherent to biological data. To address this gap, we introduce BioArc, a novel framework designed to move beyond intuition-driven architecture design towards principled, automated architecture discovery for biological foundation models. Leveraging Neural Architecture Search (NAS), BioArc systematically explores a vast architecture design space, evaluating architectures across multiple biological modalities while rigorously analyzing the interplay between architecture, tokenization, and training strategies. This large-scale analysis identifies novel, high-performance architectures, allowing us to distill a set of empirical design principles to guide future model development. Furthermore, to make the best of this set of discovered principled architectures, we propose and compare several architecture prediction methods that effectively and efficiently predict optimal architectures for new biological tasks. Overall, our work provides a foundational resource and a principled methodology to guide the creation of the next generation of task-specific and foundation models for biology.

Figures

Figures reproduced from arXiv: 2512.00283 by the authors.

**Figure 2.** This figure illustrates the top five performing DNA model architectures (Arch 1-5), iden [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Performance of BIOARC-Discovered Architecture as a Foundation Model Backbone, noted as BIOARC-F. The architecture is selected by the top average performance across all tasks and pretrained for 1/10 training steps of baselines. performance comparable to task-specific optimization. BIOARC-based foundation model shown in [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (12 more)

**Figure 4.** Figure 4: Performance of different training strategies on DNA. In the left panel, each cell shows [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: Effect of different tokenization on various architectures training from scratch on the DNA [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: This figure illustrates the top five performing protein model architectures (Arch 1-5). [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

**Figure 7.** Figure 7: The heatmap showing the cosine similarity between different neural network architectures. [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

**Figure 8.** Figure 8: Performance of hybrid architecture compared with optimal single-module architectures [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]

**Figure 9.** Figure 9: Performance of different training strategies on Protein. In the left panel, each cell shows [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]

**Figure 10.** Figure 10: Our results highlight two key findings regarding tokenization. First, different architec [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

**Figure 10.** Figure 10: Performance of different tokenization methods on various architectures. [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]

**Figure 11.** Figure 11: Visualization of the top-performing architectures across different depths. [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]

**Figure 12.** Figure 12: Model Scaling Analysis (Performance vs. Parameters). Each point represents a ar [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]

**Figure 13.** Figure 13: Rank consistency analysis between Supernet initialization and Trained From Scratch. [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]

**Figure 14.** Figure 14: Layer-wise Visualization of Model Activations. Layer 0 (Hyena, Brown): Normbased activation density across the 70bp sequence. Layers 1-2 (Transformer, Purple/Pink): Attention accumulation profiles showing distinct spatial focusing patterns. Layers 3-5 (CNN, Blue/Oran…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 1 linked inside Pith

[1]

A knowledge base of similar tasks with their characteristics
[2]

Your goal is to:

Performance data showing which architectures work best for each task. Your goal is to:
[3]

Recommend the best architectures based on performance data
[4]

Provide clear reasoning for your predictions. Available Tasks in Knowledge Base: {retrieve datas} Performance Data: {performance data} Please analyze the query task and provide your recommendations in the following format: Task Analysis: - Dataset characteristics: [analyze the input task] - Problem type: [classification/regression/etc.] - Modality: [DNA/P...
[5]

Analyze the given task description
[6]

Find the most similar tasks from the knowledge base
[9]

BEST CHOICE: [architecture-experiment-model name] - [reasoning]
[10]

SECOND BEST: [architecture-experiment-model name] - [reasoning]

Show all 33 references

[11]

Use the exact architecture names from the performance data

THIRD BEST: [architecture-experiment-model name] - [reasoning] Reasoning: Detailed explanation of why these architectures are recommended based on similar tasks Instruction:Focus on recommending architectures that have shown good performance on similar tasks. Use the exact arc...

2026
[12]

Analyze the user’s dataset and task description
[13]

Extract and summarize the key information needed for architecture search
[14]

Identify the modality (DNA, Protein, etc.) and problem type
[15]

You should NOT: • Search for architectures yourself

Provide a clear summary for the Retriever Agent to use for searching. You should NOT: • Search for architectures yourself. • Make recommendations. • Call any search tools. Output Format (Response Template): Task Summary: - Dataset characteristics (size, type, format) - Problem...

2026
[16]

Receive a message from the Analyst containing task summary and search parameters
[17]

Extract the task description, problem type, modality, and key characteristics from the Analyst’s output
[18]

Use your LLM reasoning to identify which tasks in the knowledge base are most similar to the query task
[19]

Consider multiple dimensions of similarity: • Problem type (classification, regression, clustering, etc.) • Modality (DNA, Protein, text, image, etc.) • Dataset characteristics (size, complexity, format) • Task objectives and constraints • Domain-specific requirements
[20]

Select the top{top k}most relevant tasks based on your understanding
[21]

You MUST: • Use your natural language understanding to identify similar tasks

Format the retrieved similar tasks clearly for the Architecture Retriever Agent. You MUST: • Use your natural language understanding to identify similar tasks. •NOTsearch for architectures yourself - that’s the Architecture Retriever’s job. •NOTperform any scoring or evaluatio...
[22]

33 Published as a conference paper at ICLR 2026 A.9.3 ARCHITECTURERETRIEVERAGENT

Task Index: [Index] - Similarity Reasoning: [Your explanation of why this task is similar] - Task Description: [Description from knowledge base] - Problem Type: [Type from knowledge base] - Modality: [Modality from knowledge base] - Dataset Characteristics: [Details from knowl...

2026
[23]

2.IMMEDIATELYcall thearchitecture retrieval toolwith the Task Retriever’s output text

Receive a message from the Task Retriever containing similar tasks and their details. 2.IMMEDIATELYcall thearchitecture retrieval toolwith the Task Retriever’s output text
[24]

• Look up architectures that were successful on the identified similar tasks

The tool will automatically: • Extract task indices from the Task Retriever’s output. • Look up architectures that were successful on the identified similar tasks. • Return detailed architecture information including performance metrics. • Provide architecture descriptions, pe...
[25]

name": "tool name

Return the tool’s result directly - do not format or modify it. Critical Requirements: • YouMUSTcall the tool and return its EXECUTION RESULT. •NEVERreturn tool call parameters or JSON strings like{"name": "tool name", "parameters":{...}}. •NEVERreturn the raw tool call format...

2026
[26]

Receive a message from the Architecture Retriever Agent containing detailed information for each architecture and the overall problem type
[27]

For each architecture, youMUSTcall theevaluate architecturefunction with the architecture nameandproblem type
[28]

Consolidate the scores returned by theevaluate architecturetool
[29]

Analyze the scored architectures and select top{recommendation count}recommendations based on scores
[30]

Critical Output Format Requirements: • Architecture namesMUSTbe in the exact format

Provide detailed reasoning for each recommendation. Critical Output Format Requirements: • Architecture namesMUSTbe in the exact format... • Examples:mix-kmer1-path 54,cnn-kmer1-path 12,transformer-kmer1-path 8. • UseEXACTarchitecture names from the Architecture Retriever outp...
[31]

Best performance - Architecture: [EXACT ARCHITECTURE NAME]
[32]

Second best - Architecture: [EXACT ARCHITECTURE NAME]
[33]

Analysis Complete - Ready for implementation! Important:Always use the EXACT architecture names from the Architecture Retriever output

Third best - Architecture: [EXACT ARCHITECTURE NAME] (Continue for all{recommendation count}recommendations if more than 3) Detailed Analysis Architecture: [EXACT ARCHITECTURE NAME] - Justification: [Justification from evaluate architecture tool] (Continue for all architecture...
[1024]

We extend the training steps to 100,000 and keep other hyperparameters the same. As shown in Table 10, this scaling yields a improvement on 10/12 tasks, which validates the architecture’s effectiveness for large-scale pre-training.This demonstrates that our discovered architec...

2026
[4054]

positive

doi: 10.1093/bib/bbae577. URLhttps://doi.org/10.1093/bib/bbae577. Y . Zeng, J. Xie, N. Shangguan, et al. Cellfm: a large-scale foundation model pre-trained on transcriptomics of 100 million human cells.Nat Commun, 16:4679, May 2025. doi: 10.1038/s41467-025-59926-5. Zhidie Zhan...

2025 arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

A knowledge base of similar tasks with their characteristics

[2] [2]

Your goal is to:

Performance data showing which architectures work best for each task. Your goal is to:

[3] [3]

Recommend the best architectures based on performance data

[4] [4]

Provide clear reasoning for your predictions. Available Tasks in Knowledge Base: {retrieve datas} Performance Data: {performance data} Please analyze the query task and provide your recommendations in the following format: Task Analysis: - Dataset characteristics: [analyze the input task] - Problem type: [classification/regression/etc.] - Modality: [DNA/P...

[5] [5]

Analyze the given task description

[6] [6]

Find the most similar tasks from the knowledge base

[7] [9]

BEST CHOICE: [architecture-experiment-model name] - [reasoning]

[8] [10]

SECOND BEST: [architecture-experiment-model name] - [reasoning]