REVIEW 9 cited by
Archon: An Architecture Search Framework for Inference-Time Techniques
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Archon: An Architecture Search Framework for Inference-Time Techniques
read the original abstract
Inference-time techniques, such as repeated sampling or iterative revisions, are emerging as powerful ways to enhance large-language models (LLMs) at test time. However, best practices for developing systems that combine these techniques remain underdeveloped due to our limited understanding of the utility of each technique across models and tasks, the interactions between them, and the massive search space for combining them. To address these challenges, we introduce Archon, a modular and automated framework for optimizing the process of selecting and combining inference-time techniques and LLMs. Given a compute budget and a set of available LLMs, Archon explores a large design space to discover optimized configurations tailored to target benchmarks. It can design custom or general-purpose architectures that advance the Pareto frontier of accuracy vs. maximum token budget compared to top-performing baselines. Across instruction-following, reasoning, and coding tasks, we show that Archon can leverage additional inference compute budget to design systems that outperform frontier models such as OpenAI's o1, GPT-4o, and Claude 3.5 Sonnet by an average of 15.1%.
Forward citations
Cited by 9 Pith papers
-
AoA: Theorem Proving Agent over Abstract Syntax Tree of Redesigned Language
AoA proves theorems by editing a JSON-AST proof tree for the new Minilang language, reporting 2.9–6.9x fewer tokens and 2.3–4.7x lower API cost than Amazon's Isabelle agent with equal or better pass rates.
-
Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
CoT-PoT ensembling achieves self-consistency accuracy in LLMs with only two samples for 78.6% of tasks, reducing computation by 9.3x compared to standard methods.
-
Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
Cross-modal agreement between chain-of-thought and program-of-thought reasoning enables self-consistency with only two LLM samples, reducing sampling cost by 9.3x while improving accuracy.
-
Training Language Models to Cooperate with Inference-Time Controllers
Multi-controller post-training with turn-level GRPO over reusable reasoning modules improves LLM transfer to held-out and shifted inference-time controllers versus single-controller RL.
-
A Workflow-Aware Serving Layer for Agentic Applications
A workflow-aware serving layer compiles per-node model-verifier-backend plans with an ILP and adapts only uncommitted work via pre-solved pressure rungs and residual re-solves.
-
Automated Discovery Has No Universally Superior Harness
No fixed discovery harness is reliably superior across 12 model–problem pairs, OpenEvolve-style recipes underperform simpler alternatives, and online pruning of weak partial runs improves budget-matched performance.
-
Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models
Agent-GWO uses collaborative grey-wolf-inspired agents to jointly optimize LLM prompts and decoding settings, yielding higher accuracy and stability than prior single-agent prompt optimization methods on math and hybr...
-
GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysis
GenoMAS deploys six specialized LLM agents with guided planning to preprocess transcriptomic data and identify genes, reaching 89.13% composite similarity and 60.48% F1 on the GenoTEX benchmark while outperforming pri...
-
CHESS: Contextual Harnessing for Efficient SQL Synthesis
CHESS deploys four LLM agents to retrieve information, prune schemas, generate refined SQL candidates, and validate via unit tests, reporting up to 71.10% accuracy on BIRD with 83% fewer calls than leading proprietary...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.