Gatekeepers and Hallucinations: A Layered Evaluation Framework for LLM-Driven Quantum Circuit Generation

Christopher Coleman; Sharon Marfatia

arxiv: 2606.18422 · v1 · pith:Z2RQCDSDnew · submitted 2026-06-16 · 🪐 quant-ph

Gatekeepers and Hallucinations: A Layered Evaluation Framework for LLM-Driven Quantum Circuit Generation

Christopher Coleman , Sharon Marfatia This is my paper

Pith reviewed 2026-06-27 00:12 UTC · model grok-4.3

classification 🪐 quant-ph

keywords quantum circuitslarge language modelsvariational quantum eigensolverevaluation frameworkfailure modesgatekeeper validationhallucinationmaterials simulation

0 comments

The pith

LLM outputs for quantum circuits in materials simulations require gatekeeper validation to catch inherent failure modes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a layered evaluation framework to assess large language models when they generate quantum circuits for variational quantum eigensolver calculations informed by materials data. The framework combines a gatekeeper rubric checking physical and integration criteria, a fidelity comparison to known reference circuits, and a consistency measure called design entropy. It identifies five distinct failure modes that occur across models and stem from the nature of the generation task itself. These include hallucinating molecular geometries and using nonexistent APIs. Without such layered checks, errors risk propagating through costly quantum simulation pipelines, and the work also reveals that the testing setup can introduce its own silent failures.

Core claim

The central claim is that a multi-layered evaluation framework, including gatekeeper screening, circuit fidelity analysis, and design entropy, reveals five structural failure modes in LLM-driven quantum circuit generation for VQE on materials like H2, making gatekeeper-style validation necessary rather than optional for reliable deployment across multiple foundation models. A forensic audit of the evaluation platform further shows that some failures originate in the harness infrastructure, placing it within the same trust boundary.

What carries the argument

The layered evaluation framework consisting of a gatekeeper screening rubric across seven physical and framework criteria, circuit fidelity analysis against reference implementations, and design entropy as a run-to-run consistency metric.

If this is right

Gatekeeper validation must be integrated into LLM-driven quantum workflows to prevent propagation of errors through simulation pipelines.
Evaluation infrastructure requires its own forensic auditing because silent substitutions can mimic model failures.
The five failure modes each carry distinct detectability profiles that call for targeted checks rather than single-point verification.
Design entropy serves as a practical metric for quantifying behavioral consistency in repeated circuit generation runs.
The framework applies directly to Materials Project integrated pipelines for ansatze such as UCCSD on small molecules like H2.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The taxonomy of failure modes could guide creation of targeted fine-tuning datasets for quantum code generation tasks.
Similar layered screening might transfer to LLM-assisted code generation in other domains where verification carries high computational cost.
Direct embedding of gatekeeper logic into agentic pipelines could enable iterative self-correction during circuit synthesis.
Testing the framework on larger molecules would show whether failure rates scale with system size or remain task-inherent.

Load-bearing premise

The five failure modes are structural to the task of LLM-driven quantum circuit generation rather than specific to any one model.

What would settle it

A demonstration that all five failure modes can be eliminated in a production pipeline using a single foundation model without any gatekeeper screening, or conversely, that a model exhibits none of them on repeated trials for the H2 VQE task.

Figures

Figures reproduced from arXiv: 2606.18422 by Christopher Coleman, Sharon Marfatia.

**Figure 2.** Figure 2: FIG. 2: Circuit fidelity and ansatz classification (H [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: FIG. 3: Design entropy and temperature stability [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: FIG. 4: AWS service architecture for the QMI multi-agent platform. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

read the original abstract

As large language models (LLMs) become embedded in quantum simulation workflows (IDE copilots, notebook assistants, agentic pipelines), evaluation must move beyond functional correctness to anticipate and catch structured failures before they propagate through expensive pipelines. We present a layered evaluation framework for materials-informed Variational Quantum Eigensolver (VQE) circuit generation: (i) a gatekeeper screening rubric across seven physical and framework criteria; (ii) a circuit fidelity analysis comparing model outputs against analytical and reference-implementation values for H2/STO-3G/Jordan-Wigner/UCCSD, with ansatz classification and gate-composition breakdown; and (iii) design entropy, a run-to-run behavioral consistency metric. We surface a taxonomy of five distinct LLM failure modes (geometry hallucination, nonexistent API usage, runtime integration failures, constraint violations, and plausible-but-unverifiable output), each with distinct detectability profiles and structural to the task rather than to any one model. A forensic audit of the evaluation platform's own source code further establishes that two apparent model failures originated in the harness through silent fallback-template substitution, demonstrating that evaluation infrastructure belongs inside the same trust boundary as the models it tests. Applied across multiple foundation models on a Materials Project integrated pipeline, the framework shows that gatekeeper-style validation is necessary, not optional, for reliable deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper gives a concrete taxonomy and harness audit for LLM quantum circuit failures, but the claim that gatekeepers are necessary rests on descriptions rather than shown numbers or cross-model breakdowns.

read the letter

The main things here are the five-mode failure taxonomy (geometry hallucination, bad API calls, integration issues, constraint breaks, and unverifiable output) plus the design entropy metric for run-to-run consistency, all wrapped in a seven-criteria gatekeeper rubric and a fidelity check against the H2 reference. The self-audit of the testing harness that found two silent fallback errors is the clearest piece of new, useful work; it shows the authors treating their own infrastructure as part of the trust boundary, which is rare and worth copying.

The framework itself is straightforward to apply to other VQE or materials pipelines, and the taxonomy maps onto real deployment risks in agentic quantum tools. That part is worth having in the literature.

The soft spot is the central claim that these modes are structural to the task and that gatekeeper validation is therefore necessary. The abstract reports testing multiple models on a Materials Project pipeline but supplies no counts, no per-model tables, no prompt controls, and no direct comparison of how many outputs would have passed ordinary functional checks yet failed the rubric. The stress-test note is accurate on this point: without those breakdowns the necessity argument stays at the level of assertion. The forensic finding about harness errors also muddies attribution between model and test setup.

This is for groups already shipping LLM copilots into quantum simulation stacks who need a checklist and failure language. It is coherent on its own terms and shows clear thinking about evaluation infrastructure, so it deserves a serious referee even though the results section will need the missing quantitative detail to carry the stronger claims.

Referee Report

2 major / 0 minor

Summary. The paper introduces a layered evaluation framework for LLM-generated quantum circuits in materials-informed VQE workflows. It defines a seven-criteria gatekeeper rubric, a circuit fidelity analysis against reference H2/STO-3G/UCCSD implementations, and a 'design entropy' consistency metric. The work reports a taxonomy of five failure modes (geometry hallucination, nonexistent API usage, runtime integration failures, constraint violations, plausible-but-unverifiable output) claimed to be structural to the task, presents a forensic audit revealing two harness-induced failures, and concludes that gatekeeper-style validation is necessary rather than optional when the framework is applied across multiple foundation models on a Materials Project pipeline.

Significance. If the observed failure modes prove structural and the gatekeeper rubric demonstrably catches outputs that would pass functional-correctness checks, the framework could improve reliability of LLM-assisted quantum simulation pipelines. The self-audit of the evaluation harness itself is a methodological strength that places infrastructure inside the trust boundary. However, the absence of quantitative results, error bars, cross-model tabulations, or explicit counts of gatekeeper-rejected outputs limits the ability to assess whether the necessity claim holds beyond high-level description.

major comments (2)

[Abstract] Abstract (final paragraph): the claim that the five failure modes are 'structural to the task rather than to any one model' is not supported by any tabulated cross-model breakdown, controls for prompt or harness variation, or explicit count of outputs that would have passed a standard functional-correctness check but failed the seven-criteria gatekeeper rubric; without these data the necessity conclusion rests on descriptive assertion alone.
[Abstract] Abstract: the forensic audit finding that two failures originated in the harness is presented without accompanying totals (e.g., total model outputs examined, fraction attributable to harness vs. model), making it impossible to evaluate the relative contribution of infrastructure errors to the reported failure taxonomy.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and agree that the abstract claims would benefit from additional quantitative support.

read point-by-point responses

Referee: [Abstract] Abstract (final paragraph): the claim that the five failure modes are 'structural to the task rather than to any one model' is not supported by any tabulated cross-model breakdown, controls for prompt or harness variation, or explicit count of outputs that would have passed a standard functional-correctness check but failed the seven-criteria gatekeeper rubric; without these data the necessity conclusion rests on descriptive assertion alone.

Authors: The manuscript applies the framework to multiple foundation models within the Materials Project pipeline and consistently observes the same five failure modes, which underpins the structural claim. We acknowledge, however, that the current version lacks explicit tabulated cross-model breakdowns, controls, or counts of gatekeeper rejections versus functional-correctness passes. We will add a dedicated results table with these quantities and revise the abstract accordingly to make the evidence explicit. revision: yes
Referee: [Abstract] Abstract: the forensic audit finding that two failures originated in the harness is presented without accompanying totals (e.g., total model outputs examined, fraction attributable to harness vs. model), making it impossible to evaluate the relative contribution of infrastructure errors to the reported failure taxonomy.

Authors: The forensic audit consisted of a targeted source-code review that isolated two specific harness defects (silent fallback-template substitution). We agree that aggregate totals are needed to contextualize their contribution. We will report the total number of model outputs examined and the fraction attributable to harness versus model errors in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical evaluation framework with external application to multiple models

full rationale

The paper introduces a layered evaluation framework, taxonomy of failure modes, and metrics (gatekeeper rubric, fidelity analysis, design entropy) derived from direct testing on multiple foundation models via a Materials Project pipeline. No equations, fitted parameters, or predictions are present that reduce outputs to inputs by construction. The claim that failure modes are structural rests on cross-model observations rather than self-definition or self-citation chains. The forensic self-audit of the harness is presented as an independent finding and does not load-bear the central necessity-of-gatekeepers conclusion. The derivation chain is self-contained against external benchmarks with no load-bearing self-citations or ansatz smuggling.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claim rests on the domain assumption that LLM failures in this task are structured and detectable by the proposed layers, plus the invented metric design entropy and the five-mode taxonomy; no free parameters are stated.

axioms (1)

domain assumption LLM outputs for quantum circuit generation exhibit structured, task-level failure modes that are general across models
Invoked to justify the taxonomy and the claim that gatekeeper validation is necessary rather than model-specific.

invented entities (2)

design entropy no independent evidence
purpose: run-to-run behavioral consistency metric for LLM circuit generation
Newly introduced quantity whose definition and computation are not supplied in the abstract.
taxonomy of five LLM failure modes no independent evidence
purpose: classify geometry hallucination, nonexistent API usage, runtime integration failures, constraint violations, and plausible-but-unverifiable output
Classification scheme created for this study with no external validation cited in the abstract.

pith-pipeline@v0.9.1-grok · 5770 in / 1516 out tokens · 41054 ms · 2026-06-27T00:12:50.584998+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

27 extracted references · 10 canonical work pages · 1 internal anchor

[1]

A variational eigenvalue solver on a photonic quantum processor,

A. Peruzzoet al., “A variational eigenvalue solver on a photonic quantum processor,”Nat. Commun.5, 4213 (2014)

2014
[2]

The variational quantum eigensolver: a review of methods and best practices,

J. Tillyet al., “The variational quantum eigensolver: a review of methods and best practices,”Phys. Rep.986, 1 (2022)

2022
[3]

VQE method: a short survey and recent developments,

D. A. Fedorov, B. Peng, N. Govind, and Y. Alexeev, “VQE method: a short survey and recent developments,” Mater. Theory6, 2 (2022)

2022
[4]

Quantum computational chemistry,

S. McArdle, S. Endo, A. Aspuru-Guzik, S. C. Benjamin, and X. Yuan, “Quantum computational chemistry,”Rev. Mod. Phys.92, 015003 (2020)

2020
[5]

An adaptive variational algorithm for exact molecular simulations on a quantum computer,

H. R. Grimsley, S. E. Economou, E. Barnes, and N. J. Mayhall, “An adaptive variational algorithm for exact molecular simulations on a quantum computer,”Nat. Commun.10, 3007 (2019)

2019
[6]

Adaptive, problem-tailored vari- ational quantum eigensolver mitigates rough parameter landscapes and barren plateaus,

H. R. Grimsleyet al., “Adaptive, problem-tailored vari- ational quantum eigensolver mitigates rough parameter landscapes and barren plateaus,”npj Quantum Inf.9, 19 (2023)

2023
[7]

Larocca, S

M. Laroccaet al., “A review of barren plateaus in varia- tional quantum computing,”arXiv:2405.00781 (2024)

work page arXiv 2024
[8]

Characterizing barren plateaus in quantum ansätze with the adjoint representation,

E. Fontanaet al., “Characterizing barren plateaus in quantum ansätze with the adjoint representation,”Nat. Commun.15, 7171 (2024)

2024
[9]

Qiskit HumanEval: an evaluation benchmark for quantum code generative models,

S. Vishwakarmaet al., “Qiskit HumanEval: an evaluation benchmark for quantum code generative models,”Proc. IEEE QCE, pp. 1169–1176 (2024). arXiv:2406.14712

work page arXiv 2024
[10]

QCircuitBench: a large-scale dataset for benchmarking quantum algorithm design,

R. Yang, Z. Wang, Y. Gu, T. Chen, Y. Liang, and T. Li, “QCircuitBench: a large-scale dataset for benchmarking quantum algorithm design,”arXiv:2410.07961 (2024)

work page arXiv 2024
[11]

Agent-Q: fine-tuning large language models for quantum circuit generation and optimization,

L. Jern, V. Uotila, C. Yu, and B. Zhao, “Agent-Q: fine-tuning large language models for quantum circuit generation and optimization,”Proc. IEEE QCE(2025). arXiv:2504.11109

work page arXiv 2025
[12]

Quasar: Quantum assembly code generation using tool-augmented llms via agentic rl

C. Yuet al., “QUASAR: quantum assembly code gen- eration using tool-augmented LLMs via agentic RL,” arXiv:2510.00967 (2025)

work page arXiv 2025
[13]

The generative quantum eigensolver (GQE) and its application for ground state search,

K. Nakajiet al., “The generative quantum eigensolver (GQE) and its application for ground state search,” arXiv:2401.09253 (2024)

work page arXiv 2024
[14]

Quantum verifiable rewards for post-training qiskit code assistant

N. Dupuiset al., “Quantum verifiable rewards for post- training Qiskit code assistant,”arXiv:2508.20907 (2025)

work page arXiv 2025
[15]

Beyond Functional Correctness : Exploring Hallucinations in LLM - Generated Code , January 2026

F. Liuet al., “Beyond functional correctness: ex- ploring hallucinations in LLM-generated code,” arXiv:2404.00971 (2024); accepted,IEEE Trans. Softw. Eng

work page arXiv 2024
[16]

El Agente: an autonomous agent for quan- tum chemistry,

Y. Zouet al., “El Agente: an autonomous agent for quan- tum chemistry,”arXiv:2505.02484 (2025)

work page arXiv 2025
[17]

Automating quantum computing labora- tory experiments with an agent-based AI framework,

S. Caoet al., “Automating quantum computing labora- tory experiments with an agent-based AI framework,” Patterns6, 101372 (2025)

2025
[18]

Commentary: The Materials Project: a materials genome approach to accelerating materials in- novation,

A. Jainet al., “Commentary: The Materials Project: a materials genome approach to accelerating materials in- novation,”APL Mater.1, 011002 (2013)

2013
[19]

Accelerated data-driven materials 8 science with the Materials Project,

M. K. Hortonet al., “Accelerated data-driven materials 8 science with the Materials Project,”Nat. Mater.24, 1522 (2025)

2025
[20]

Quantum computing with Qiskit

A. Javadi-Abhariet al., “Quantum computing with Qiskit,”arXiv:2405.08810 (2024)

work page internal anchor Pith review Pith/arXiv arXiv 2024
[21]

Simulating the elec- tronic structure of spin defects on quantum computers,

B. Huang, M. Govoni, and G. Galli, “Simulating the elec- tronic structure of spin defects on quantum computers,” PRX Quantum3, 010339 (2022)

2022
[22]

Python Materials Genomics (pymat- gen): a robust, open-source Python library for materials analysis,

S. P. Onget al., “Python Materials Genomics (pymat- gen): a robust, open-source Python library for materials analysis,”Comput. Mater. Sci.68, 314 (2013). [23]https://github.com/UBC-CIC/ Quantum-Matter-Institute-Streamlit-App, file models/base_model.py. Appendix A: AWS Platform Architecture and Implementation This appendix describes the cloud infrastructur...

2013
[23]

Authentication is handled by Amazon Cognito with JWT tokens; API cre- dentials are stored in AWS Secrets Manager and never exposed to the application layer

System Overview The platform is a containerised web application de- ployed on AWS Elastic Beanstalk, accessible to re- searchers via a Streamlit front-end. Authentication is handled by Amazon Cognito with JWT tokens; API cre- dentials are stored in AWS Secrets Manager and never exposed to the application layer. Global traffic is routed through Amazon Clou...
[24]

Three layers of agents are composed at runtime

Multi-Agent Architecture Model inference and workflow orchestration are imple- mented using the AWS Strands Agents SDK, an open- source, model-driven framework for building production- ready AI agents. Three layers of agents are composed at runtime. The Supervisor Agent acts as the top-level co- ordinator, receiving user queries, selecting the appropri- a...
[25]

Two MCP servers were deployed

Model Context Protocol Server Integration External scientific data sources are accessed through ModelContextProtocol(MCP)servers—astandardised protocol that provides AI agents with secure, structured access to external databases and tools. Two MCP servers were deployed. The Materials Project MCP Server in- terfaces with the Materials Project database [18]...
[26]

Deployment Configuration All resources were deployed in thecacentral1AWS region. The application container runs on at3.large EC2instancemanagedbyElasticBeanstalk, withDocker packaging the Streamlit application and all Python de- pendencies for reproducible deployment
[27]

Post-Proof-of-Concept Development Roadmap Several infrastructure and capability improvements have been identified for production deployment beyond the current proof-of-concept. On the data side, in- tegrating Amazon OpenSearch would enable retrieval- augmented generation over a searchable corpus of quan- tum research papers and previously generated circui...

[1] [1]

A variational eigenvalue solver on a photonic quantum processor,

A. Peruzzoet al., “A variational eigenvalue solver on a photonic quantum processor,”Nat. Commun.5, 4213 (2014)

2014

[2] [2]

The variational quantum eigensolver: a review of methods and best practices,

J. Tillyet al., “The variational quantum eigensolver: a review of methods and best practices,”Phys. Rep.986, 1 (2022)

2022

[3] [3]

VQE method: a short survey and recent developments,

D. A. Fedorov, B. Peng, N. Govind, and Y. Alexeev, “VQE method: a short survey and recent developments,” Mater. Theory6, 2 (2022)

2022

[4] [4]

Quantum computational chemistry,

S. McArdle, S. Endo, A. Aspuru-Guzik, S. C. Benjamin, and X. Yuan, “Quantum computational chemistry,”Rev. Mod. Phys.92, 015003 (2020)

2020

[5] [5]

An adaptive variational algorithm for exact molecular simulations on a quantum computer,

H. R. Grimsley, S. E. Economou, E. Barnes, and N. J. Mayhall, “An adaptive variational algorithm for exact molecular simulations on a quantum computer,”Nat. Commun.10, 3007 (2019)

2019

[6] [6]

Adaptive, problem-tailored vari- ational quantum eigensolver mitigates rough parameter landscapes and barren plateaus,

H. R. Grimsleyet al., “Adaptive, problem-tailored vari- ational quantum eigensolver mitigates rough parameter landscapes and barren plateaus,”npj Quantum Inf.9, 19 (2023)

2023

[7] [7]

Larocca, S

M. Laroccaet al., “A review of barren plateaus in varia- tional quantum computing,”arXiv:2405.00781 (2024)

work page arXiv 2024

[8] [8]

Characterizing barren plateaus in quantum ansätze with the adjoint representation,

E. Fontanaet al., “Characterizing barren plateaus in quantum ansätze with the adjoint representation,”Nat. Commun.15, 7171 (2024)

2024

[9] [9]

Qiskit HumanEval: an evaluation benchmark for quantum code generative models,

S. Vishwakarmaet al., “Qiskit HumanEval: an evaluation benchmark for quantum code generative models,”Proc. IEEE QCE, pp. 1169–1176 (2024). arXiv:2406.14712

work page arXiv 2024

[10] [10]

QCircuitBench: a large-scale dataset for benchmarking quantum algorithm design,

R. Yang, Z. Wang, Y. Gu, T. Chen, Y. Liang, and T. Li, “QCircuitBench: a large-scale dataset for benchmarking quantum algorithm design,”arXiv:2410.07961 (2024)

work page arXiv 2024

[11] [11]

Agent-Q: fine-tuning large language models for quantum circuit generation and optimization,

L. Jern, V. Uotila, C. Yu, and B. Zhao, “Agent-Q: fine-tuning large language models for quantum circuit generation and optimization,”Proc. IEEE QCE(2025). arXiv:2504.11109

work page arXiv 2025

[12] [12]

Quasar: Quantum assembly code generation using tool-augmented llms via agentic rl

C. Yuet al., “QUASAR: quantum assembly code gen- eration using tool-augmented LLMs via agentic RL,” arXiv:2510.00967 (2025)

work page arXiv 2025

[13] [13]

The generative quantum eigensolver (GQE) and its application for ground state search,

K. Nakajiet al., “The generative quantum eigensolver (GQE) and its application for ground state search,” arXiv:2401.09253 (2024)

work page arXiv 2024

[14] [14]

Quantum verifiable rewards for post-training qiskit code assistant

N. Dupuiset al., “Quantum verifiable rewards for post- training Qiskit code assistant,”arXiv:2508.20907 (2025)

work page arXiv 2025

[15] [15]

Beyond Functional Correctness : Exploring Hallucinations in LLM - Generated Code , January 2026

F. Liuet al., “Beyond functional correctness: ex- ploring hallucinations in LLM-generated code,” arXiv:2404.00971 (2024); accepted,IEEE Trans. Softw. Eng

work page arXiv 2024

[16] [16]

El Agente: an autonomous agent for quan- tum chemistry,

Y. Zouet al., “El Agente: an autonomous agent for quan- tum chemistry,”arXiv:2505.02484 (2025)

work page arXiv 2025

[17] [17]

Automating quantum computing labora- tory experiments with an agent-based AI framework,

S. Caoet al., “Automating quantum computing labora- tory experiments with an agent-based AI framework,” Patterns6, 101372 (2025)

2025

[18] [18]

Commentary: The Materials Project: a materials genome approach to accelerating materials in- novation,

A. Jainet al., “Commentary: The Materials Project: a materials genome approach to accelerating materials in- novation,”APL Mater.1, 011002 (2013)

2013

[19] [19]

Accelerated data-driven materials 8 science with the Materials Project,

M. K. Hortonet al., “Accelerated data-driven materials 8 science with the Materials Project,”Nat. Mater.24, 1522 (2025)

2025

[20] [20]

Quantum computing with Qiskit

A. Javadi-Abhariet al., “Quantum computing with Qiskit,”arXiv:2405.08810 (2024)

work page internal anchor Pith review Pith/arXiv arXiv 2024

[21] [21]

Simulating the elec- tronic structure of spin defects on quantum computers,

B. Huang, M. Govoni, and G. Galli, “Simulating the elec- tronic structure of spin defects on quantum computers,” PRX Quantum3, 010339 (2022)

2022

[22] [22]

Python Materials Genomics (pymat- gen): a robust, open-source Python library for materials analysis,

S. P. Onget al., “Python Materials Genomics (pymat- gen): a robust, open-source Python library for materials analysis,”Comput. Mater. Sci.68, 314 (2013). [23]https://github.com/UBC-CIC/ Quantum-Matter-Institute-Streamlit-App, file models/base_model.py. Appendix A: AWS Platform Architecture and Implementation This appendix describes the cloud infrastructur...

2013

[23] [23]

Authentication is handled by Amazon Cognito with JWT tokens; API cre- dentials are stored in AWS Secrets Manager and never exposed to the application layer

System Overview The platform is a containerised web application de- ployed on AWS Elastic Beanstalk, accessible to re- searchers via a Streamlit front-end. Authentication is handled by Amazon Cognito with JWT tokens; API cre- dentials are stored in AWS Secrets Manager and never exposed to the application layer. Global traffic is routed through Amazon Clou...

[24] [24]

Three layers of agents are composed at runtime

Multi-Agent Architecture Model inference and workflow orchestration are imple- mented using the AWS Strands Agents SDK, an open- source, model-driven framework for building production- ready AI agents. Three layers of agents are composed at runtime. The Supervisor Agent acts as the top-level co- ordinator, receiving user queries, selecting the appropri- a...

[25] [25]

Two MCP servers were deployed

Model Context Protocol Server Integration External scientific data sources are accessed through ModelContextProtocol(MCP)servers—astandardised protocol that provides AI agents with secure, structured access to external databases and tools. Two MCP servers were deployed. The Materials Project MCP Server in- terfaces with the Materials Project database [18]...

[26] [26]

Deployment Configuration All resources were deployed in thecacentral1AWS region. The application container runs on at3.large EC2instancemanagedbyElasticBeanstalk, withDocker packaging the Streamlit application and all Python de- pendencies for reproducible deployment

[27] [27]

Post-Proof-of-Concept Development Roadmap Several infrastructure and capability improvements have been identified for production deployment beyond the current proof-of-concept. On the data side, in- tegrating Amazon OpenSearch would enable retrieval- augmented generation over a searchable corpus of quan- tum research papers and previously generated circui...