LogicHunter combines specification-driven test generation with a ReAct-based agentic oracle to discover 40 previously unknown bugs in LangChain, LlamaIndex, and CrewAI, achieving 91.17% oracle precision.
Title resolution pending
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
A domain-validity rubric and MR-card format screen candidate metamorphic relations into auditable test assets for SciML surrogates, separating model violations from out-of-domain applications.
Neuroforger generates certified violation witnesses for smart contracts by representing specs as Solidity tests with abstract-type variables, using LLMs to instantiate them, and validating via type checking plus execution.
LGMT is a logic-grounded metamorphic testing framework that detects hidden reasoning defects in LLMs by checking consistency on semantically invariant inputs derived from FOL equivalences.
MR-Coupler leverages functional coupling analysis and LLMs to generate valid metamorphic test cases for over 90% of tasks while detecting 44% of real bugs, outperforming baselines by 64.90% in validity and 36.56% in false-alarm reduction.
CoCoMagic applies constrained cooperative co-evolution to metamorphic and differential testing to find up to 287% more distinct behavioral divergences in an end-to-end ADS than baseline search methods.
MR-Scout extracts over 11,000 metamorphic-relation-encoded test cases from 701 OSS projects, codifies 97% of them as high-quality generators, and shows they raise line coverage by 13.52% and mutation score by 9.42% on programs that already have developer tests.
The authors describe an LLM-based two-stage workflow for static verification of code against natural-language requirements via rule extraction and auditing in a cybersecurity case study.
Gimitest is an open-source tool that decorates RL environment APIs to enable search-based, metamorphic, and adversarial testing of single- and multi-agent policies.
citing papers explorer
-
LogicHunter: Testing LLM Agent Frameworks with an Agentic Oracle
LogicHunter combines specification-driven test generation with a ReAct-based agentic oracle to discover 40 previously unknown bugs in LangChain, LlamaIndex, and CrewAI, achieving 91.17% oracle precision.
-
Domain-Validity-Gated Metamorphic Testing of Scientific ML Surrogates
A domain-validity rubric and MR-card format screen candidate metamorphic relations into auditable test assets for SciML surrogates, separating model violations from out-of-domain applications.
-
Neuroforger: certified violation witnesses for smart contracts verification via LLMs
Neuroforger generates certified violation witnesses for smart contracts by representing specs as Solidity tests with abstract-type variables, using LLMs to instantiate them, and validating via type checking plus execution.
-
LGMT: Logic-Grounded Metamorphic Testing for Evaluating the Reasoning Reliability of LLMs
LGMT is a logic-grounded metamorphic testing framework that detects hidden reasoning defects in LLMs by checking consistency on semantically invariant inputs derived from FOL equivalences.
-
MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis
MR-Coupler leverages functional coupling analysis and LLMs to generate valid metamorphic test cases for over 90% of tasks while detecting 44% of real bugs, outperforming baselines by 64.90% in validity and 36.56% in false-alarm reduction.
-
Constrained Co-evolutionary Metamorphic Differential Testing for Autonomous Systems with an Interpretability Approach
CoCoMagic applies constrained cooperative co-evolution to metamorphic and differential testing to find up to 287% more distinct behavioral divergences in an end-to-end ADS than baseline search methods.
-
MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases
MR-Scout extracts over 11,000 metamorphic-relation-encoded test cases from 701 OSS projects, codifies 97% of them as high-quality generators, and shows they raise line coverage by 13.52% and mutation score by 9.42% on programs that already have developer tests.
-
LLM-Based Static Verification of Code Against Natural-Language Requirements: An Industrial Experience Report
The authors describe an LLM-based two-stage workflow for static verification of code against natural-language requirements via rule extraction and auditing in a cybersecurity case study.
-
Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies
Gimitest is an open-source tool that decorates RL environment APIs to enable search-based, metamorphic, and adversarial testing of single- and multi-agent policies.