AgentForesight introduces an online auditor model that predicts decisive errors in multi-agent trajectories at the earliest step using a coarse-to-fine reinforcement learning recipe on a new curated dataset AFTraj-2K.
Canonical reference
G-eval: Nlg evaluation using gpt-4 with better human alignment
Canonical reference. 100% of citing Pith papers cite this work as background.
5
Pith papers citing it
Background
100% of classified citations
citation-role summary
background 5
citation-polarity summary
years
2026 5verdicts
UNVERDICTED 5roles
background 4polarities
background 4representative citing papers
ASTRA-QA is a benchmark for abstract document question answering that uses explicit topic sets, unsupported content annotations, and evidence alignments to enable direct scoring of coverage and hallucination.
LLM safety judges flip verdicts on equivalent policy rewrites up to 9.1% of the time and cannot distinguish meaningful from meaningless changes, requiring new invariance-based reliability metrics.
A persona-driven multi-agent framework with a three-dimensional decision-theoretic evaluation shows that agent-persona alignment significantly impacts performance and coordination in O-RAN optimization challenges.
Fine-tuned simulators grounded in real human data produce LLM assistants that win more often against real users than those trained against role-playing simulators.
citing papers explorer
-
AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems
AgentForesight introduces an online auditor model that predicts decisive errors in multi-agent trajectories at the earliest step using a coarse-to-fine reinforcement learning recipe on a new curated dataset AFTraj-2K.
-
ASTRA-QA: A Benchmark for Abstract Question Answering over Documents
ASTRA-QA is a benchmark for abstract document question answering that uses explicit topic sets, unsupported content annotations, and evidence alignments to enable direct scoring of coverage and hallucination.
-
Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges
LLM safety judges flip verdicts on equivalent policy rewrites up to 9.1% of the time and cannot distinguish meaningful from meaningless changes, requiring new invariance-based reliability metrics.
-
Decision-Theoretic Safety Assessment of Persona-Driven Multi-Agent Systems in O-RAN
A persona-driven multi-agent framework with a three-dimensional decision-theoretic evaluation shows that agent-persona alignment significantly impacts performance and coordination in O-RAN optimization challenges.
-
Quantifying the Utility of User Simulators for Building Collaborative LLM Assistants
Fine-tuned simulators grounded in real human data produce LLM assistants that win more often against real users than those trained against role-playing simulators.