Typed states for the displayed outbound observations.
Source: paper_references, paper_reference_links, observed 2026-07-04T21:40:04.594380Z
Paper Citation Record · LEDGER
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
As of 15 August 2026, this Paper Citation Record lists 15 of 15 outbound references and 3 inbound Pith citation observations for arXiv:2605.28840.
A citation records a reference. It does not transfer a finding from one paper to another.
Coverage vector
One-hop event checks from named stored sources.
Source: scholarly_work_events, retraction_status_cache, observed 2026-08-15T06:32:42.880941+00:00
Pith citing papers itemized under the disclosed page cap.
Source: paper_references, paper_reference_links, observed 2026-08-14T04:20:34.128103Z
A source-named dated measurement, never combined with another source.
Source: pith, observed 2026-08-11T20:08:14.591497Z
Reference resolution
15 of 15 outbound references displayed
- verified exact5
- verified fuzzy9
- unresolved0
- parse uncertain0
- malformed identifier0
- metadata mismatch1
External citation measurements
No source-named external measurement is stored.
Outbound references
Observation 2820413a-b0d2-471e-b7ea-cae3e27d243a · outbound
This paper cites AI Agents That Matter.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines AI Agents That Matter
Reference 1
- Resolution
- verified exact
- arxiv_id, observed 2026-07-04T21:40:08.959105Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 24bd9db3-4bf6-4676-8d65-159c7b04e6ae · outbound
This paper cites Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles
Reference 2
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.971067Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation c4923d33-4936-449a-ba5f-50e2e39040cf · outbound
This paper cites API-Bank : A Comprehensive Benchmark for Tool-Augmented LLMs.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines API-Bank : A Comprehensive Benchmark for Tool-Augmented LLMs
Reference 3
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.980586Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 29f4bf33-5b75-4cae-a1b0-dccee256f807 · outbound
This paper cites AgentBench : Evaluating LLMs as Agents.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines AgentBench : Evaluating LLMs as Agents
Reference 4
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.978307Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 8826c55f-c90b-4d34-afb6-e4ed7f29e5c4 · outbound
This paper cites Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Reference 5
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.985460Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 58ea8c18-cbd7-4fe5-8569-fcf3b5f5f337 · outbound
This paper cites When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents
Reference 6
- Resolution
- verified exact
- arxiv_id, observed 2026-07-16T01:21:45.045605Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 5cc656bf-e0b3-437f-8d11-29abf3cd8675 · outbound
This paper cites Gorilla: Large Language Model Connected with Massive APIs.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Gorilla: Large Language Model Connected with Massive APIs
Reference 7
- Resolution
- metadata mismatch
- local_arxiv, observed 2026-07-04T21:40:08.951739Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation a0b67295-ebb5-4149-a60b-5683717d0fd4 · outbound
This paper cites True Few-Shot Learning with Language Models.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines True Few-Shot Learning with Language Models
Reference 8
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.975849Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation e3e80ddd-7a48-40a1-8a4f-eff821a61795 · outbound
This paper cites ToolLLM : Facilitating Large Language Models to Master 16000+ Real-World APIs.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines ToolLLM : Facilitating Large Language Models to Master 16000+ Real-World APIs
Reference 9
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.988145Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 68a76fa4-6aeb-4cb1-9f99-57a638fb2337 · outbound
This paper cites Self-Reflection in LLM Agents: Effects on Problem-Solving Performance.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
Reference 10
- Resolution
- verified exact
- arxiv_id, observed 2026-07-04T21:40:08.954429Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 7c2c9479-1b95-4a89-be6f-3fc238a75c18 · outbound
This paper cites Toolformer : Language Models Can Teach Themselves to Use Tools.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Toolformer : Language Models Can Teach Themselves to Use Tools
Reference 11
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.982920Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation b750e918-9a7b-430d-a5cb-c5eef6908d31 · outbound
This paper cites Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
Reference 12
- Resolution
- verified exact
- local_arxiv, observed 2026-07-04T21:40:08.966556Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 50b41652-8c61-4d32-8590-7a1624e5e315 · outbound
This paper cites Self-Consistency Improves Chain of Thought Reasoning in Language Models.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Self-Consistency Improves Chain of Thought Reasoning in Language Models
Reference 13
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.983094Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 8da8965d-35cb-4628-a274-be211553ffdf · outbound
This paper cites Ethical and social risks of harm from Language Models.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines Ethical and social risks of harm from Language Models
Reference 14
- Resolution
- verified exact
- local_arxiv, observed 2026-07-04T21:40:08.955390Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation d33d085a-1893-442d-946c-b2933f12688c · outbound
This paper cites ReAct : Synergizing Reasoning and Acting in Language Models.
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines ReAct : Synergizing Reasoning and Acting in Language Models
Reference 15
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-07-04T21:40:08.980637Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Pith citing papers
Observation 1d6a2fe6-0b4f-4d8c-b9ae-2b04ab3f8272 · inbound
Determinants and Limits of LLM Security-Tool Orchestration: A Study with HexStrike-AI cites this paper.
Determinants and Limits of LLM Security-Tool Orchestration: A Study with HexStrike-AI How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
Reference 19
- Resolution
- unresolved
- no resolver link, observed 2026-07-12T06:28:52.906906Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8291d6d5-5a89-4c07-b573-664cdbabcc8f · inbound
OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review cites this paper.
OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
Reference 24
- Resolution
- verified exact
- local_arxiv, observed 2026-08-11T20:08:14.595449Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-15T06:32:42.880941+00:00.
Observation 2cae528b-4854-46f2-851b-fdaec5cea5f4 · inbound
OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review cites this paper.
OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
Reference 24
- Resolution
- unresolved
- no resolver link, observed 2026-08-14T04:20:34.128103Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.