Typed states for the displayed outbound observations.
Source: paper_references, paper_reference_links, observed 2026-08-07T14:26:51.305191Z
Paper Citation Record · LEDGER
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
As of 7 August 2026, this Paper Citation Record lists 19 of 19 outbound references and 10 inbound Pith citation observations for arXiv:2505.18878.
A citation records a reference. It does not transfer a finding from one paper to another.
Coverage vector
One-hop event checks from named stored sources.
Source: scholarly_work_events, retraction_status_cache, observed 2026-08-07T06:34:17.273281+00:00
Pith citing papers itemized under the disclosed page cap.
Source: paper_references, paper_reference_links, observed 2026-08-06T21:53:41.111110Z
A source-named dated measurement, never combined with another source.
Source: arxiv_reference, observed 2026-08-05T02:28:24.338817Z
Reference resolution
19 of 19 outbound references displayed
- verified exact0
- verified fuzzy8
- unresolved11
- parse uncertain0
- malformed identifier0
- metadata mismatch0
External citation measurements
0
arxiv_reference, observed 2026-08-05T02:28:24.338817Z
Outbound references
Observation 9e98a1a0-404a-438b-ae1b-5eb844666e85 · outbound
This paper cites Workarena++: Towards compositional planning and reasoning-based common knowledge work tasks.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Workarena++: Towards compositional planning and reasoning-based common knowledge work tasks
Reference 1
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:52.800947Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 83b4a218-2f98-460a-b38d-5e3a32799fd8 · outbound
This paper cites Laradji, Manuel Del Verme, Tom Marty, David Vazquez, Nicolas Chapados, and Alexandre Lacoste.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Laradji, Manuel Del Verme, Tom Marty, David Vazquez, Nicolas Chapados, and Alexandre Lacoste
Reference 2
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:52.596552Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 0aa7387d-7e1a-4f64-ae14-6a3259c68fce · outbound
This paper cites The Llama 3 Herd of Models.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions The Llama 3 Herd of Models
Reference 3
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:49.762943Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 5f17979a-3431-43bc-9e5b-da8d141986da · outbound
This paper cites DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Reference 4
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:49.891756Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation f51b9c15-41e7-461d-a285-060a99eb5326 · outbound
This paper cites Thinking harder: Gemini models get smarter, faster and more helpful.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Thinking harder: Gemini models get smarter, faster and more helpful
Reference 5
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:52.473576Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 6a66226c-85bf-419f-8592-c0065aad960d · outbound
This paper cites Crmarena: Understanding the capacity of llm agents to perform professional crm tasks in realistic environments.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Crmarena: Understanding the capacity of llm agents to perform professional crm tasks in realistic environments
Reference 6
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:52.329499Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 4ca1057d-105e-402d-ad93-133fd56e5132 · outbound
This paper cites Introducing llama 4: Our most advanced models for multimodal intelligence.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Introducing llama 4: Our most advanced models for multimodal intelligence
Reference 7
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:52.160474Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 0c2724bb-9af4-400f-9776-67816c8e5c86 · outbound
This paper cites Gpt-4o system card, 2024.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Gpt-4o system card, 2024
Reference 8
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:50.278286Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 47418978-03be-4ab7-9c16-bf07b3e18751 · outbound
This paper cites Openai o1 system card, 2024.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Openai o1 system card, 2024
Reference 9
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:50.369687Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 3d37a918-40c8-4091-a4ab-4571af6c1432 · outbound
This paper cites APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
Reference 10
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:50.476436Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation ae1c8d42-09b4-4795-b093-f52fa8306145 · outbound
This paper cites Evaluating cultural and social awareness of LLM web agents.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Evaluating cultural and social awareness of LLM web agents
Reference 11
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:52.028887Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation ff08e080-b45b-4f4e-9992-f6608aa53da3 · outbound
This paper cites Multimodal cul- tural safety: Evaluation frameworks and alignment strategies.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Multimodal cul- tural safety: Evaluation frameworks and alignment strategies
Reference 12
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:50.660795Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 35deb571-1eec-4348-8729-65927387dfea · outbound
This paper cites Workbench: a benchmark dataset for agents in a realistic workplace setting.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions Workbench: a benchmark dataset for agents in a realistic workplace setting
Reference 13
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:51.864990Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 4063f8c8-8f83-4315-beca-689cfb025a2d · outbound
This paper cites SafeArena: Evaluating the Safety of Autonomous Web Agents.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions SafeArena: Evaluating the Safety of Autonomous Web Agents
Reference 14
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:50.841665Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation dc8b4365-b852-458f-8625-40b1f4759b1f · outbound
This paper cites The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
Reference 15
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:50.913820Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4b803d66-211c-45eb-b6d9-2e82b9ba95dc · outbound
This paper cites TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
Reference 16
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:51.007765Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 6b683cd4-fa89-4017-aa73-f0c1f5d76ecc · outbound
This paper cites $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Reference 17
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:51.131341Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation b2ef0c9c-7f3b-45a4-90dd-7867627619de · outbound
This paper cites React: Synergizing reasoning and acting in language models.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions React: Synergizing reasoning and acting in language models
Reference 18
- Resolution
- unresolved
- no resolver link, observed 2026-08-07T14:26:51.251074Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4906744e-5568-4ccf-a305-499d257a3bf3 · outbound
This paper cites How often do you use Salesforce CRM?.
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions How often do you use Salesforce CRM?
Reference 19
- Resolution
- verified fuzzy
- raw_fallback, observed 2026-08-07T14:26:51.723498Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Pith citing papers
Observation 1c87699f-104d-4b6a-891b-f1c16a2ba330 · inbound
Benchmarking Deep Search over Heterogeneous Enterprise Data cites this paper.
Benchmarking Deep Search over Heterogeneous Enterprise Data CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 13
- Resolution
- unresolved
- no resolver link, observed 2026-08-06T21:53:41.111110Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 72e96064-d961-430d-b7ce-ebeedc3b138c · inbound
Magentic-UI: Towards Human-in-the-loop Agentic Systems cites this paper.
Magentic-UI: Towards Human-in-the-loop Agentic Systems CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 32
- Resolution
- unresolved
- no resolver link, observed 2026-08-06T11:52:12.148260Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 41590be3-4235-4ddc-b351-610025672d99 · inbound
RecoWorld: Building Simulated Environments for Agentic Recommender Systems cites this paper.
RecoWorld: Building Simulated Environments for Agentic Recommender Systems CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 21
- Resolution
- unresolved
- no resolver link, observed 2026-08-04T17:56:38.790240Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4b9a66a7-eca5-4fce-a420-1ef63c3e6563 · inbound
Beyond the Assistant Turn: User Turn Generation as a Probe of Interaction Awareness in Language Models cites this paper.
Beyond the Assistant Turn: User Turn Generation as a Probe of Interaction Awareness in Language Models CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 9
- Resolution
- metadata mismatch
- arxiv_id, observed 2026-05-13T21:03:20.168057Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation b6139ce9-4b57-4bf3-8e91-8563b62bfa42 · inbound
Absurd World: A Simple Yet Powerful Method to Absurdify the Real-world for Probing LLM Reasoning Capabilities cites this paper.
Absurd World: A Simple Yet Powerful Method to Absurdify the Real-world for Probing LLM Reasoning Capabilities CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 6
- Resolution
- verified exact
- arxiv_id, observed 2026-05-12T07:46:25.917950Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 028d1dbc-fb1c-4ad0-841c-b5a451803420 · inbound
PQR: A Framework to Generate Diverse and Realistic User Queries that Elicit QA Agent Failures cites this paper.
PQR: A Framework to Generate Diverse and Realistic User Queries that Elicit QA Agent Failures CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 39
- Resolution
- verified exact
- arxiv_id, observed 2026-05-20T18:03:36.794050Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation 823a9f21-6ea0-46f0-b00d-f610583eeb58 · inbound
Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking cites this paper.
Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 26
- Resolution
- metadata mismatch
- arxiv_id, observed 2026-07-02T10:56:53.302045Z
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.
Observation d176b4eb-3a38-4dd3-8af8-482a6468a169 · inbound
Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning cites this paper.
Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 11
- Resolution
- unresolved
- no resolver link, observed 2026-08-01T21:31:52.403495Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 7f71e50e-2488-45ab-87a0-c7840abcb2f4 · inbound
Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning cites this paper.
Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 11
- Resolution
- unresolved
- no resolver link, observed 2026-08-03T00:49:50.847591Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 7dc42bde-9141-4db0-8523-e4d7aa7adc26 · inbound
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following cites this paper.
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
Reference 9
- Resolution
- unresolved
- no resolver link, observed 2026-08-01T02:36:07.859860Z
Source-reported events for the cited work
Unavailable: canonical work link unavailable.