REVIEW 8 cited by
The Design of an LLM-powered Unstructured Analytics System
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
LLMs demonstrate an uncanny ability to process unstructured data, and as such, have the potential to go beyond search and run complex, semantic analyses at scale. We describe the design of an unstructured analytics system, Aryn, and the tenets and use cases that motivate its design. With Aryn, users specify queries in natural language and the system automatically determines a semantic plan and executes it to compute an answer from a large collection of unstructured documents. At the core of Aryn is Sycamore, a declarative document processing engine, that provides a reliable distributed abstraction called DocSets. Sycamore allows users to analyze, enrich, and transform complex documents at scale. Aryn includes Luna, a query planner that translates natural language queries to Sycamore scripts, and DocParse, which takes raw PDFs and document images, and converts them to DocSets for downstream processing. We show how these pieces come together to achieve better accuracy than RAG on analytics queries over real world reports from the National Transportation Safety Board (NTSB). Also, given current limitations of LLMs, we argue that an analytics system must provide explainability to be practical, and show how Aryn's user interface does this to help build trust.
Forward citations
Cited by 8 Pith papers
-
Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees
BARGAIN uses betting-based anytime-valid tests and adaptive, target-aware sampling to set model-cascade thresholds, delivering non-asymptotic quality guarantees and up to 86% greater cost savings than SUPG.
-
GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings
GUIDE is a governed multi-agent pipeline that converts enterprise guideline documents into validated rules and deployment artifacts, claiming 96% document success and 40 to 125 minute turnaround on 120 confidential documents.
-
AgenticData: An Agentic Data Analytics System for Heterogeneous Data
A feedback-driven multi-agent system with plan validation and smart memory claims to outperform prior NL-to-analytics systems on heterogeneous benchmarks, though its evaluation is problematic.
-
Steering Semantic Data Processing With DocWrangler
An IDE for LLM-powered text data processing, with user studies showing that people convert open-ended operations into structured classifiers and use vague prompts to explore their data.
-
CoddLLM: Empowering Large Language Models for Data Analytics
CoddLLM, a Mistral-NeMo-12B model post-trained on synthetic analytics instruction data, outperforms GPT-4o on the paper's table-selection average and improves text-to-SQL over its base model, with the largest gains on...
-
Towards Resource-Efficient Compound AI Systems
A declarative workflow system with an adaptive runtime can cut compound AI workflow completion time and energy use by auto-selecting resources and parallelism.
-
Metadata Management for AI-Augmented Data Workflows
TableVault is a metadata governance framework that records ingestion events, operation status, execution parameters, and lineage for human-AI data workflows, demonstrated on a document classification case study.
-
A Survey on Open Dataset Search in the LLM Era: Retrospectives and Perspectives
A structured review of open dataset search across tabular, spatial, JSON, graph, and vector data, plus the two-way relationship with LLMs.
Discussion (0). Continue with ORCID to comment.