REVIEW 2 cited by
The Vera C. Rubin Observatory Data Butler and Pipeline Execution System
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The Rubin Observatory's Data Butler is designed to allow data file location and file formats to be abstracted away from the people writing the science pipeline algorithms. The Butler works in conjunction with the workflow graph builder to allow pipelines to be constructed from the algorithmic tasks. These pipelines can be executed at scale using object stores and multi-node clusters, or on a laptop using a local file system. The Butler and pipeline system are now in daily use during Rubin construction and early operations.
Forward citations
Cited by 2 Pith papers
-
Data Movement Model for the Vera C. Rubin Observatory
Rubin Observatory's data movement relies on Rucio and FTS for transfers, plus three custom tools that tie Rucio to the Data Butler registry.
-
Implementing SIAv2 Over Rubin Observatory's Data Butler
Rubin Observatory has implemented an SIAv2 image access service that queries the Data Butler directly, with some metadata gaps for coadded images.
Discussion (0). Continue with ORCID to comment.