Making the complete OpenAIRE citation graph easily accessible through compact data representation
Pith reviewed 2026-05-16 05:28 UTC · model grok-4.3
The pith
The full OpenAIRE citation graph is reduced to a 16 GB dataset while preserving every publication and citation.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We processed the OpenAIRE citation graph, which contains over 200 million publications and over 2 billion citations, into a compact representation that requires only 16 GB of RAM while preserving the full graph structure. The data are released in a simple format that supports straightforward manipulation, together with a Python pipeline for processing future releases and an expanded dataset that includes additional fields such as titles and author lists.
What carries the argument
Compact representation of the citation graph stored in a minimal file format that encodes all nodes and directed edges with low memory overhead.
If this is right
- Citation-network studies that require the complete graph become feasible on ordinary laptops and desktops.
- The supplied Python pipeline lets users regenerate the compact dataset from each new OpenAIRE release.
- Analyses such as global centrality or community detection can run on the full network without subsampling.
- The simple data format allows direct loading into common graph libraries with no custom parsing code.
Where Pith is reading between the lines
- Lowering the memory requirement could expand the set of researchers who examine citation dynamics at the scale of the entire scientific literature.
- If downstream tasks need fields that were stripped during compaction, users may still need to join the compact graph with the original metadata.
- The same compaction approach could be tested on other large open citation or collaboration graphs to check how widely it applies.
Load-bearing premise
The chosen compact format and processing steps keep every detail needed for typical citation analyses without introducing hidden losses or forcing users to recover missing fields later.
What would settle it
Running the same citation-analysis query on both the original 2.5 TB dump and the 16 GB version and obtaining measurably different results.
read the original abstract
The OpenAIRE graph contains a large citation graph dataset, with over 200 million publications and over 2 billion citations. The current graph is available as a dump with metadata which, when uncompressed, totals $\sim$2.5 TB. This makes it hard to process on conventional computers. To make this network more accessible for the community, we provide a processed OpenAIRE graph which is downscaled to 16 GB RAM, while preserving the full graph structure. Apart from this we offer the processed data in a very simple format, which allows for further straightforward manipulation. We also provide (1) a Python pipeline, which can be used to process the next releases of the OpenAIRE graph, and (2) a larger version of the dataset including more publication fields such as, the title, list of authors.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a data-processing pipeline that converts the uncompressed ~2.5 TB OpenAIRE citation-graph dump (over 200 million publications and 2 billion citations) into a compact representation loadable in 16 GB RAM while preserving the complete node and edge set. The authors release the processed data in a simple format, supply an open Python pipeline for re-processing future OpenAIRE releases, and provide an extended version that includes additional metadata fields such as titles and author lists.
Significance. If the released artifacts match the stated size and completeness claims, the contribution materially lowers the computational barrier to working with the full OpenAIRE graph. By supplying both the compact files and the reproducible pipeline, the work enables standard-hardware analyses that previously required specialized resources, directly supporting citation-network and scientometric research.
minor comments (1)
- The abstract and introduction would benefit from a brief, concrete illustration (one or two lines) of the simple output format, e.g., how a node record and an edge record are represented, to make the claim of 'straightforward manipulation' immediately verifiable by readers.
Simulated Author's Rebuttal
We thank the referee for the positive assessment of the manuscript and the recommendation to accept. We are pleased that the contribution is recognized as materially lowering the barrier to working with the complete OpenAIRE citation graph through the release of compact data and a reproducible pipeline.
Circularity Check
No significant circularity in data-processing pipeline
full rationale
The paper describes an open-source processing pipeline that converts the uncompressed OpenAIRE dump into a compact 16 GB representation while preserving all nodes and edges. No mathematical derivations, fitted parameters, predictions, or self-citation chains are present. The central claim is directly verifiable by running the released code on the public data, with no load-bearing step that reduces to its own inputs by construction.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.