International Lattice Data Grid 2.0: Status and Progress

Basavaraja Bheemalingappa Sagar; Carsten Urbach; Dirk Pleiter; Georg von Hippel; Giannis Koutsou; Hideo Matsufuru; Hubert Simma

REVIEW 3 major objections 5 minor 26 references

International Lattice Data Grid 2.0: Status and Progress

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The International Lattice Data Grid is being rebuilt around token-based access, richer metadata, and containerised catalogues, to make lattice QCD gauge configurations shareable and FAIR-compliant.

desk verdict A candid, useful status report on the ILDG 2.0 overhaul; the one real flaw is that the token-based access claim in the conclusions is broader than what the body actually reports. read the letter →

arxiv 2502.09253 v1 pith:46A6PPF7 submitted 2025-02-13 hep-lat

Basavaraja Bheemalingappa Sagar , Georg von Hippel , Giannis Koutsou , Hideo Matsufuru , Dirk Pleiter , Hubert Simma , Carsten Urbach This is my paper · ORCID

classification hep-lat

keywords InternationalLatticeDataGridQCDgaugeconfigurationsmetadataschemataFAIRtoken-basedauthenticationOAuth2sharing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports on ILDG 2.0, the modernised version of the International Lattice Data Grid through which lattice QCD groups share gauge-configuration ensembles. It aims to establish that the federation has made major progress: new ensemble and configuration metadata schemata, an extended file format, a new user registration and single-sign-on service, re-implemented metadata and file catalogues with REST APIs, and storage access controlled by OAuth2 tokens instead of grid certificates. The point of the modernisation is to make shared data findable, accessible, interoperable, and reusable in the FAIR sense, while supporting embargoes and controlled access for collaborations. If the claims hold, the ILDG would again be a working infrastructure rather than the degraded service of the past decade.

What carries the argument

The mechanism that carries the modernisation is capability-based authorisation via OAuth2 access tokens. A token carries a scope such as storage.read:/x/y, and every storage element or catalogue service is supposed to grant access only when the path-like scope encompasses the path of the requested data object, with issuer and audience claims also checked. Around this sit the XML/XSD metadata schemata (QCDmlEnsemble and QCDmlConfig) and the LIME-packed binary file format; together they let a configuration file link to its ensemble and to the physical file via logical file names, while the token layer decides who may read or write it.

What would settle it

Take a token with scope storage.read:/x/y and try, on each storage system enrolled in the grid, to read a file addressed as /x/y/../z or /x/z, or to list the parent directory; any successful read or listing would show that the capability boundary is not enforced. A second check is to request a file from each regional grid in turn with a valid token; the first grid that cannot honour the token would falsify the claim of a single uniform access mechanism.

Watch

Extended reading notes

Core claim

The central claim is that ILDG 2.0 has moved from design to deployment: the QCDml ensemble and configuration schemata now include licences with embargo dates, funding references, ORCID-based participant identification, a wider set of gauge groups and actions, and annotation elements; the file format has been extended from single SU(3) configurations to multiple configurations, other gauge groups, and reduced storage forms; and the core services have been re-implemented around token-based authentication, with a new federated user registration, containerised REST catalogue services, and storage elements that base access decisions on the scope claim of an OAuth2 token compared with the path of the requested data object. The paper states that the European regional grid's storage elements are already token-enabled, while one regional storage system is still being adapted.

Load-bearing premise

The modernisation rests on the assumption that every participating storage element can correctly and securely enforce the rule that a token allowing access to one directory cannot reach data outside that directory, and that all regional grids will actually deploy and maintain that enforcement. If any storage system bypasses or misreads the scope claim, the claimed ILDG-wide uniform token-based access would not hold.

Editorial extensions

If this is right

A user authenticated once through a home institution can in principle reach data on any participating regional grid with a single token, without maintaining separate grid certificates.
Data producers can publish configurations immediately while restricting read access through scoped tokens, so embargoes no longer require delaying the upload.
The richer metadata, including licences and funding references, makes it possible for downstream users to know how to legally reuse an ensemble and how to acknowledge it.
Because the metadata catalogue supports multiple XML document collections with separate schemata, the same infrastructure can later be extended to other data types, such as correlation functions or analysis workflows.
Storage elements can enforce access control at directory or even individual-file granularity, which is finer than the previous certificate-based all-or-nothing model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension not pursued in the paper: measure the query latency of the re-factored catalogue for realistic ensemble sizes, since the paper itself notes that XPath searches can be slow; if quick-search keys cover the common access patterns, the service will be practical at scale.
If the scoped-token model proves robust, ILDG-style federations could adopt standard cloud object-store semantics, where short-lived tokens are passed to analysis jobs; this would make the grid look like a normal remote file system to modern compute workflows.
The same architectural pattern — XML schemata plus token-protected storage and REST catalogues — could be applied to other disciplines that share large numerical ensembles, since none of the core services depend on the physics content of the data.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Honestly, this is a proceedings status report, not a research paper, and it should be read that way. The authors have made concrete progress on the ILDG 2.0 stack: the XML schema extensions look sensible (multiple configurations per file, ORCID for participants, license/embargo elements, funding references, expanded gauge groups and actions), the catalogue services have been re-factored behind REST APIs and containerized, and user registration has moved to an INDIGO IAM instance that issues OAuth2 tokens. The paper is also unusually candid about what is not done: JLDG Gfarm token access is 'in progress,' the next steps mention 're-activation of still missing services,' and they explicitly note that some regional services degraded. That honesty is worth crediting.

The soft spot is exactly where the stress-test lands. The conclusion states that ILDG 2.0 includes 'token-based (meta-)data access' as if it were a federation-wide property. But in Section 3.3 the only concrete statement is that the four LDG storage elements (dCache and StoRM) are 'fully enabled.' JLDG is in progress, and CSSM, UKLFT, and USQCD are not mentioned at all. There are also no end-to-end test results showing that the OAuth2 scope-to-path comparison is enforced correctly and rejected outside the path. So the conclusion overstates the current state. This is a minor flaw for a status report, but it is worth flagging because the whole point of the report is to tell the community where things stand. If I were the editor, I'd ask them to add one sentence: 'token-based access is deployed on LDG, under development elsewhere.'

The paper has no equations or data, and the citation pattern is appropriate. It is a project update for the lattice QCD community and for anyone building FAIR data grids. It wouldn't merit a full journal review; as a proceedings contribution it is fine. I'd desk-reject it as a research article, but I wouldn't object to it appearing in PoS LATTICE as a report.

Referee Report

3 major / 5 minor

Summary. The paper is a proceedings contribution reporting the status and recent progress of the International Lattice Data Grid (ILDG) towards a modernised 'ILDG 2.0'. It describes extensions to the QCDml ensemble and configuration XML schemata (licenses, funding references, ORCID identification, new gauge groups and lattice actions, multi-configuration files), a revision of the ILDG file format to version 1.2, migration from grid certificates to an INDIGO IAM-based registration and single-sign-on service, re-implementation of metadata and file catalogues with REST APIs, and deployment of token-based, capability-style authorisation for storage elements following the WLCG profile. The conclusions summarise this as 'major progress' on metadata schemata, user registration, catalogue services, and token-based (meta-)data access, and list next steps such as reactivating missing regional services and migrating legacy metadata.

Significance. If the reported claims hold, ILDG 2.0 would provide a working FAIR-aligned infrastructure for sharing lattice gauge configurations across regional grids, with modern authentication and authorisation mechanisms. The paper is valuable as a status report because it documents concrete design decisions, points to specifications and repositories, and makes falsifiable claims (e.g., that LDG storage elements are fully token-enabled and that certain action types are supported by the schemata). Its limitations as an unrefereed proceedings report are the absence of test data, endpoint measurements, or external validation; the word 'progress' is appropriate, but the strength of the cross-federation 'token-based data access' claim is not yet supported by the evidence presented.

major comments (3)

[Section 3.3 and Section 4] The conclusion in Section 4 claims 'token-based (meta-)data access' as a general ILDG 2.0 achievement, but the evidence in Section 3.3 only demonstrates token-based access for the four LDG storage elements. For JLDG the text explicitly says the Gfarm token development 'is in progress', and no statement is made about CSSM, UKLFT, or USQCD storage elements. This overstates the deployment status. Please qualify the conclusion, e.g., 'token-based data access deployed on LDG, under development elsewhere', or add evidence for the remaining regional grids.
[Section 3.3] The claim that storage elements 'fully enabled for token-based access' is not supported by any reported end-to-end verification. The paper states that dCache and StoRM 'can be configured to work with capability-based authorisation', but provides no test results showing that a token with scope storage.read:/x/y is accepted for paths under /x/y and rejected for other paths, nor that issuer, audience, and expiry claims are validated by all four LDG elements. Without such evidence the uniform, safe, controlled access described in Section 3 is an assumption rather than a demonstrated property. Please add a reference to test reports, or explicitly describe the configuration checks that were performed.
[Section 3.2] Token-based authorisation for the metadata and file catalogues is described as 'implemented in an analogous way', but the deployment status is unclear. Is this feature already active in the production MDC and FC instances, or only in the re-factored code base? If it is active, which registry or federation instances use it? Please make the status explicit, since this is one of the four headline claims in the conclusions.

minor comments (5)

[Section 2.3] The text says 'web-DaV/http' in Section 3.3; this should read 'WebDAV/http'.
[References] Reference [24] contains a typo: 'INGIGO' should be 'INDIGO'.
[References] The reference list appears garbled in the version I reviewed (e.g., 'hep-lat//zero.alt32/zero.alt39121' in [1], '25/zero.alt32./zero.alt33593' in [22]). If these are author-introduced artifacts, they need correcting; if they are extraction artifacts, please ensure the published PDF has complete citation data.
[Section 2.1] The list of new action specifiers is compact and useful, but a short explanation of at least one representative new element (e.g., the C* boundary-condition photon actions) would help readers who are not members of the Metadata Working Group.
[Section 4] The outlook mentions 'massive uploads of new data' as a next step; since no ensemble has yet been uploaded under ILDG 2.0, consider stating explicitly whether any ensemble has been registered to date, as this would calibrate how much of the infrastructure is now in live use.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a status report on infrastructure modernization, with no fitted inputs, predictions, or derivation chain that reduces to its own claims.

full rationale

This is a proceedings-style status report describing the modernization of the International Lattice Data Grid (ILDG). It contains no mathematical derivation, no fitted parameters, and no quantity that is predicted from data. The central claims are factual statements about deployed services and schema revisions: a new INDIGO IAM registration service, re-factored metadata and file catalogues, updated QCDml schemata, an extended file format, and token-based access enabled on LDG storage elements. None of these claims is obtained by construction from an input, and none is asserted solely by a self-citation. The paper cites prior ILDG documents, including one with overlapping authorship (Ref. [14]), but that citation is background context, not the load-bearing justification for any stated result. The skepticism that token-based authorisation has not been demonstrated uniformly across all regional grids is a legitimate evidence-completeness concern, not a circularity concern: absence of end-to-end test data weakens the breadth of the status claim but does not make the claim equivalent to its inputs. Under the required standard, where circularity must be exhibited by a specific reduction, no such reduction exists here.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper introduces no free parameters, no mathematical axioms, and no new physical entities. The two domain assumptions listed above are load-bearing for the migration and deployment claims.

assumptions (2)

domain assumption The LIME-based file format v1.2 remains backward compatible with existing ILDG binary files.
Stated in Section 2.3 without proof; if false, legacy configurations could not be read after migration.
domain assumption The token-based authorization model is correctly enforceable on all regional storage elements.
Assumed in Section 3.3 based on dCache/StoRM configurability; no test results or security analysis are provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of International Lattice Data Grid 2.0: Status and Progress." pith.science (2026). https://pith.science/paper/46A6PPF7

@misc{pith2026250209253,
  author       = {Pith},
  title        = {Pith review of: International Lattice Data Grid 2.0: Status and Progress},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/46A6PPF7}},
  note         = {Machine review of arXiv:2502.09253}
}

read the original abstract

In this proceeding contribution we discuss the status and progress towards a modernised and extended International Lattice Data Grid (ILDG), which has seen major developments, updates, and upgrades over the last year. In particular, metadata and file schemata have been extended. Moreover, the registration and authentication services have been modernised, and the file and metadata catalogues re-implemented.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 22 canonical work pages

[1]

UKQCD collaboration, International lattice data grid , Nucl. Phys. B Proc. Suppl. 119 (2003) 225 [hep-lat//zero.alt32/zero.alt39121]

work page 2003
[2]

Irving, R.D

A.C. Irving, R.D. Kenway, C.M. Maynard and T. Y oshie, Progress in building an International Lattice Data Grid , Nucl. Phys. B Proc. Suppl. 129 (2004) 159 [hep-lat//zero.alt33/zero.alt39/zero.alt329]

work page 2004
[3]

Ukawa, Status of international lattice data grid: An Overview , Nucl

A. Ukawa, Status of international lattice data grid: An Overview , Nucl. Phys. B Proc. Suppl. 140 (2005) 207 [hep-lat//zero.alt34/zero.alt39/zero.alt384]. 8 International Lattice Data Grid 2.0: Status and Progress Carsten Urbach

work page 2005
[4]

Beckett, B

M.G. Beckett, B. Joo, C.M. Maynard, D. Pleiter, O. Tatebe and T. Y oshie,Building the International Lattice Data Grid , Comput. Phys. Commun. 182 (2011) 1208 [/zero.alt391/zero.alt3.1692]

work page 2011
[5]

International Lattice Data Grid

ILDG, “International Lattice Data Grid.” https://hpc.desy.de/ildg/organization/
[6]

Centre for the Subatomic Structure of Matter

CSSM, “Centre for the Subatomic Structure of Matter.” https://set.adelaide.edu.au/physics-chemistry-earth -sciences/our-research/physics/nucle

work page
[7]

Japan Lattice Data Grid

JLDG, “Japan Lattice Data Grid.” https://www.jldg.org/
[8]

Latfor DataGrid

LDG, “Latfor DataGrid.” https://hpc.desy.de/ldg/

work page

Show all 26 references

[9]

UK Lattice Field Theory

UKLFT, “UK Lattice Field Theory.” http://generic.wordpress.soton.ac.uk/uklft/
[10]

US Lattice Quantum Chromodynamics

USQCD, “US Lattice Quantum Chromodynamics.” https://www.usqcd.org/
[11]

Wilkinson et al., The FAIR Guiding Principles for scientiﬁc data management a nd stewardship, Sci

M.D. Wilkinson et al., The FAIR Guiding Principles for scientiﬁc data management a nd stewardship, Sci. Data 3 (2016) 160018

2016
[12]

Nationale Forschungsdaten Infrastruktur

NFDI, “Nationale Forschungsdaten Infrastruktur.” https://www.nfdi.de/
[13]

Particles, Universe, NuClei and Hadrons for the NFDI

PUNCH4NFDI consortium, “Particles, Universe, NuClei and Hadrons for the NFDI.” https://www.punch4nfdi.de
[14]

Karsch, H

F. Karsch, H. Simma and T. Y oshie, The International Lattice Data Grid – towards FAIR data, PoS LATTICE2022 (2023) 244 [2212./zero.alt38392]

2023
[15]

Indigo Identity Access Management

Indigo IAM, “Indigo Identity Access Management.” https://indigo-iam.github.io/v/current/
[16]

World Wide Web Consortium Standards

W3C, “World Wide Web Consortium Standards.” https://www.w3.org/TR/
[17]

gitlab repository of ﬁle format speciﬁcation

ILDG Metadata Working Group, “gitlab repository of ﬁle format speciﬁcation.” https://gitlab.desy.de/ildg/mdwg/file-format/
[18]

gitlab repository of QCD ml schema and examples

ILDG Metadata Working Group, “gitlab repository of QCD ml schema and examples.” https://gitlab.desy.de/ildg/mdwg/qcdml/
[19]

speciﬁcations

ILDG, “speciﬁcations.” https://hpc.desy.de/ildg/specifications/
[20]

DataCite Metadata Schema

DataCite, “DataCite Metadata Schema.” https://schema.datacite.org/
[21]

USQCD, “C-LIME.” https://usqcd-software.github.io/c-lime/
[22]

Bennett, A

E. Bennett, A. Athenodorou, L. Chisholm, A. Hasenfratz and C. Urbach, Reproducibility and Open Science in Lattice Quantum Field Theory , PoS LATTICE2024 (2025) 023 [25/zero.alt32./zero.alt33593]

2025
[23]

attribute release check

eduGAIN, “attribute release check.” https://release-check.edugain.org/. 9 International Lattice Data Grid 2.0: Status and Progress Carsten Urbach
[24]

ILDG INGIGO IAM user registration

“ILDG INGIGO IAM user registration.” https://iam-ildg.cloud.cnaf.infn.it/
[25]

Pederiva et al., Enhanced Data Management Tools for ILDG and PUNCH4NFDI , PoS LATTICE2024 (2025) 450

G. Pederiva et al., Enhanced Data Management Tools for ILDG and PUNCH4NFDI , PoS LATTICE2024 (2025) 450

2025
[26]

WLCG common JWT proﬁles

WLCG AuthZ working group, “WLCG common JWT proﬁles.” https://zenodo.org/record/346/zero.alt3258. 10

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

UKQCD collaboration, International lattice data grid , Nucl. Phys. B Proc. Suppl. 119 (2003) 225 [hep-lat//zero.alt32/zero.alt39121]

work page 2003

[2] [2]

Irving, R.D

A.C. Irving, R.D. Kenway, C.M. Maynard and T. Y oshie, Progress in building an International Lattice Data Grid , Nucl. Phys. B Proc. Suppl. 129 (2004) 159 [hep-lat//zero.alt33/zero.alt39/zero.alt329]

work page 2004

[3] [3]

Ukawa, Status of international lattice data grid: An Overview , Nucl

A. Ukawa, Status of international lattice data grid: An Overview , Nucl. Phys. B Proc. Suppl. 140 (2005) 207 [hep-lat//zero.alt34/zero.alt39/zero.alt384]. 8 International Lattice Data Grid 2.0: Status and Progress Carsten Urbach

work page 2005

[4] [4]

Beckett, B

M.G. Beckett, B. Joo, C.M. Maynard, D. Pleiter, O. Tatebe and T. Y oshie,Building the International Lattice Data Grid , Comput. Phys. Commun. 182 (2011) 1208 [/zero.alt391/zero.alt3.1692]

work page 2011

[5] [5]

International Lattice Data Grid

ILDG, “International Lattice Data Grid.” https://hpc.desy.de/ildg/organization/

[6] [6]

Centre for the Subatomic Structure of Matter

CSSM, “Centre for the Subatomic Structure of Matter.” https://set.adelaide.edu.au/physics-chemistry-earth -sciences/our-research/physics/nucle

work page

[7] [7]

Japan Lattice Data Grid

JLDG, “Japan Lattice Data Grid.” https://www.jldg.org/

[8] [8]

Latfor DataGrid

LDG, “Latfor DataGrid.” https://hpc.desy.de/ldg/

work page