Pith. sign in

REVIEW 3 major objections 5 minor 25 references

Towards Sustainability Model Cards

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sustainability Model Cards introduce a DSL that makes ML energy, carbon, and water reporting machine-readable and automatically processable.

desk verdict Useful DSL for sustainability model cards; the 'formal enough' claim is undercut by a validation rule that contradicts the paper's own canonical example. read the letter →

arxiv 2507.19559 v1 pith:T3236YLR submitted 2025-07-25 cs.CY cs.AIcs.LG

Gwendal Jouneaux , Jordi Cabot This is my paper · ORCID
classification cs.CYcs.AIcs.LG
keywords AIModelsModelCardsSustainabilityEnergyQualityDomain-SpecificLanguageGreenMLOps
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that sustainability reporting for machine-learning models is still too informal for automation: current reports and energy benchmarks provide numbers but no machine-checkable structure, so sustainability information cannot be automatically compared, selected, or certified. It proposes a Domain-Specific Language (DSL) whose schema fixes the concepts a sustainability report must contain—metadata, training, inference tasks, and platform—and ties each computation to energy, carbon, and water metrics. Cards written in the language are YAML documents that a validating parser checks before they are exported in the familiar model-card format. If the language delivers the promised precision, sustainability data becomes a first-class input to model comparison, deployment optimization, and service-level agreements.

What carries the argument

The mechanism that carries the argument is the metamodel of the DSL: the schema defining which concepts a Sustainability Model Card can contain and how they relate. Its load-bearing move is the Computation abstraction, from which both Training and each inference Task inherit, so that every environmentally impactful computation is described with the same energy, carbon, and water metrics plus a timestamp and a platform reference. The YAML concrete syntax maps these classes to nested key-value structures, and the validating parser checks the YAML against the metamodel before instantiating it in Python. That chain—schema, serialization, validation, instantiation—is what turns a narrative report into data.

What would settle it

Run the provided parser on the paper's own example card: if a 100.0 percent carbon-offset credit is accepted despite the stated [0,1] bound for percentages, or rejected when the example intends it, the validation behavior is not as specified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the sustainability aspects of an ML model can be captured in a formal, machine-readable language instead of free-text or loosely structured benchmark reports. It defines Sustainability Model Cards as a YAML-based DSL with a metamodel whose root card contains metadata, training, and inference components; training and inference share a Computation abstraction that carries energy consumption, carbon emissions, water consumption, a timestamp, and a link to the platform used. A validating parser enforces unit presence and correctness, allowed inference and energy types, and value bounds, then builds an in-memory instance of the metamodel so that any other tool can process the card automatically. This makes the card simultaneously a human-readable report and a formal input to automated analysis.

Load-bearing premise

The claim of precision rests on the language's schema and validation rules being internally consistent and exactly implemented; if the schema, the example, and the parser disagree, the cards are not yet the machine-checkable artifact described.

Editorial extensions

If this is right

  • Sustainability cards can be validated automatically, catching malformed or unit-inconsistent reports before they enter a pipeline.
  • Model comparison and selection can rank candidates by environmental impact without manual reading of prose reports.
  • Platform details such as region, hardware, energy mix, and offsets can be reused across training and inference entries, enabling infrastructure-aware analysis.
  • The same formal card can be exported in the familiar model-card format for human readers and used as machine input for MLOps workflows.
  • Future service-level agreements can monitor and enforce sustainability commitments based on card data at runtime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Adoption of the DSL will require a shared measurement protocol; the paper defines fields and units but not how to measure, so the same model could yield different cards depending on the tool used.
  • The stated validation bound for percentages and the example's 100.0 percent offset credit would need reconciliation, suggesting the validation suite needs explicit tests and a decision on whether percentages are fractions or whole percents.
  • A concrete demonstration would be to generate cards for a handful of openly available models and run cross-model queries for deployment decisions; the paper sets this as future work but does not run it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes Sustainability Model Cards, a domain-specific language (DSL) for describing sustainability-related information about machine learning models, including training and inference energy consumption, carbon emissions, water consumption, and platform details. The DSL is given an abstract syntax as a metamodel, a YAML-based concrete syntax with an illustrative example, and a Python validating parser generated with the authors' BESSER platform. The authors claim the DSL is 'formal enough to be input of any other model description automatic process' and outline a research roadmap for automatic model selection, deployment optimization, and SLA enforcement. The paper does not include an empirical evaluation on real models; it is a design proposal with a reference implementation.

Significance. If the DSL were made internally consistent, this proposal would fill a real gap: existing model reporting initiatives (Model Cards, Dataset Cards, AI Energy Score) do not provide a machine-readable, validated format for sustainability information. The paper connects quality models, Model Cards, and Green AI, and its open-source parser and metamodel are concrete artifacts that can be reused. The design choice of YAML facilitates adoption and integration with Hugging Face cards. However, the paper's contribution is currently a design rather than a validated result, and the internal inconsistencies in the specification prevent the central 'formal enough' claim from being established.

major comments (3)
  1. [Section 4.3, Listing 1] The validation rule stated in Section 4.3, item (4), requires that 'values representing percentage are bound to the [0,1] interval,' yet Listing 1, presented as a 'Syntax example using all the metamodel concepts,' contains a carbon_offset_credit value of 100.0 with unit PERCENTAGE and an energy_mix ratio of 100.0. If the parser enforces the [0,1] bound, Listing 1 is invalid; if it accepts 100.0 as a percentage, then the stated rule is not implemented. Either way, the specification and the canonical artifact contradict each other, and external tooling cannot rely on the DSL as specified.
  2. [Section 4.1 vs Section 3 and Listing 1] Section 3 describes the Sustainability Model Card as having four main sections—Metadata, Training, Inference, and Platform—and Listing 1 indeed contains top-level 'platforms' and 'energy_sources' keys. Section 4.1, however, states that the root SustainabilityModelCard 'is composed of three subcomponents: MetaData, Training and Inference,' and Section 4.2 calls Platform and EnergySource a 'special case.' These statements are mutually incompatible. The metamodel in Figure 1 is the authoritative abstract syntax, and the text does not state whether Platform is a component of the root class or an associated external class. This ambiguity directly affects the validation logic and any tool that interprets the schema.
  3. [Abstract and Section 5] The paper's central claim is that the DSL is 'formal enough to be input of any other model description automatic process,' and Section 5 lists automatic model selection, deployment optimization, and SLA enforcement as scenarios. However, the paper provides no evaluation or demonstration on any real model; the only example is the synthetic YAML snippet in Listing 1, which itself conflicts with the stated validation rule. At least one end-to-end case study on a publicly available model card (with the parser run on it) is needed to substantiate the claim that the DSL is usable for automatic processing in MLOps pipelines.
minor comments (5)
  1. [Section 5] There is a typo 'aspeect' that should be 'aspect.'
  2. [Section 6] The conclusion refers to the 'Models Card formalism' where 'Model Cards' is the intended term.
  3. [Throughout] The paper inconsistently uses 'MetaData' (Section 4.1) and 'Metadata' (Section 3); please unify the spelling.
  4. [Section 4.3] The validation rule on percentage bounds should clarify whether values are expressed as fractions (0 to 1) or as percentage points (0 to 100); this would remove the apparent contradiction with Listing 1.
  5. [Section 4.2] The paper should state explicitly whether YAML key order is significant; the parser behavior on reordered keys is currently unspecified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a design proposal whose DSL is assembled from external reporting standards and author-defined metamodel concepts; the BESSER self-citation is a tooling choice, not a load-bearing derivation.

full rationale

This paper does not present an empirical derivation or a fitted prediction; its contribution is a proposed Domain-Specific Language for sustainability model cards. The language's content is assembled from external sources: Model Cards [3], AI Energy Score [4], Hugging Face sustainability reporting [14], Strubell et al. [1], and other surveyed work cited in Sections 2 and 5. The metamodel is defined directly in Section 4.1, and the concrete syntax is specified by explicit rules in Section 4.2. The authors' use of their own BESSER platform [18] to generate Python classes is a tooling choice: the DSL's syntax and validation behavior are stated in the paper independently of BESSER, and no claim in the paper reduces by construction to a BESSER output or to any self-cited theorem. The paper's key claim, that the DSL is 'formal enough to be input of any other model description automatic process,' is supported by the metamodel and parser description rather than by self-citation. The main risks are internal inconsistencies, notably Section 4.3's statement that percentage values must be bound to the [0,1] interval while Listing 1 uses values such as 100.0 for carbon_offset_credit and energy_mix ratio, and Section 4.1's three-subcomponent root card versus Section 3's four main sections including Platform. These are correctness and precision concerns, not circularity. No circular step meets the required evidentiary standard.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

The paper introduces no empirical parameters, but relies on the assumption that sustainability is reducible to the chosen metrics and that a YAML-based DSL plus parser yields formal descriptions.

assumptions (4)
  • domain assumption Sustainability of an ML model can be captured by the listed dimensions: energy consumption, carbon emissions, water consumption, training duration, inference task metrics, and platform details.
    Section 3 defines these as the four sections of a Sustainability Model Card based on a union of surveyed papers, but this choice is not validated against an external standard or community consensus.
  • domain assumption YAML is a suitable concrete syntax for integration with existing Hugging Face Model Cards and for automatic processing.
    Section 4.2 chooses YAML; the paper offers usability and integration arguments but no empirical comparison with alternative notations.
  • domain assumption A metamodel plus a validating Python parser is sufficient to guarantee 'formal' descriptions usable by other automatic processes.
    Section 4.3 describes validation checks but does not establish semantic equivalence or completeness of the validation; no test suite or formal verification is provided.
  • domain assumption The surveyed literature provides a complete enough union of sustainability concepts.
    Section 5 states the DSL was made by taking the union of concepts in refs [1,21,14,4,22,16,23,24,17], but the field is rapidly evolving and standardization is unresolved.
invented entities (1)
  • SustainabilityModelCard DSL
    purpose: Formal description language for ML model sustainability reporting, extending Model Cards with training, inference, platform, water, and carbon offset data.
    The DSL is the paper's contribution; it has no external benchmark or falsifiable prediction outside the proposed artifacts, so independent evidence is absent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Sustainability Model Cards." pith.science (2026). https://pith.science/paper/T3236YLR

@misc{pith2026250719559,
  author       = {Pith},
  title        = {Pith review of: Towards Sustainability Model Cards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T3236YLR}},
  note         = {Machine review of arXiv:2507.19559}
}
read the original abstract

The growth of machine learning (ML) models and associated datasets triggers a consequent dramatic increase in energy costs for the use and training of these models. In the current context of environmental awareness and global sustainability concerns involving ICT, Green AI is becoming an important research topic. Initiatives like the AI Energy Score Ratings are a good example. Nevertheless, these benchmarking attempts are still to be integrated with existing work on Quality Models and Service-Level Agreements common in other, more mature, ICT subfields. This limits the (automatic) analysis of this model energy descriptions and their use in (semi)automatic model comparison, selection, and certification processes. We aim to leverage the concept of quality models and merge it with existing ML model reporting initiatives and Green/Frugal AI proposals to formalize a Sustainable Quality Model for AI/ML models. As a first step, we propose a new Domain-Specific Language to precisely define the sustainability aspects of an ML model (including the energy costs for its different tasks). This information can then be exported as an extended version of the well-known Model Cards initiative while, at the same time, being formal enough to be input of any other model description automatic process.

Figures

Figures reproduced from arXiv: 2507.19559 by the authors.

Figure 1
Figure 1. Metamodel of the Sustainability Model Cards DSL The MetaData class represents the metadata section of the card defining the name, version, model type, provider, and license as strings. The Training class models the training section of the card and defines the duration of the training. The Inference class encompasses all the inference tasks (represented by the Task class) addressed by the model. Each Task defines the… view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

  1. [1]

    Strubell, A

    E. Strubell, A. Ganesh, A. McCallum, Energy and policy considerations for deep learning in NLP, in: A. Korhonen, D. Traum, L. Màrquez (Eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 3645–3650. doi:10.18653/v1/P19-1355

    doi:10.18653/v1/p19-1355 2019

  2. [2]

    Verdecchia, J

    R. Verdecchia, J. Sallou, L. Cruz, A systematic review of green ai, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 13 (2023) e1507

    work page 2023

  3. [3]

    Mitchell, S

    M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, T. Gebru, Model cards for model reporting, in: Proceedings of the conference on fairness, accountability, and transparency, 2019, pp. 220–229

    work page 2019

  4. [4]

    URL: https://huggingface.github.io/AIEnergyScore, [Online; accessed 26

    HuggingFace, AI Energy Score, 2025. URL: https://huggingface.github.io/AIEnergyScore, [Online; accessed 26. May 2025]

    work page 2025

  5. [5]

    Felderer, R

    M. Felderer, R. Ramler, Quality assurance for ai-based systems: Overview and challenges, in: Software Quality: Future Perspectives on Software Engineering Quality: 13th International Con- ference, SWQD 2021, Vienna, Austria, January 19–21, 2021, Proceedings 13, Springer, 2021, pp. 33–42

    work page 2021

  6. [6]

    Gezici, A

    B. Gezici, A. K. Tarhan, Systematic literature review on software quality for ai-based software, Empirical Software Engineering 27 (2022) 66

    work page 2022

  7. [7]

    M. A. Ali, N. K. Yap, A. A. A. Ghani, H. Zulzalil, N. I. Admodisastro, A. A. Najafabadi, A systematic mapping of quality models for ai systems, software and components, Applied Sciences 12 (2022) 8700

    work page 2022

  8. [8]

    L. Pons, I. Ozkaya, Priority quality attributes for engineering ai-enabled systems, arXiv preprint arXiv:1911.02912 (2019)

    work page Pith review arXiv 2019

Show all 25 references
  1. [9]

    Siebert, L

    J. Siebert, L. Joeckel, J. Heidrich, K. Nakamichi, K. Ohashi, I. Namba, R. Yamamoto, M. Aoyama, Towards guidelines for assessing qualities of machine learning systems, in: Quality of Information and Communications Technology: 13th International Conference, QUATIC 2020, Faro, P...

    2020

  2. [10]

    Horkoff, Non-functional requirements for machine learning: Challenges and new directions, in: 2019 IEEE 27th international requirements engineering conference (RE), IEEE, 2019, pp

    J. Horkoff, Non-functional requirements for machine learning: Challenges and new directions, in: 2019 IEEE 27th international requirements engineering conference (RE), IEEE, 2019, pp. 386–391

    2019

  3. [11]

    URL: https://huggingface.co/docs/hub/en/datasets-cards, [On- line; accessed 26

    HuggingFace, Dataset Cards, 2025. URL: https://huggingface.co/docs/hub/en/datasets-cards, [On- line; accessed 26. May 2025]

    2025

  4. [12]

    Giner-Miguelez, A

    J. Giner-Miguelez, A. Gómez, J. Cabot, A domain-specific language for describing machine learning datasets, Journal of Computer Languages 76 (2023) 101209

    2023

  5. [13]

    Akhtar, O

    M. Akhtar, O. Benjelloun, C. Conforti, L. Foschini, J. Giner-Miguelez, P. Gijsbers, S. Goswami, N. Jain, M. Karamousadakis, M. Kuchnik, et al., Croissant: A metadata format for ml-ready datasets, Advances in Neural Information Processing Systems 37 (2024) 82133–82148

    2024

  6. [14]

    URL: https: //huggingface.co/blog/carbon-emissions-on-the-hub, [Online; accessed 26

    HuggingFace, CO2 Emissions and the Hugging Face Hub: Leading the Charge, 2022. URL: https: //huggingface.co/blog/carbon-emissions-on-the-hub, [Online; accessed 26. May 2025]

    2022

  7. [15]

    L. Cruz, J. P. Fernandes, M. H. Kirkeby, S. Martínez-Fernández, J. Sallou, H. Anwar, E. B. Roque, J. Bogner, J. Castaño, F. Castor, A. Chasmawala, S. Cunha, D. Feitosa, A. González, A. Jedlitschka, P. Lago, H. Muccini, A. Oprescu, P. Rani, J. Saraiva, F. Sarro, R. Selvan, K. V...

    2025 arXiv

  8. [16]

    Yu, C.-H

    J.-R. Yu, C.-H. Chen, T.-W. Huang, J.-J. Lu, C.-R. Chung, T.-W. Lin, M.-H. Wu, Y.-J. Tseng, H.-Y. Wang, Energy efficiency of inference algorithms for clinical laboratory data sets: green artificial intelligence study, Journal of Medical Internet Research 24 (2022) e28036

    2022

  9. [17]

    Ristic, K

    B. Ristic, K. Madani, Z. Makuch, The water footprint of data centers, Sustainability 7 (2015) 11260–11284

    2015

  10. [18]

    Alfonso, A

    I. Alfonso, A. Conrardy, A. Sulejmani, A. Nirumand, F. Ul Haq, M. Gomez-Vazquez, J.-S. Sottet, J. Cabot, Building BESSER: an open-source low-code platform, in: International Conference on Business Process Modeling, Development and Support, Springer, 2024, pp. 203–212

    2024

  11. [19]

    Daoudi, I

    N. Daoudi, I. Alfonso, J. Cabot, Modelling neural network models, in: International Conference on Research Challenges in Information Science, Springer, 2025, pp. 130–139

    2025

  12. [20]

    Ul Haq, I

    F. Ul Haq, I. Alfonso, A. Sulejmani, J. Cabot, Extending a low-code tool with multi-cloud deployment capabilities, in: European Conference on Software Architecture, Springer, 2024, pp. 39–46

    2024

  13. [21]

    Strubell, A

    E. Strubell, A. Ganesh, A. McCallum, Energy and policy considerations for modern deep learning research, in: Proceedings of the AAAI conference on artificial intelligence, volume 34, 2020, pp. 13693–13696

    2020

  14. [22]

    Guldner, S

    A. Guldner, S. Kreten, S. Naumann, Exploration and systematic assessment of the resource efficiency of machine learning, in: INFORMATIK 2021, Gesellschaft für Informatik, Bonn, 2021, pp. 287–299

    2021

  15. [23]

    L. F. W. Anthony, B. Kanding, R. Selvan, Carbontracker: Tracking and predicting the carbon footprint of training deep learning models, arXiv preprint arXiv:2007.03051 (2020)

    2020 arXiv

  16. [24]

    García-Martín, C

    E. García-Martín, C. F. Rodrigues, G. Riley, H. Grahn, Estimation of energy consumption in machine learning, Journal of Parallel and Distributed Computing 134 (2019) 75–88

    2019

  17. [25]

    A. S. Luccioni, G. Pistilli, R. Sefala, N. Moorosi, Bridging the gap: Integrating ethics and envi- ronmental sustainability in ai research and practice, 2025. URL: https://arxiv.org/abs/2504.00797. arXiv:2504.00797

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.