pith. sign in

arxiv: 2512.11868 · v2 · submitted 2025-12-05 · 💻 cs.CY · cs.AI

Industrial AI Robustness Card for Time Series Models

Alexander Windmann , Benedikt Stratmann , Mariya Lyashenko , Oliver Niggemann This is my paper

Pith reviewed 2026-05-17 00:46 UTC · model grok-4.3

classification 💻 cs.CY cs.AI
keywords robustness cardtime series modelsindustrial AIEU AI Actdrift monitoringuncertainty quantificationstress testingregulatory compliance
0
0 comments X

The pith

A new protocol called IARC-TS gives industrial practitioners concrete steps to document and test the robustness of time series AI models against regulatory requirements.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the Industrial AI Robustness Card for Time Series (IARC-TS) as a practical way to address vague robustness rules in regulations. It defines specific fields to fill and an empirical protocol that includes monitoring for drift and operational domains, quantifying uncertainty, and running stress tests. These are then linked to parts of the EU AI Act that cover documentation, testing, and ongoing monitoring. The approach is shown working in a biopharmaceutical soft sensor example, where it helps create reproducible evidence and sets up triggers for when to check the model again.

Core claim

IARC-TS specifies required fields and an empirical measurement and reporting protocol that combines drift and operational domain monitoring, uncertainty quantification, and stress tests, and maps these to selected EU AI Act documentation, testing, and monitoring obligations.

What carries the argument

The IARC-TS card, which is a structured template of documentation fields paired with a measurement protocol that generates evidence through monitoring, uncertainty estimates, and stress testing.

If this is right

  • If adopted, practitioners can produce documented evidence that meets selected regulatory obligations for AI systems in industry.
  • The protocol supports reproducible robustness evaluations for time series models in operational settings.
  • Monitoring triggers can be established based on the measured drift, uncertainty, and stress test results.
  • Case studies like the biopharmaceutical soft sensor demonstrate how the card turns abstract rules into actionable checks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar cards could be developed for other AI application domains such as computer vision or natural language processing.
  • The protocol might integrate with existing industrial monitoring systems to automate parts of the robustness checks.
  • Generalization beyond the biopharmaceutical case would require testing in additional industrial time series scenarios like manufacturing or energy.

Load-bearing premise

That the selected elements of monitoring, uncertainty quantification, and stress tests together provide enough evidence to fulfill the relevant regulatory obligations.

What would settle it

A review or audit showing that the evidence generated by following the IARC-TS protocol is insufficient to satisfy the documentation, testing, or monitoring requirements under the EU AI Act.

Figures

Figures reproduced from arXiv: 2512.11868 by Alexander Windmann, Benedikt Stratmann, Mariya Lyashenko, Oliver Niggemann.

Figure 1
Figure 1. Figure 1: Example IARC for the IndPenSim dataset. Note that some fields and plots are hidden due to space constraints. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
read the original abstract

Industrial AI practitioners face vague robustness requirements in emerging regulations and standards but lack concrete, implementation-ready protocols. This paper introduces the Industrial AI Robustness Card for Time Series (IARC-TS), a lightweight protocol for documenting and evaluating industrial time series models. IARC-TS specifies required fields and an empirical measurement and reporting protocol that combines drift and operational domain monitoring, uncertainty quantification, and stress tests, and maps these to selected EU AI Act documentation, testing, and monitoring obligations. A biopharmaceutical soft sensor case study illustrates how IARC-TS supports reproducible robustness evidence and defines monitoring triggers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces the Industrial AI Robustness Card for Time Series (IARC-TS), a lightweight protocol specifying required fields and an empirical measurement/reporting protocol that combines drift and operational domain monitoring, uncertainty quantification, and stress tests. It maps these elements to selected EU AI Act documentation, testing, and monitoring obligations and illustrates the approach with a biopharmaceutical soft-sensor case study that claims to support reproducible robustness evidence and define monitoring triggers.

Significance. If the protocol proves sufficient for regulatory compliance, IARC-TS could fill a practical gap for industrial AI practitioners facing vague robustness requirements in regulations such as the EU AI Act. The structured combination of monitoring, UQ, and stress testing offers a concrete documentation template that might improve reproducibility and auditability. Its significance remains provisional, however, because the manuscript provides no comparative evaluation, cross-domain testing, or external validation of the mapping.

major comments (2)
  1. [Case Study] The central claim that IARC-TS generates evidence sufficient to support compliance with selected EU AI Act obligations rests on a single biopharmaceutical soft-sensor case study. No quantitative metrics, error bars, baseline comparisons, or cross-validation across additional industrial time-series domains are reported, leaving the sufficiency of the chosen monitoring+UQ+stress-test combination untested.
  2. [Mapping to EU AI Act] The mapping from IARC-TS fields to EU AI Act obligations is asserted without documented review by legal or regulatory experts and without demonstration that the selected elements close coverage gaps for other obligations or domains. This makes the regulatory utility claim load-bearing yet unsupported beyond description.
minor comments (2)
  1. [Introduction] Clarify whether the protocol is intended as a minimal checklist or a prescriptive standard; the current wording leaves this ambiguous for practitioners.
  2. [Protocol Definition] Add explicit reproducibility instructions (e.g., data splits, seed values, or public repository references) to the reporting fields so that the claimed 'reproducible robustness evidence' can be verified.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We provide point-by-point responses to the major comments below and describe the revisions we plan to make.

read point-by-point responses

  1. Referee: [Case Study] The central claim that IARC-TS generates evidence sufficient to support compliance with selected EU AI Act obligations rests on a single biopharmaceutical soft-sensor case study. No quantitative metrics, error bars, baseline comparisons, or cross-validation across additional industrial time-series domains are reported, leaving the sufficiency of the chosen monitoring+UQ+stress-test combination untested.

    Authors: We agree that the case study is based on a single domain and does not provide comparative evaluations or cross-domain validation. The manuscript positions the biopharmaceutical soft-sensor example as an illustration of how the IARC-TS protocol can be applied to generate reproducible robustness evidence and define monitoring triggers, rather than as a comprehensive validation of the protocol's sufficiency for regulatory compliance. To strengthen the presentation, we will revise the case study section to include more specific quantitative details on the metrics, such as the drift scores, uncertainty intervals, and stress test outcomes observed, along with error bars where applicable. We will also explicitly note the limitations regarding generalizability and suggest that future work should include multi-domain evaluations. This addresses the concern while maintaining the paper's focus on protocol definition. revision: partial

  2. Referee: [Mapping to EU AI Act] The mapping from IARC-TS fields to EU AI Act obligations is asserted without documented review by legal or regulatory experts and without demonstration that the selected elements close coverage gaps for other obligations or domains. This makes the regulatory utility claim load-bearing yet unsupported beyond description.

    Authors: We acknowledge that the mapping is derived from our technical analysis of the EU AI Act without consultation with legal or regulatory experts. The paper selects particular obligations concerning documentation, testing, and monitoring for high-risk AI systems and maps IARC-TS components to them as a proposed approach for industrial practitioners. We do not claim that this mapping is exhaustive or legally validated. In the revised manuscript, we will clarify the scope by stating that the mapping represents a technical proposal based on the Act's text and will recommend seeking expert legal advice for actual compliance. We will also avoid overclaiming regulatory utility and emphasize that IARC-TS provides a practical template for selected aspects. revision: yes

Circularity Check

0 steps flagged

No circularity: IARC-TS is an original definitional protocol

full rationale

The paper defines the IARC-TS protocol by specifying required documentation fields and an empirical reporting procedure that combines drift/operational-domain monitoring, uncertainty quantification, and stress tests, then maps the resulting evidence to selected EU AI Act obligations. This is a constructive specification rather than a derivation from equations or prior results. No fitted parameters are relabeled as predictions, no self-citation chain is invoked to justify uniqueness or an ansatz, and the single biopharmaceutical case study functions only as an illustration of how the protocol is applied, not as input data from which the protocol itself is derived. The contribution is therefore self-contained as a new framework.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the assumption that the listed robustness components are both necessary and sufficient for regulatory compliance and that a single case study demonstrates general utility; no free parameters or invented physical entities are described.

axioms (1)
  • domain assumption The selected tests and monitoring methods are sufficient to meet selected EU AI Act obligations.
    The abstract states that the protocol maps to documentation, testing, and monitoring obligations without providing independent justification for completeness.
invented entities (1)
  • IARC-TS no independent evidence
    purpose: Lightweight protocol for documenting and evaluating robustness of industrial time series models
    Newly introduced framework whose value depends on adoption and further validation.

pith-pipeline@v0.9.0 · 5394 in / 1367 out tokens · 36557 ms · 2026-05-17T00:46:23.439716+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    IARC-TS specifies required fields and an empirical measurement and reporting protocol that combines drift and operational domain monitoring, uncertainty quantification, and stress tests, and maps these to selected EU AI Act documentation, testing, and monitoring obligations.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

  1. [1]

    doi: 10.1147/JRD.2019.2942288

    ISSN 0018-8646. doi: 10.1147/JRD.2019.2942288. Rob Ashmore, Radu Calinescu, and Colin Paterson. Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges.ACM Computing Surveys, 54(5):1–39, May

    work page doi:10.1147/jrd.2019.2942288 2019

  2. [2]

    doi: 10.1145/3453444

    ISSN 0360-0300, 1557-7341. doi: 10.1145/3453444. BCG. Using AI in Industrial Operations Guidebook. Technical report,

    work page doi:10.1145/3453444

  3. [3]

    Merging (EU)-Regulation and Model Reporting

    Danilo Brajovic, Vincent Philipp Göbels, Janika Kutz, and Marco Huber. Merging (EU)-Regulation and Model Reporting. InNeurIPS 2023 Workshop on Regulatable ML, December

    work page 2023

  4. [4]

    doi: 10.1016/j.inffus.2023.101896

    ISSN 1566-2535. doi: 10.1016/j.inffus.2023.101896. DIN SPEC 92001-2. DIN SPEC 92001-2 - Artificial Intelligence - Life Cycle Processes and Quality Requirements - Part 2: Robustness, December

    work page doi:10.1016/j.inffus.2023.101896 2023

  5. [5]

    doi: 10.1016/j.eng.2021.03.019

    ISSN 20958099. doi: 10.1016/j.eng.2021.03.019. EASA. EASA Artificial Intelligence (AI) Concept Paper Issue 2: Guidance for Level 1&2 machine learning applications. Technical report, April

    work page doi:10.1016/j.eng.2021.03.019 2021

  6. [6]

    EU AI Act. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial In...

    work page 2024

  7. [7]

    Gawlikowski, C

    ISSN 1573-7462. doi: 10.1007/s10462-023-10562-9. Stephen Goldrick, Carlos A. Duran-Villalobos, Karolis Jankauskas, David Lovett, Suzanne S. Farid, and Barry Lennox. Modern day monitoring and control challenges outlined on an industrial-scale benchmark fermentation process.Computers & Chemical Engineering, 130:106471,

    work page doi:10.1007/s10462-023-10562-9

  8. [8]

    gradient descent

    ISSN 0098-1354. doi: https://doi.org/10.1016/j. compchemeng.2019.05.037. Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization. In2021 IEEE/...

    work page doi:10.1016/j 2019

  9. [9]

    Walk in the cloud: Learning curves for point clouds shape analysis, pp

    IEEE. ISBN 978-1-6654-2812-5. doi: 10.1109/ICCV48922.2021.00823. Ben Hutchinson, Andrew Smart, Alex Hanna, Remi Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure. InProceedings of the 2021 ACM Conference on Fairness,...

    work page doi:10.1109/iccv48922.2021.00823 2021

  10. [10]

    Fair Classification with Group-Dependent Label Noise

    ACM. ISBN 978-1-4503-8309-7. doi: 10.1145/3442188.3445918. 6 IEEE 3129-2023. IEEE 3129-2023, June

    work page doi:10.1145/3442188.3445918 2023

  11. [11]

    ISO/IEC 23894:2023 - Information technology — Artificial intelligence — Guidance on risk management,

    ISO/IEC 23894. ISO/IEC 23894:2023 - Information technology — Artificial intelligence — Guidance on risk management,

    work page 2023

  12. [12]

    ISO/IEC 24029-2:2023 — Artificial intelligence (AI) — Assessment of the robustness of neural networks — Part 2: Methodology for the use of formal methods,

    ISO/IEC 24029-2. ISO/IEC 24029-2:2023 — Artificial intelligence (AI) — Assessment of the robustness of neural networks — Part 2: Methodology for the use of formal methods,

    work page 2023

  13. [13]

    ISO/IEC 42001:2023 - Information technology — Artificial intelligence — Management system,

    ISO/IEC 42001. ISO/IEC 42001:2023 - Information technology — Artificial intelligence — Management system,

    work page 2023

  14. [14]

    ISO/IEC TR 24029-1:2021 — Artificial Intelligence (AI) — Assessment of the robustness of neural networks — Part 1: Overview,

    ISO/IEC TR 24029-1. ISO/IEC TR 24029-1:2021 — Artificial Intelligence (AI) — Assessment of the robustness of neural networks — Part 1: Overview,

    work page 2021

  15. [15]

    doi: 10.1038/s41467-022-33128-9

    ISSN 2041-1723. doi: 10.1038/s41467-022-33128-9. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model Cards for Model Reporting. InProceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, pages 220–229, New York, NY , USA, January

    work page doi:10.1038/s41467-022-33128-9 2041

  16. [16]

    Model Cards for Model Reporting,

    Association for Computing Machinery. ISBN 978-1-4503-6125-5. doi: 10.1145/3287560.3287596. Oliver Niggemann, Bernd Zimmering, Henrik Steude, Jan Lukas Augustin, Alexander Windmann, and Samim Multaheb. Machine Learning for Cyber-Physical Systems. In Birgit V ogel-Heuser and Manuel Wimmer, editors,Digital Transformation: Core Technologies and Emerging Topic...

    work page doi:10.1145/3287560.3287596

  17. [17]

    doi: 10.1007/978-3-662-65004-2\_17

    ISBN 978-3-662-65004-2. doi: 10.1007/978-3-662-65004-2\_17. Jon Perez-Cerrolaza, Jaume Abella, Markus Borg, Carlo Donzella, Jesús Cerquides, Francisco J. Cazorla, Cristofer Englund, Markus Tauber, George Nikolakopoulos, and Jose Luis Flores. Artificial Intelligence for Safety-Critical Systems in Industrial and Transportation Domains: A Survey.ACM Comput. ...

    work page doi:10.1007/978-3-662-65004-2

  18. [18]

    doi: 10.1145/3626314

    ISSN 0360-0300. doi: 10.1145/3626314. Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying Some Distributional Robustness with Principled Adversarial Training. InInternational Conference on Learning Representations, February

    work page doi:10.1145/3626314

  19. [19]

    Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research

    Daniel Vranješ, Jonas Ehrhardt, René Heesch, Lukas Moddemann, Henrik Sebastian Steude, and Oliver Niggemann. Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research. In35th International Conference on Principles of Diagnosis and Resilient Systems (DX 2024),

    work page 2024

  20. [20]

    DX.2024.7

    doi: 10.4230/OASIcs. DX.2024.7. Geoffrey I. Webb, Roy Hyde, Hong Cao, Hai Long Nguyen, and Francois Petitjean. Characterizing concept drift.Data Mining and Knowledge Discovery, 30(4):964–994, April

    work page doi:10.4230/oasics 2024

  21. [21]

    doi: 10.1007/s10618-015-0448-4

    ISSN 1573-756X. doi: 10.1007/s10618-015-0448-4. Alexander Windmann, Philipp Wittenberg, Marvin Schieseck, and Oliver Niggemann. Artificial Intelligence in Industry 4.0: A Review of Integration Challenges for Industrial Systems. In2024 IEEE 22nd International Conference on Industrial Informatics (INDIN), pages 1–8, August

    work page doi:10.1007/s10618-015-0448-4

  22. [22]

    Grande, R., Walsh, T., and How, J

    doi: 10.1109/INDIN58382.2024.10774364. ISSN: 2378-363X. Alexander Windmann, Henrik Steude, Daniel Boschmann, and Oliver Niggemann. Quantifying Robustness: A Benchmarking Framework for Deep Learning Forecasting in Cyber-Physical Systems. In2025 IEEE 30th Interna- tional Conference on Emerging Technologies and Factory Automation (ETFA), pages 1–8, September

    work page doi:10.1109/indin58382.2024.10774364 2024

  23. [23]

    7 General Information Data-driven soft sensor estimating penicillin concentration on the IndPenSim benchmark

    doi: 10.1109/ETFA65518.2025.11205527. 7 General Information Data-driven soft sensor estimating penicillin concentration on the IndPenSim benchmark. MODEL LSTM DATASET IndPenSim TARGET Penicillin concentration (g/L) LOSS FUNCTION MSE WINDOW 90 → 30 DATE 2025-12-04 17:12 UTC MODEL ID 08432de6d0cf4eeeb0dbb0a3a1efdf3e PROVIDER Bio Data Science CONTACT penicil...

    work page doi:10.1109/etfa65518.2025.11205527 2025