pith. sign in

arxiv: 2604.14787 · v1 · submitted 2026-04-16 · 📡 eess.SY · cs.LG· cs.NI· cs.SY

Towards Trustworthy 6G Network Digital Twins: A Framework for Validating Counterfactual What-If Analysis in Edge Computing Resources

Julian Jimenez Agudelo , Paola Soto , Ayat Zaki-Hindi , Jean-S\'ebastien Sottet , S\'ebastien Faye , Nina Slamnik-Krije\v{s}torac , Johann Marquez-Barja , Miguel Camelo Botero This is my paper

Pith reviewed 2026-05-10 11:09 UTC · model grok-4.3

classification 📡 eess.SY cs.LGcs.NIcs.SY
keywords network digital twins6Gedge computingcounterfactual analysiswhat-if validationresource scalingkubernetes telemetrymachine learning regression
0
0 comments X

The pith

A validation framework makes network digital twins reliable for what-if analysis of 6G edge resource scaling.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a data-driven framework that collects and aligns cloud-edge telemetry into unified models for Network Digital Twins. It adds regime-aware feature engineering to capture scaling behavior and validates predictions with Sign Agreement and Directional Sensitivity metrics. Tested on a Kubernetes cluster, the models accurately regress performance and extrapolate to unseen high-load regimes. This approach would let operators run safe counterfactual tests to guide proactive resource decisions without risking live infrastructure.

Core claim

The authors claim that combining scalable telemetry aggregation, regime-aware feature engineering, and validation based on Sign Agreement and Directional Sensitivity turns Network Digital Twins into trustworthy tools. On a Kubernetes-managed cluster both DNN and XGBoost models reach R2 above 0.99 while XGBoost achieves Sa above 0.90, allowing reliable extrapolation of performance to out-of-distribution high-load conditions.

What carries the argument

Regime-aware feature engineering that captures network scaling behavior, paired with Sign Agreement and Directional Sensitivity metrics to validate counterfactual predictions.

Load-bearing premise

The chosen metrics and features derived from one Kubernetes cluster's telemetry are sufficient to confirm reliable what-if analysis and extrapolation to real 6G deployments.

What would settle it

Applying the trained models to a second cluster under actual high-load conditions and observing that predicted resource needs deviate in sign or sensitivity from measured scaling behavior.

Figures

Figures reproduced from arXiv: 2604.14787 by Ayat Zaki-Hindi, Jean-S\'ebastien Sottet, Johann Marquez-Barja, Julian Jimenez Agudelo, Miguel Camelo Botero, Nina Slamnik-Krije\v{s}torac, Paola Soto, S\'ebastien Faye.

Figure 1
Figure 1. Figure 1: ITU Reference architecture [1] A. ITU Reference Architecture As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: 6G-TWIN enhanced architecture including the Telemetry Data Layer (TDL) and Harmonization Data Layer (HDL) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example of metamodel schemas representing CEs in edge/cloud [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: NDT Framework for Validating Counterfactual What-If Analysis in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: True versus predicted latency for the unseen 600 user regime. The dashed diagonal indicates ideal prediction. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
read the original abstract

Network Digital Twins (NDTs) enable safe what-if analysis for 6G cloud-edge infrastructures, but adoption is often limited by fragmented workflows from telemetry to validation. We present a data-driven NDT framework that extends 6G-TWIN with a scalable pipeline for cloud-edge telemetry aggregation and semantic alignment into unified data models. Our contributions include: (i) scalable cloud-edge telemetry collection, (ii) regime-aware feature engineering capturing the network's scaling behavior, and (iii) a validation methodology based on Sign Agreement and Directional Sensitivity. Evaluated on a Kubernetes-managed cluster, the framework extrapolates performance to unseen high-load regimes. Results show both Deep Neural Network (DNN) and XGBoost achieve high regression accuracy (R2 > 0.99), while the XGBoost model delivers superior directional reliability (Sa > 0.90), making the NDT a trustworthy tool for proactive resource scaling in out-of-distribution scenarios.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes a data-driven framework extending 6G-TWIN for Network Digital Twins (NDTs) in cloud-edge infrastructures. It contributes a scalable telemetry aggregation and semantic alignment pipeline, regime-aware feature engineering to capture scaling behavior, and a validation methodology based on Sign Agreement (Sa) and Directional Sensitivity metrics. Evaluated on a single Kubernetes-managed cluster, DNN and XGBoost models are reported to achieve R² > 0.99 regression accuracy with XGBoost attaining Sa > 0.90, supporting claims of trustworthy extrapolation to unseen high-load regimes for counterfactual what-if analysis and proactive 6G resource scaling.

Significance. If the extrapolation and counterfactual claims hold under rigorous validation, the work could provide a practical pipeline for building trustworthy NDTs, addressing fragmentation in telemetry-to-validation workflows for 6G edge systems. The focus on directional metrics like Sa is a positive step beyond standard regression for scaling decisions. However, the single-cluster, in-distribution evaluation limits immediate significance for real 6G deployments involving wireless variability and multi-vendor orchestration.

major comments (3)
  1. [Evaluation] Evaluation section: The abstract and results claim R² > 0.99 and Sa > 0.90 for extrapolation to unseen high-load regimes, but provide no details on data splits, how the high-load test regimes were constructed from the telemetry, statistical tests for significance, or controls for overfitting. This directly undermines the central claim that the NDT enables robust out-of-distribution performance.
  2. [Feature Engineering and Validation] Feature engineering and validation methodology: Regime-aware features are derived from observed scaling behavior in the same Kubernetes telemetry dataset used for model training and testing. Combined with predictive metrics (R², Sa) computed on held-out points from this dataset, this creates circularity; high Sa may reflect captured in-distribution patterns rather than independent validation of counterfactual what-if behavior under interventions.
  3. [Validation Methodology] Validation methodology: The framework positions Sa and Directional Sensitivity as sufficient to validate counterfactual analysis and trustworthiness for 6G, yet no causal model, explicit intervention experiments (e.g., actual resource scaling actions), or out-of-cluster ground truth is presented. Predictive accuracy on single-cluster telemetry does not establish that directional agreement implies correct behavior under true 6G factors absent from the testbed.
minor comments (2)
  1. [Abstract] Abstract: Reports inequalities (R² > 0.99, Sa > 0.90) without exact values, confidence intervals, or sample sizes; include these for reproducibility.
  2. [Introduction and Conclusions] The manuscript would benefit from a clearer distinction between in-distribution prediction and true counterfactual validation in the introduction and conclusions.

Simulated Author's Rebuttal

3 responses · 1 unresolved

We thank the referee for their constructive and detailed feedback, which helps clarify the presentation of our evaluation and validation approach. We respond to each major comment below and indicate planned revisions.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: The abstract and results claim R² > 0.99 and Sa > 0.90 for extrapolation to unseen high-load regimes, but provide no details on data splits, how the high-load test regimes were constructed from the telemetry, statistical tests for significance, or controls for overfitting. This directly undermines the central claim that the NDT enables robust out-of-distribution performance.

    Authors: We agree that the Evaluation section requires additional detail to support the extrapolation claims. In the revised manuscript we will expand this section to specify: the train/test split (training on low-to-medium load telemetry and testing on held-out high-load points above an 80% utilization threshold); the exact construction of high-load regimes from the Kubernetes logs; results of statistical significance tests (e.g., paired t-tests on R² and Sa across regimes); and overfitting controls including L2 regularization, early stopping, and 5-fold cross-validation on the training set. These additions will make the out-of-distribution evaluation transparent. revision: yes

  2. Referee: [Feature Engineering and Validation] Feature engineering and validation methodology: Regime-aware features are derived from observed scaling behavior in the same Kubernetes telemetry dataset used for model training and testing. Combined with predictive metrics (R², Sa) computed on held-out points from this dataset, this creates circularity; high Sa may reflect captured in-distribution patterns rather than independent validation of counterfactual what-if behavior under interventions.

    Authors: The regime-aware features are computed from aggregate scaling patterns in the training data only and then applied to held-out high-load test points. This is intended to test generalization of observed scaling laws rather than memorization. We will add an ablation study in the revision showing performance with and without these features to quantify their contribution to extrapolation. We will also clarify the train/test separation in the text. While this mitigates circularity, we acknowledge the evaluation remains within a single-cluster distribution and will discuss this explicitly. revision: partial

  3. Referee: [Validation Methodology] Validation methodology: The framework positions Sa and Directional Sensitivity as sufficient to validate counterfactual analysis and trustworthiness for 6G, yet no causal model, explicit intervention experiments (e.g., actual resource scaling actions), or out-of-cluster ground truth is presented. Predictive accuracy on single-cluster telemetry does not establish that directional agreement implies correct behavior under true 6G factors absent from the testbed.

    Authors: We agree that predictive accuracy on held-out single-cluster telemetry does not constitute full causal validation or out-of-cluster ground truth. This is a limitation of the present study, which focuses on a practical data-driven pipeline. In the revision we will add a dedicated Limitations section that acknowledges the absence of explicit intervention experiments and causal models, and we will moderate claims from 'trustworthy counterfactual what-if analysis' to 'supporting reliable directional extrapolation for what-if scenarios based on observed patterns.' Future work on causal methods and multi-cluster interventions will be outlined. revision: yes

standing simulated objections not resolved
  • The absence of explicit causal intervention experiments and out-of-cluster ground truth, which would require new experimental infrastructure and data collection beyond the scope of the current manuscript.

Circularity Check

1 steps flagged

Validation of counterfactual extrapolation reduces to in-distribution fit on single-cluster telemetry via regime-aware features

specific steps
  1. fitted input called prediction [Abstract (contributions ii-iii and evaluation paragraph)]
    "Our contributions include: (i) scalable cloud-edge telemetry collection, (ii) regime-aware feature engineering capturing the network's scaling behavior, and (iii) a validation methodology based on Sign Agreement and Directional Sensitivity. Evaluated on a Kubernetes-managed cluster, the framework extrapolates performance to unseen high-load regimes. Results show both Deep Neural Network (DNN) and XGBoost achieve high regression accuracy (R2 > 0.99), while the XGBoost model delivers superior directional reliability (Sa > 0.90)"

    Regime-aware features are derived directly from observed scaling patterns in the telemetry; models are then trained on this data and evaluated on 'unseen' high-load subsets of the same data. The high R² and Sa metrics therefore quantify in-distribution predictive fidelity on the fitted inputs rather than providing independent validation of counterfactual what-if analysis or out-of-cluster extrapolation.

full rationale

The paper claims the NDT enables trustworthy what-if analysis and extrapolation to unseen high-load regimes for 6G scaling, supported by R² > 0.99 and Sa > 0.90 from DNN/XGBoost. However, the regime-aware features are explicitly engineered to capture scaling behavior from the same Kubernetes telemetry, and all validation occurs on held-out points from that single dataset. This makes the reported directional reliability and regression accuracy a direct measure of how well the models fit patterns already present in the training distribution, rather than independent evidence of causal counterfactual validity or generalization beyond the testbed. No external ground truth, causal model, or cross-environment checks are provided to break the dependence on the fitted inputs.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that collected telemetry represents general 6G edge scaling behavior and that the proposed validation metrics capture trustworthiness for out-of-distribution what-if scenarios; no explicit free parameters or invented entities are stated, but ML model training implicitly involves fitting choices.

free parameters (1)
  • ML model hyperparameters and feature selection thresholds
    Implicit in training DNN and XGBoost and defining regime-aware features; not specified but required for the reported accuracy.
axioms (1)
  • domain assumption Telemetry data from the Kubernetes-managed cluster accurately represents the scaling behavior of 6G cloud-edge infrastructures.
    Invoked to justify training and extrapolation to unseen high-load regimes.

pith-pipeline@v0.9.0 · 5524 in / 1542 out tokens · 69582 ms · 2026-05-10T11:09:13.201126+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    Digital Twin Network - Requirements and Architecture,

    ITU-T, “Digital Twin Network - Requirements and Architecture,” International Telecommunication Union, Recommendation Y .3090, Feb. 2022. [Online]. Available: https://www.itu.int/rec/T-REC-Y .3090- 202202-I/

    work page 2022

  2. [2]

    Network digital twin: Concepts and reference architecture,

    C. Zhouet al., “Network digital twin: Concepts and reference architecture,” Internet Engineering Task Force (IETF), Internet-Draft draft-irtf-nmrg-network-digital-twin-arch-07, Sep. 2024, informational draft, expires 30 March 2025. [Online]. Available: https://www.ietf.org/archive/id/draft-irtf-nmrg- network-digital-twin-arch-07.html

    work page 2024

  3. [3]

    Integrating network digital twinning into future ai-based 6g systems: The 6g-twin vision,

    S. Fayeet al., “Integrating network digital twinning into future ai-based 6g systems: The 6g-twin vision,” in2024 Joint European Conference on Networks and Communications ’&’ 6G Summit (EuCNC/6G Summit), 2024, pp. 883–888

    work page 2024

  4. [4]

    Architectural design for digital twin networks,

    J. Wiemeet al., “Architectural design for digital twin networks,”Network, vol. 5, no. 3, 2025. [Online]. Available: https://www.mdpi.com/2673-8732/5/3/24

    work page 2025

  5. [5]

    A comprehensive survey of net- work digital twin architecture, capabilities, challenges, and requirements for edge–cloud continuum,

    S. M. Razaet al., “A comprehensive survey of net- work digital twin architecture, capabilities, challenges, and requirements for edge–cloud continuum,”Computer Commu- nications, vol. 236, p. 108144, 2025. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S014036642500101X

    work page 2025

  6. [6]

    Network digital twins for 6g: Defining data taxonomy and data models,

    A. Zaki-Hindiet al., “Network digital twins for 6g: Defining data taxonomy and data models,” in2024 IEEE Conference on Standards for Communications and Networking (CSCN), 2024, pp. 124–128

    work page 2024

  7. [7]

    A functional framework for network digital twins,

    S. Fayeet al., “A functional framework for network digital twins,” in 2025 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2025, pp. 01–06

    work page 2025

  8. [8]

    An architectural framework for 6g network digital twins system,

    Z. Yanget al., “An architectural framework for 6g network digital twins system,” inProceedings of the 30th Annual International Conference on Mobile Computing and Networking, ser. ACM MobiCom ’24. New York, NY , USA: Association for Computing Machinery, 2024, p. 2437–2441. [Online]. Available: https://doi.org/10.1145/3636534.3696730

    work page doi:10.1145/3636534.3696730 2024

  9. [9]

    Digital twin enabled cellu- lar network management and prediction,

    N. U. Saqibet al., “Digital twin enabled cellu- lar network management and prediction,”ICT Express, vol. 10, no. 3, pp. 479–484, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405959524000249

    work page 2024

  10. [10]

    A modular network digital twin for radio coverage prediction: From theory to practice,

    A. Zaki-Hindiet al., “A modular network digital twin for radio coverage prediction: From theory to practice,” Sep. 2025. [Online]. Available: https://doi.org/10.5281/zenodo.17086138

    work page doi:10.5281/zenodo.17086138 2025

  11. [11]

    Avoiding sdn application conflicts with digital twins: Design, models and proof of concept,

    M. Polveriniet al., “Avoiding sdn application conflicts with digital twins: Design, models and proof of concept,”IEEE Transactions on Network and Service Management, vol. 23, pp. 2038–2050, 2026

    work page 2038

  12. [12]

    Building a digital twin for network optimization using graph neural networks,

    M. Ferriol-Galm ´eset al., “Building a digital twin for network optimization using graph neural networks,”Computer Networks, vol. 217, p. 109329, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1389128622003681

    work page 2022

  13. [13]

    Atari: A graph convolutional neural network approach for performance prediction in next-generation wlans,

    P. Sotoet al., “Atari: A graph convolutional neural network approach for performance prediction in next-generation wlans,”Sensors, vol. 21, no. 13, 2021. [Online]. Available: https://www.mdpi.com/1424- 8220/21/13/4321

    work page 2021

  14. [14]

    D1.3 – Frameworks for Zero-Touch Service and Network Management and the Orchestration of its AI-based NF and NS,

    6G-TWIN Consortium, “D1.3 – Frameworks for Zero-Touch Service and Network Management and the Orchestration of its AI-based NF and NS,” 6G-TWIN, Public Deliverable D1.3, Dec. 2025. [Online]. Available: https://6g-twin.eu/resources/#deliverables

    work page 2025

  15. [15]

    Network telemetry framework,

    H. Songet al., “Network telemetry framework,” Internet Engineering Task Force (IETF), RFC 9232, Informational 9232, May 2022. [Online]. Available: https://datatracker.ietf.org/doc/rfc9232/

    work page 2022

  16. [16]

    Apache kafka on big data event streaming for enhanced data flows,

    K. Padmanabanet al., “Apache kafka on big data event streaming for enhanced data flows,” in2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), 2024, pp. 977– 983

    work page 2024

  17. [17]

    Sonata: query-driven streaming network telemetry,

    A. Guptaet al., “Sonata: query-driven streaming network telemetry,” inProceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, ser. SIGCOMM ’18. New York, NY , USA: Association for Computing Machinery, 2018, p. 357–371. [Online]. Available: https://doi.org/10.1145/3230543.3230555

    work page doi:10.1145/3230543.3230555 2018

  18. [18]

    Ai-driven digital twin frame- work for security threat simulation and compliance optimization,

    V . Venkatesan and C. Arunachalam, “Ai-driven digital twin frame- work for security threat simulation and compliance optimization,” in 2025 IEEE International Carnahan Conference on Security Technology (ICCST), 2025, pp. 1–6

    work page 2025

  19. [19]

    Sdn-integrated cloud-edge digital twin framework for real-time monitoring in additive manufacturing,

    H. B. Tsegayeet al., “Sdn-integrated cloud-edge digital twin framework for real-time monitoring in additive manufacturing,” in2025 IEEE 30th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), 2025, pp. 1–6

    work page 2025

  20. [20]

    Smart data models initiative,

    “Smart data models initiative,” 2026, available at https://smartdatamodels.org/, accessed February 2026

    work page 2026

  21. [21]

    A data space for 6g network digital twins: Challenges and opportunities,

    J.-S. Sottetet al., “A data space for 6g network digital twins: Challenges and opportunities,” in2025 IEEE CSCN. IEEE, 2025, pp. 1–2

    work page 2025

  22. [22]

    ETSI TS 128 622, “Universal mobile telecommunications system (umts); lte; 5g; management and orchestration; generic network resource model (nrm) integration reference point (irp); information service (3gpp ts 28.622 release 18),” European Telecommunications Standards Institute (ETSI), Technical Specification (TS), 2024

    work page 2024

  23. [23]

    Network functions virtualisation (nfv) release 4; management and orchestration; vnf descriptor and packaging spec- ification,

    ETSI GS NFV-IFA 011, “Network functions virtualisation (nfv) release 4; management and orchestration; vnf descriptor and packaging spec- ification,” European Telecommunications Standards Institute (ETSI), Group Specification (GS), 2023

    work page 2023

  24. [24]

    Network functions virtualisation (nfv) release 4; management and orchestration; os-ma-nfvo reference point - interface and information model specification,

    ETSI GS NFV-IFA 013, “Network functions virtualisation (nfv) release 4; management and orchestration; os-ma-nfvo reference point - interface and information model specification,” European Telecommunications Standards Institute (ETSI), Group Specification (GS), 2023

    work page 2023

  25. [25]

    Chorus: Harmonizing context and sensing signals for data-free model customization in iot,

    L. Zhanget al., “Chorus: Harmonizing context and sensing signals for data-free model customization in iot,”arXiv e-prints, pp. arXiv–2512, 2025

    work page 2025

  26. [26]

    Zero-touch network and Service Management (ZSM); Net- work Digital Twin,

    ETSI, “Zero-touch network and Service Management (ZSM); Net- work Digital Twin,” European Telecommunications Standards Institute (ETSI), ETSI Group Report ETSI GR ZSM 015, Feb 2024

    work page 2024