pith. sign in

arxiv: 2503.12946 · v2 · submitted 2025-03-17 · 💻 cs.AR · cs.AI

Open3DBench: Open-Source Benchmark for 3D-IC Backend Implementation and PPA Evaluation

Pith reviewed 2026-05-22 23:55 UTC · model grok-4.3

classification 💻 cs.AR cs.AI
keywords 3D-ICbenchmarkbackend implementationPPA evaluationplacement algorithmsOpenROADwirelengththermal simulation
0
0 comments X

The pith

Open3DBench supplies an open-source flow for 3D-IC backend design that reports over 50 percent area reduction versus 2D methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Open3DBench, a benchmark platform built on open flow scripts to evaluate 3D integrated circuit backend steps including partitioning, placement, routing, extraction, and thermal analysis. It supplies two placement approaches that together produce measured gains in area, wirelength, timing, and power over standard 2D designs. Experiments on the platform also show that shorter wirelength does not reliably deliver better overall power-performance-area results. The work positions the benchmark as a reproducible way to test 3D methods without relying solely on closed commercial tools.

Core claim

Open3DBench enables modular 3D-IC backend implementation and PPA evaluation on an open platform. Two placement algorithms are presented: Open3D-Tiling for regular macro placement and Open3D-DMP for cross-die co-placement. On test cases the flow records 51.19 percent area reduction, 24.06 percent wirelength reduction, 30.84 percent timing improvement, and 5.72 percent power reduction relative to 2D flows, while demonstrating that wirelength gains alone do not ensure PPA gains and therefore PPA-driven methods are required.

What carries the argument

Modular 3D-IC flow supporting partitioning, placement, 3D routing, RC extraction, and thermal simulation together with the Open3D-Tiling and Open3D-DMP placement algorithms.

If this is right

  • 3D-IC backend flows can deliver simultaneous reductions in area, wirelength, timing, and power.
  • Placement methods must optimize directly for PPA metrics rather than wirelength alone.
  • Standardized open benchmarks allow consistent comparison of 3D EDA techniques.
  • Modular integration of thermal simulation is feasible within open backend flows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The benchmark could serve as a starting point for teams developing new 3D partitioning or routing heuristics.
  • Future extensions might add support for multi-tier stacking beyond two dies.
  • Widespread use could reduce reliance on proprietary 3D-IC tool suites for academic and small-team research.
  • The observation that wirelength and PPA are imperfectly correlated may guide placement research toward multi-objective optimizers.

Load-bearing premise

The open modular flow produces results that align with those from advanced 3D-IC design practices that combine commercial tools and custom scripts.

What would settle it

Running the same benchmark designs through a commercial 3D-IC tool chain and finding no area, timing, or power gains would falsify the reported improvements.

Figures

Figures reproduced from arXiv: 2503.12946 by Chao Qian, Chengrui Gao, Ke Xue, Mingxuan Yuan, Peng Xie, Siyuan Xu, Wanqi Ren, Yunqi Shi, Zhi-Hua Zhou.

Figure 1
Figure 1. Figure 1: The illustrations of micro-bumping, hybrid bonding, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of routing wirelength (rWL), WNS, and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed 3D backend implementation [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of our analytical pseudo-3D placement [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Correlation analysis between rWL and key performance metrics comparing 2D and 3D methods. We plot the improvement [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Correlation analysis between rWL and key performance metrics comparing two 3D methods. We plot the improvement [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Heat map visualization of Hier-RTLMP-2D and [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
read the original abstract

This work introduces Open3DBench, an open-source 3D-IC backend implementation benchmark built upon the OpenROAD-flow-scripts framework, enabling comprehensive evaluation of power, performance, area, and thermal metrics. Our proposed flow supports modular integration of 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation, aligning with advanced 3D flows that rely on commercial tools and in-house scripts. We present two foundational 3D placement algorithms: Open3D-Tiling, which emphasizes regular macro placement, and Open3D-DMP, which enhances wirelength optimization through cross-die co-placement with analytical placer DREAMPlace. Experimental results show significant improvements in area (51.19%), wirelength (24.06%), timing (30.84%), and power (5.72%) compared to 2D flows. The results also highlight that better wirelength does not necessarily lead to PPA gain, emphasizing the need of developing PPA-driven methods. Open3DBench offers a standardized, reproducible platform for evaluating 3D EDA methods, effectively bridging the gap between open-source tools and commercial solutions in 3D-IC design.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces Open3DBench, an open-source 3D-IC backend benchmark built on OpenROAD-flow-scripts. It proposes two placement algorithms (Open3D-Tiling for regular macro placement and Open3D-DMP for cross-die wirelength optimization via DREAMPlace) and a modular flow supporting 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation. Experimental results claim PPA gains versus 2D flows (area 51.19%, wirelength 24.06%, timing 30.84%, power 5.72%) and note that wirelength improvement does not guarantee PPA gains, positioning the benchmark as a reproducible platform bridging open-source and commercial 3D-IC flows.

Significance. If the open-source components fully implement the claimed modules without undisclosed commercial dependencies and the results prove reproducible, the work would deliver a valuable standardized benchmark and two baseline algorithms for 3D-IC research. The explicit release of the benchmark and algorithms constitutes a concrete strength that could enable community-driven PPA-driven method development.

major comments (2)
  1. [§3] §3 (Proposed Flow description): the assertion that the flow 'supports modular integration of 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation' while aligning with commercial-tool flows does not map each module to its open-source implementation or identify any remaining commercial-tool fallbacks; this directly affects whether the reported PPA deltas can be attributed to the open-source methods alone.
  2. [§5] §5 (Experimental Results): the headline improvements (area 51.19%, wirelength 24.06%, timing 30.84%, power 5.72%) are reported without stating the number of test cases, statistical significance tests, run-to-run variance, or data-selection criteria, leaving the robustness of the cross-flow comparison unclear.
minor comments (2)
  1. Figure captions and axis labels should explicitly distinguish 2D baseline from the two 3D variants (Open3D-Tiling, Open3D-DMP) for immediate readability.
  2. Ensure first-use definitions for all acronyms (PPA, DMP, RC) and consistent notation for the two proposed algorithms across text and tables.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight opportunities to improve transparency and robustness in the manuscript. We address each major comment below and will incorporate revisions to strengthen the paper.

read point-by-point responses
  1. Referee: [§3] §3 (Proposed Flow description): the assertion that the flow 'supports modular integration of 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation' while aligning with commercial-tool flows does not map each module to its open-source implementation or identify any remaining commercial-tool fallbacks; this directly affects whether the reported PPA deltas can be attributed to the open-source methods alone.

    Authors: We agree that the current §3 description lacks an explicit mapping. The revised manuscript will include a new table (or expanded subsection) that maps each module (3D partitioning, placement, 3D routing, RC extraction, thermal simulation) to its concrete open-source implementation within OpenROAD-flow-scripts, along with any identified commercial-tool fallbacks or dependencies. This will allow readers to directly assess attribution of the reported PPA deltas. revision: yes

  2. Referee: [§5] §5 (Experimental Results): the headline improvements (area 51.19%, wirelength 24.06%, timing 30.84%, power 5.72%) are reported without stating the number of test cases, statistical significance tests, run-to-run variance, or data-selection criteria, leaving the robustness of the cross-flow comparison unclear.

    Authors: We acknowledge the need for greater experimental detail. The revised §5 will explicitly state the number of test cases, the data-selection criteria used, any statistical significance tests applied, and run-to-run variance (where relevant) to substantiate the cross-flow PPA comparisons and improve reproducibility. revision: yes

Circularity Check

0 steps flagged

Empirical benchmark with no self-referential derivations or fitted predictions

full rationale

The paper introduces Open3DBench as an open-source framework extending OpenROAD-flow-scripts, presents two placement algorithms (Open3D-Tiling and Open3D-DMP), and reports direct empirical PPA deltas versus 2D baselines. No equations, uniqueness theorems, ansatzes, or predictions appear that reduce by construction to fitted inputs or prior self-citations. The central claims rest on experimental measurements of area, wirelength, timing, and power, which are externally falsifiable and do not rely on any load-bearing self-referential step.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms beyond standard EDA tool assumptions, or invented entities are introduced; the work rests on existing OpenROAD infrastructure.

pith-pipeline@v0.9.0 · 5765 in / 1116 out tokens · 27297 ms · 2026-05-22T23:55:00.304892+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A PPA-Driven 3D-IC Partitioning Selection Framework with Surrogate Models

    cs.LG 2026-04 unverdicted novelty 5.0

    DOPP applies surrogate models to select 3D-IC partitioning candidates that optimize true PPA, matching exhaustive-search quality with substantially lower evaluation cost across eight designs.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    HotSpot,

    “HotSpot,” https://github.com/uvahotspot/HotSpot

  2. [2]

    OpenRCX,

    “OpenRCX,” https://github.com/The-OpenROAD-Project/OpenRCX

  3. [3]

    OpenSTA,

    “OpenSTA,” https://github.com/parallaxsw/OpenSTA

  4. [4]

    OpenROAD: Toward a self-driving, open-source digital layout implementation tool chain,

    T. Ajayi, D. Blaauw, T. Chan, C. Cheng, V . Chhabria, D. Choo, M. Coltella, S. Dobre, R. Dreslinski, M. Fogac ¸a et al. , “OpenROAD: Toward a self-driving, open-source digital layout implementation tool chain,” in Proceedings of Government Microcircuit Applications and Critical Technology Conference , Albuquerque, NM, 2019, pp. 1105– 1110

  5. [5]

    Macro-3D: A physical design methodology for face-to-face-stacked heterogeneous 3D ICs,

    L. Bamberg, A. Garc ´ıa-Ortiz, L. Zhu, S. Pentapati, S. K. Lim et al. , “Macro-3D: A physical design methodology for face-to-face-stacked heterogeneous 3D ICs,” in Proceedings of Design, Automation & Test in Europe Conference , Grenoble, France, 2020, pp. 37–42

  6. [6]

    Cascade2D: A design-aware partitioning approach to mono- lithic 3D IC with 2D commercial tools,

    K. Chang, S. Sinha, B. Cline, R. Southerland, M. Doherty, G. Yeric, and S. K. Lim, “Cascade2D: A design-aware partitioning approach to mono- lithic 3D IC with 2D commercial tools,” in Proceedings of IEEE/ACM International Conference on Computer-Aided Design , Austin, TX, 2016, pp. 1–8

  7. [7]

    Stronger mixed-size placement backbone considering second-order information,

    Y . Chen, Z. Wen, Y . Liang, and Y . Lin, “Stronger mixed-size placement backbone considering second-order information,” in Proceedings of IEEE/ACM International Conference on Computer-Aided Design , San Francisco, CA, 2023, pp. 1–9

  8. [8]

    Die to wafer 3D stacking for below 10um pitch microbumps,

    J. Derakhshandeh, L. Hou, I. De Preter, C. Gerets, S. Suhard, V . Dubey, G. Jamieson, F. Inoue, T. Webers, P. Bex, G. Capuz, E. Beyne, J. Slabbekoorn, T. Wang, A. Jourdain, G. Beyer, K. June Rebibis, and A. Miller, “Die to wafer 3D stacking for below 10um pitch microbumps,” in Proceedings of IEEE International 3D Systems Integration Confer- ence, San Fran...

  9. [9]

    From 2.5D to 3D chiplet systems: Investigation of thermal implications with HotSpot 7.0,

    J.-H. Han, X. Guo, K. Skadron, and M. R. Stan, “From 2.5D to 3D chiplet systems: Investigation of thermal implications with HotSpot 7.0,” in Proceedings of IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems , San Diego, CA, 2022, pp. 1–6

  10. [10]

    TSV-aware analytical placement for 3-D IC designs based on a novel weighted-average wirelength model,

    M.-K. Hsu, V . Balabanov, and Y .-W. Chang, “TSV-aware analytical placement for 3-D IC designs based on a novel weighted-average wirelength model,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 32, no. 4, pp. 497–509, 2013

  11. [11]

    2023 ICCAD CAD contest problem B: 3D placement with macros,

    K.-S. Hu, H.-Y . Chi, I.-J. Lin, Y .-H. Wu, W.-H. Chen, and Y .-T. Hsieh, “2023 ICCAD CAD contest problem B: 3D placement with macros,” in Proceedings of IEEE/ACM International Conference on Computer-Aided Design, San Francisco, CA, 2023, pp. 1–6

  12. [12]

    2022 ICCAD CAD contest problem B: 3D placement with D2D vertical connections,

    K.-S. Hu, I.-J. Lin, Y .-H. Huang, H.-Y . Chi, Y .-H. Wu, and C.-F. C. Shen, “2022 ICCAD CAD contest problem B: 3D placement with D2D vertical connections,” in Proceedings of IEEE/ACM International Conference on Computer-Aided Design , New York, NY , 2022, pp. 1–5

  13. [13]

    Hier-RTLMP: A hier- archical automatic macro placer for large-scale complex IP blocks,

    A. B. Kahng, R. Varadarajan, and Z. Wang, “Hier-RTLMP: A hier- archical automatic macro placer for large-scale complex IP blocks,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 43, no. 5, pp. 1552–1565, 2024

  14. [14]

    TritonRoute: The open-source detailed router,

    A. B. Kahng, L. Wang, and B. Xu, “TritonRoute: The open-source detailed router,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 40, no. 3, pp. 547–559, 2021

  15. [15]

    TA3D: Timing- aware 3D IC partitioning and placement by optimizing the critical path,

    D. Kim, M. Kim, J. Hur, J. Lee, J. Cho, and S. Kang, “TA3D: Timing- aware 3D IC partitioning and placement by optimizing the critical path,” in Proceedings of the ACM/IEEE International Symposium on Machine Learning for CAD , New York, NY , 2024, pp. 1–7

  16. [16]

    Micro- bumping, hybrid bonding, or monolithic? A PPA study for heterogeneous 3D IC options,

    J. Kim, L. Zhu, H. M. Torun, M. Swaminathan, and S. K. Lim, “Micro- bumping, hybrid bonding, or monolithic? A PPA study for heterogeneous 3D IC options,” in Proceedings of ACM/IEEE Design Automation Conference, San Francisco, CA, 2021, pp. 1189–1194

  17. [17]

    Compact-2D: A physical design methodology to build commercial-quality face-to-face-bonded 3D ICs,

    B. W. Ku, K. Chang, and S. K. Lim, “Compact-2D: A physical design methodology to build commercial-quality face-to-face-bonded 3D ICs,” in Proceedings of the International Symposium on Physical Design , Monterey, CA, 2018, pp. 90–97

  18. [18]

    Analytical die-to-die 3-D placement with bistratal wirelength model and GPU acceleration,

    P. Liao, Y . Zhao, D. Guo, Y . Lin, and B. Yu, “Analytical die-to-die 3-D placement with bistratal wirelength model and GPU acceleration,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 43, no. 6, pp. 1624–1637, 2024

  19. [19]

    DREAM- Place: Deep learning toolkit-enabled GPU acceleration for modern VLSI placement,

    Y . Lin, S. Dhar, W. Li, H. Ren, B. Khailany, and D. Z. Pan, “DREAM- Place: Deep learning toolkit-enabled GPU acceleration for modern VLSI placement,” in Proceedings of ACM/IEEE Design Automation Conference, Las Vegas, NV , 2019, pp. 1–6

  20. [20]

    Routing- aware legal hybrid bonding terminal assignment for 3D face-to-face stacked ICs,

    S. Liu, J. Jiang, Z. He, Z. Wang, Y . Lin, B. Yu, and M. Wong, “Routing- aware legal hybrid bonding terminal assignment for 3D face-to-face stacked ICs,” inProceedings of the International Symposium on Physical Design, Taipei, Taiwan, 2024, pp. 75–82

  21. [21]

    ePlace-3D: Electrostatics based placement for 3D-ICs,

    J. Lu, H. Zhuang, I. Kang, P. Chen, and C.-K. Cheng, “ePlace-3D: Electrostatics based placement for 3D-ICs,” in Proceedings of the International Symposium on Physical Design , Santa Rosa, CA, 2016, pp. 11–18

  22. [22]

    Cramming more components onto integrated circuits,

    G. Moore, “Cramming more components onto integrated circuits,” Proceedings of the IEEE , vol. 86, no. 1, pp. 82–85, 1998

  23. [23]

    Art- 3D: Analytical 3D placement with reinforced parameter tuning for monolithic 3D ICs,

    G. Murali, S. M. Shaji, A. Agnesina, G. Luo, and S. K. Lim, “Art- 3D: Analytical 3D placement with reinforced parameter tuning for monolithic 3D ICs,” in Proceedings of the International Symposium on Physical Design , Virtual, Canada, 2022, pp. 97–104

  24. [24]

    FastRoute: A step to integrate global routing into placement,

    M. Pan and C. Chu, “FastRoute: A step to integrate global routing into placement,” in Proceedings of IEEE/ACM International Conference on Computer Aided Design , San Jose, CA, 2006, pp. 464–471

  25. [25]

    Shrunk-2-D: A physical design methodology to build commercial-quality monolithic 3-D ICs,

    S. Panth, K. Samadi, Y . Du, and S. K. Lim, “Shrunk-2-D: A physical design methodology to build commercial-quality monolithic 3-D ICs,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 36, no. 10, pp. 1716–1724, 2017

  26. [26]

    Pin-3D: A physical synthesis and post-layout optimization flow for heterogeneous monolithic 3D ICs,

    S. S. K. Pentapati, K. Chang, V . Gerousis, R. Sengupta, and S. K. Lim, “Pin-3D: A physical synthesis and post-layout optimization flow for heterogeneous monolithic 3D ICs,” in Proceedings of IEEE/ACM International Conference on Computer-Aided Design , San Diego, CA, 2020, pp. 1–9

  27. [27]

    In- creMacro: Incremental macro placement refinement,

    Y . Pu, T. Chen, Z. He, C. Bai, H. Zheng, Y . Lin, and B. Yu, “In- creMacro: Incremental macro placement refinement,” in Proceedings of the International Symposium on Physical Design , Taipei, Taiwan, 2024, pp. 169–176

  28. [28]

    Nangate technology node,

    A. Rob, “Nangate technology node,” https://si2.org/open-cell-library/

  29. [29]

    Timing-driven global placement by efficient critical path extraction,

    Y . Shi, S. Xu, S. Kai, X. Lin, K. Xue, M. Yuan, and C. Qian, “Timing-driven global placement by efficient critical path extraction,” in Proceedings of Design, Automation & Test in Europe Conference , Lyon, France, 2025, in press

  30. [30]

    Snap-3D: A constrained placement-driven physical design methodology for face-to-face-bonded 3D ICs,

    P. Vanna-Iampikul, C. Shao, Y .-C. Lu, S. Pentapati, and S. K. Lim, “Snap-3D: A constrained placement-driven physical design methodology for face-to-face-bonded 3D ICs,” in Proceedings of the International Symposium on Physical Design , Virtual, USA, 2021, pp. 39–46

  31. [31]

    ATPlace2.5D: Analytical thermal-aware chiplet placement framework for large-scale 2.5D-IC,

    Q. Wang, X. Li, T. Jia, Y . Lin, R. Wang, and R. Huang, “ATPlace2.5D: Analytical thermal-aware chiplet placement framework for large-scale 2.5D-IC,” in Proceedings of IEEE/ACM International Conference on Computer-Aided Design , New York, NY , 2024, pp. 1–9

  32. [32]

    Benchmarking end-to-end performance of AI- based chip placement algorithms,

    Z. Wang, Z. Geng, Z. Tu, J. Wang, Y . Qian, Z. Xu, Z. Liu, S. Xu, Z. Tang, S. Kai et al. , “Benchmarking end-to-end performance of AI- based chip placement algorithms,” arXiv preprint, 2024

  33. [33]

    A skyline heuristic for the 2D rectangular packing and strip packing problems,

    L. Wei, W.-C. Oon, W. Zhu, and A. Lim, “A skyline heuristic for the 2D rectangular packing and strip packing problems,” European Journal of Operational Research , vol. 215, no. 2, pp. 337–346, 2011

  34. [34]

    C. Wolf, J. Glaser, and J. Kepler, “Yosys,” https://github.com/YosysHQ/ yosys

  35. [35]

    Reinforcement learning policy as macro regulator rather than macro placer,

    K. Xue, R.-T. Chen, X. Lin, Y . Shi, S. Kai, S. Xu, and C. Qian, “Reinforcement learning policy as macro regulator rather than macro placer,” in Advances in Neural Information Processing Systems , Van- couver, Canada, 2024, pp. 140 565–140 588

  36. [36]

    Standard cells do matter: Uncovering hidden connections for high-quality macro placement,

    X. Zhao, T. Wang, R. Jiao, and X. Guo, “Standard cells do matter: Uncovering hidden connections for high-quality macro placement,” in Proceedings of Design, Automation & Test in Europe Conference , Valencia, Spain, 2024, pp. 1–6

  37. [37]

    Physical design for advanced 3D ICs: Challenges and solutions,

    Y . Zhao, L. Zou, and B. Yu, “Physical design for advanced 3D ICs: Challenges and solutions,” in Proceedings of the International Sympo- sium on Physical Design , Austin, TX, 2025, in press

  38. [38]

    Analytical het- erogeneous die-to-die 3-D placement with macros,

    Y . Zhao, P. Liao, S. Liu, J. Jiang, Y . Lin, and B. Yu, “Analytical het- erogeneous die-to-die 3-D placement with macros,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 44, no. 2, pp. 402–415, 2025

  39. [39]

    Z.-H. Zhou, Y . Yu, and C. Qian, Evolutionary Learning: Advances in Theories and Algorithms . Springer, 2019

  40. [40]

    High-performance logic-on-memory monolithic 3-D IC designs for Arm Cortex-A processors,

    L. Zhu, L. Bamberg, S. S. K. Pentapati, K. Chang, F. Catthoor, D. Milojevic, M. Komalan, B. Cline, S. Sinha, X. Xu, A. Garcia-Ortiz, and S. K. Lim, “High-performance logic-on-memory monolithic 3-D IC designs for Arm Cortex-A processors,” IEEE Transactions on V ery Large Scale Integration (VLSI) Systems , vol. 29, no. 6, pp. 1152–1163, 2021