Pith. sign in

REVIEW 3 major objections 6 minor 39 references

ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Anira decouples neural-network inference from the audio callback via a static thread pool, achieving real-time safety while showing ONNX Runtime is fastest for stateless models and LibTorch for stateful ones.

desk verdict Solid, useful engineering benchmark with rigorous stats, but the engine rankings rest on mean runtimes while the paper's own real-time criterion is worst-case time, so the practical guidance needs tail-latency analysis. read the letter →

arxiv 2506.12665 v1 pith:JCBIUTMP submitted 2025-06-14 cs.SD cs.AIcs.LGeess.ASeess.SP

Valentin Ackva , Fares Schulz This is my paper · ORCID
classification cs.SDcs.AIcs.LGeess.ASeess.SP
keywords real-timeaudioneuralnetworkinferenceONNXRuntimeLibTorchTensorFlowLitethreadpooleffectemulationlinearmixed-effectsmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces anira, a C++ library that lets audio plugins run neural-network inference without breaking real-time guarantees. The core design is to move inference off the audio callback onto a static thread pool, so blocking operations and memory allocation never happen on the time-critical audio thread. Using three audio-effect models (a temporal convolutional network, a recurrent long short-term memory network, and a hybrid network), the authors benchmark ONNX Runtime, LibTorch, and TensorFlow Lite across buffer sizes and operating systems, and fit linear mixed-effects models to the runtime data. They find that ONNX Runtime is fastest for stateless models, LibTorch is fastest for the stateful RNN, and that early inferences are slower, recommending a warm-up phase. If the architecture works as claimed, it gives plugin developers a cross-platform way to integrate neural nets while preserving real-time safety.

What carries the argument

The load-bearing mechanism is the static thread pool that decouples inference from the audio callback: the InferenceHandler submits inference tasks to a pool of pre-created threads, with atomic or semaphore-based structures synchronizing the audio thread with the inference threads. Latency is managed by a formula that adds the host-buffer adaptation delay, the worst-case inference time rounded up to a multiple of the host buffer size, and the model's internal latency: $$L_{total} = H_{adapt} + \left\lceil \frac{I_{max}}{H_{host}} \right\rceil \cdot H_{host} + M_{int}.$$ An optional proportional wait time can trade a controlled blocking operation for reduced latency. The built-in benchmarking module generates the per-sample runtime datasets that feed the statistical models.

What would settle it

Run the same benchmark suite but fit the statistical models to the maximum (or 99th percentile) runtime per buffer instead of the mean; if ONNX Runtime no longer leads for stateless models or LibTorch for the RNN under tail metrics, the paper's engine ranking does not support its real-time deployment guidance.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that anira's thread-pool decoupling eliminates real-time violations in the audio callback: a code sanitizer that intercepts system calls such as malloc, free, and pthread_mutex_lock reports no violations in the library's audio-path processing, even though all three underlying inference engines show such violations on every inference. The benchmark study then establishes an engine ranking that depends on model type: ONNX Runtime has the lowest runtimes per sample for the stateless CNN and hybrid models, while LibTorch is fastest for the stateful RNN (ONNX Runtime is excluded there because it only supports stateless operations). The paper also shows that the first inferences are significantly slower for certain engine–model pairs, and that per-sample runtime decreases as buffer size increases.

Load-bearing premise

The engine rankings and real-time conclusions are inferred from mean runtime per sample, while the paper itself defines real-time suitability by worst-case execution time; if the worst-case ordering differs from the mean ordering, the practical recommendations may not hold.

Editorial extensions

If this is right

  • Plugin developers can integrate neural networks into real-time audio without priority inversions or missed deadlines, provided they follow anira's decoupling pattern and account for its latency formula.
  • For stateless models, ONNX Runtime is the recommended backend; for stateful models such as the LSTM RNN, LibTorch is recommended over TensorFlow Lite.
  • Warm-up inferences before the audio callback can avoid the significantly slower early runtimes seen for some engine–model combinations.
  • Because per-sample runtime drops as buffer size grows, applications with looser latency constraints should prefer larger model input sizes.
  • Anira's built-in benchmarking lets developers estimate worst-case inference time, which is a prerequisite for the library's own latency calculation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the statistical rankings are based on mean runtime per sample rather than worst-case or tail latencies, a deployment that actually misses audio deadlines could find a different engine order; the paper's own criterion is maximum inference time.
  • The thread pool's parallel-inference capability is not benchmarked; on multicore systems, throughput-oriented configurations could change the relative standing of the engines.
  • The real-time safety test used one Linux x64 system and 50 inferences per model; portability of the safety claim to other operating systems or sustained workloads is not established by the paper.
  • Since ONNX Runtime cannot run stateful models directly, users with LSTM-based effects must either convert to stateless graphs or manage state externally, a workflow the paper does not demonstrate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents anira, a C++ library for neural network inference in real-time audio applications. The library decouples inference from the audio callback via a static thread pool, supports ONNX Runtime, LibTorch, and TensorFlow Lite, and provides latency management based on worst-case inference time estimates. The authors benchmark three audio effect models (a TCN, a stateful LSTM RNN, and a hybrid stateless-LSTM CNN) across three operating systems, buffer sizes from 64 to 8192 samples, and the three engines plus a bypass condition. Real-time safety is assessed with the RadSan sanitizer, and runtime performance is analyzed with linear mixed-effects models followed by corrected post-hoc tests. The main reported findings are that ONNX Runtime has the lowest average runtimes for stateless models, LibTorch is fastest for the stateful RNN, initial inferences are slower for certain engine-model pairs, and anira does not trigger RadSan violations in the audio callback.

Significance. The work is a solid, well-documented contribution to the practical deployment of neural audio effects. Its strengths include the public release of code and benchmark datasets, the use of a sanitizer to quantify real-time violations, a rigorous statistical methodology (LMMs, estimated marginal means, Bonferroni-Holm correction), and explicit discussion of limitations. If the worst-case timing analysis is added, the engine rankings would directly support deployment decisions. As it stands, the significance is reduced because the central ranking is based on means, not on the worst-case behavior that the paper itself identifies as the real-time criterion.

major comments (3)
  1. [Section III-C and IV-C; Eq. (1)] The benchmarks operationalize performance as mean runtime per sample (RpS) and the LMMs compare engines on that quantity, but the paper's own real-time criterion, stated in Sections I and II-D, is minimizing maximum or worst-case inference time, and Eq. (1) depends on Imax. No maximum, quantile, or distributional statistics are reported per engine and model. Since the RadSan results (Section IV-A) show real-time-unsafe operations (malloc, free, pthread locks) that are expected to produce tail spikes, an engine with a lower mean RpS could still miss audio deadlines more often. The conclusion that 'ONNX Runtime exhibits the lowest runtimes' is therefore a claim about average throughput, not about real-time suitability. I recommend re-analyzing the datasets with max/quantile statistics and ranking engines by those tail measures.
  2. [Section IV-C1 and Section V] The statement 'ONNX Runtime consistently outperforms the other two inference engines across all model architectures' (Section V) is not supported by the reported statistics. For the HNN-11k model, the ONNX Runtime vs. TensorFlow Lite difference has p = 0.045, which is not significant at the paper's stated threshold p < 0.0001 (Section III-E). Also, ONNX Runtime is excluded from all RNN comparisons, so 'all model architectures' is an overgeneralization. The abstract's 'for stateless models, ONNX Runtime exhibits the lowest runtimes' should be qualified to the specific model classes and to the significance levels actually obtained.
  3. [Section V (limitations) and Sections II-C, II-D] The authors acknowledge that buffer-size mismatch and parallel inference were not benchmarked, but these are central features of the architecture. The latency formula (1) includes the Hadapt term for buffer adaptation, and the thread pool's parallel inference is a key design choice for performance (Section II-C). Without benchmarks of mismatched host/model buffer sizes or a single parallel-inference scenario, the library's core mechanisms for real-time operation remain unvalidated, and the practical suggestion that anira is suitable for typical audio applications (where host and model buffer sizes often differ) is not supported. I ask the authors to add at least one mismatched-buffer configuration and a simple parallel-inference benchmark, or to frame the paper's contribution more narrowly.
minor comments (6)
  1. [Section III-A] The dilation recurrence is written as di = di−1, which is self-referential; it should presumably be d_i = d \cdot d_{i-1} or d^i. Please correct this typographical/notational error.
  2. [Section IV-C1] The phrase 'with TensorFlow Lite, performing worse than LibTorch' has a comma error; it should read 'with TensorFlow Lite performing worse than LibTorch'.
  3. [Table I and Eq. (1)] The max_inference_time parameter is given in milliseconds, while Imax in Eq. (1) is in samples. Please clarify the conversion and state the sample rate used in that conversion.
  4. [Section II-C] The term 'ThreadSafeStucts' appears to be a typo for 'ThreadSafeStructs'.
  5. [Section II-E] 'Optinally' should be spelled 'Optionally'.
  6. [Section V] The sentence 'While ONNX Runtime consistently outperforms the other two inference engines across all model architectures' conflicts with the RNN exclusion and with the non-significant HNN-11k contrast; see the second major comment.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the engine rankings and latency formulas are based on direct measurements and openly stated inputs, not on assumptions equivalent to the conclusions.

full rationale

The paper's central claims are empirical and self-contained. The engine rankings in Section IV-C are derived from measured runtimes per sample (RpS) in the three public benchmark datasets, analyzed with linear mixed-effects models whose dependent variable is the measured RpS and whose fixed effects are the experimental factors (system, engine, model, buffer size, iteration). No fitted parameter is renamed as a prediction: the latency formula in Eq. (1) uses Imax as a measured worst-case inference time supplied by the user or benchmark, not as a quantity derived from the same data that is then claimed as a prediction. The real-time safety conclusion for anira is supported by an independent RadSan instrumentation test over the library's process method, which is a direct measurement rather than an assumption. Self-citations are limited to the authors' own code and data repositories (refs. [13] and [32]) and to a forward reference to their own follow-up study; these are not load-bearing in the derivation of the paper's conclusions, and the cited prior work on inference-engine safety ([11], [12]) is external. The skeptical concern that rankings rely on mean RpS rather than worst-case tail latencies is a substantive validity or correctness risk, but it is not circularity: the paper does not define its conclusion into its input, nor does it fit a parameter and then report that parameter as an independent result. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no fitted physical constants or invented entities. Its central claims rely on three modeling assumptions: cross-engine model equivalence, RadSan as a complete safety oracle, and mean RpS as a real-time proxy. These are reasonable engineering assumptions but are load-bearing for the conclusions and are not all fully validated.

assumptions (3)
  • domain assumption The exported models are functionally equivalent across engines despite differing parameter counts and framework-specific implementations.
    Engine comparisons in Section IV-C assume runtime differences reflect engine efficiency rather than model implementation differences. Section III-A notes the RNN-2k has 1781 vs 1861 parameters depending on framework, and TensorFlow models were not exported to ONNX, so graph implementations differ.
  • domain assumption RadSan-detected real-time violations are a complete proxy for real-time safety of the audio callback.
    Section III-B equates absence of RadSan violations in anira's process method with real-time safety. RadSan intercepts system library calls such as malloc and pthread_mutex_lock, but may miss other blocking behaviors like page faults or priority inversion.
  • domain assumption Mean runtime per sample (RpS) is a valid proxy for real-time suitability despite the stated need for worst-case execution time.
    Section I states real-time audio requires minimizing maximum inference time, but Section III-C operationalizes the dependent variable as average RpS and all statistical models (Section IV-C) analyze means. This is acknowledged as a limitation in Section V.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications." pith.science (2026). https://pith.science/paper/JCBIUTMP

@misc{pith2026250612665,
  author       = {Pith},
  title        = {Pith review of: ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JCBIUTMP}},
  note         = {Machine review of arXiv:2506.12665}
}
read the original abstract

Numerous tools for neural network inference are currently available, yet many do not meet the requirements of real-time audio applications. In response, we introduce anira, an efficient cross-platform library. To ensure compatibility with a broad range of neural network architectures and frameworks, anira supports ONNX Runtime, LibTorch, and TensorFlow Lite as backends. Each inference engine exhibits real-time violations, which anira mitigates by decoupling the inference from the audio callback to a static thread pool. The library incorporates built-in latency management and extensive benchmarking capabilities, both crucial to ensure a continuous signal flow. Three different neural network architectures for audio effect emulation are then subjected to benchmarking across various configurations. Statistical modeling is employed to identify the influence of various factors on performance. The findings indicate that for stateless models, ONNX Runtime exhibits the lowest runtimes. For stateful models, LibTorch demonstrates the fastest performance. Our results also indicate that for certain model-engine combinations, the initial inferences take longer, particularly when these inferences exhibit a higher incidence of real-time violations.

Figures

Figures reproduced from arXiv: 2506.12665 by the authors.

Figure 1
Figure 1. Architectural overview of the anira library. The diagram illustrates the design and the principle components involved in the inference process. The overview is divided into three main sections: the Constructor, the Initialization Callback, and the Real-Time Audio Callback. The "Warm Up" operation, marked with 1 , is optional and allows for a set number of inferences before the start of the audio callback. The "Get N… view at source ↗
Figure 2
Figure 2. Schematical representation of the selected CNN model architecture. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Schematical representation of the selected RNN model architecture. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Schematical representation of the selected hybrid model architecture. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Total number of real-time violations detected by RadSan for each [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Estimated marginal means and 95% CI of the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Estimated marginal means and 95% CI of the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Estimated marginal means and 95% CI of the [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Estimated marginal means and 95% CI of the [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 33 canonical work pages

  1. [1]

    CNN architectures for large-scale audio classification,

    S. Hershey, S. Chaudhuri, D. P. W. Ellis, et al. , “CNN architectures for large-scale audio classification,” in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Process., New Orleans, LA, USA, Mar. 5–9, 2017, pp. 131–135

    work page 2017

  2. [2]

    A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation,

    R. M. Bittner, J. J. Bosch, D. Rubinstein, G. Meseguer-Brocal, and S. Ewert, “A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation,” in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Process. , Singapore, Singapore, Mar. 22– 27, 2022, pp. 781–785

    work page 2022

  3. [3]

    Wave-u-net: A multi-scale neural network for end-to-end audio source separation,

    D. Stoller, S. Ewert, and S. Dixon, “Wave-u-net: A multi-scale neural network for end-to-end audio source separation,” in Proc. Int. Soc. Music Inf. Retrieval , Paris, France, Sep. 23–27, 2018, pp. 334–340

    work page 2018

  4. [4]

    DDSP: Differentiable digital signal processing,

    J. Engel, L. Hantrakul, C. Gu, and A. Roberts, “DDSP: Differentiable digital signal processing,” 2020. arXiv: 2001.04643

    arXiv 2020

  5. [5]

    WaveNet: A generative model for raw audio,

    A. v. d. Oord, S. Dieleman, H. Zen, et al. , “WaveNet: A generative model for raw audio,” 2016. arXiv: 1609.03499

    arXiv 2016

  6. [6]

    RA VE: A variational autoencoder for fast and high-quality neural audio synthesis,

    A. Caillon and P. Esling, “RA VE: A variational autoencoder for fast and high-quality neural audio synthesis,” 2021. arXiv: 2111.05011

    arXiv 2021

  7. [7]

    Real-time guitar amplifier emulation with deep learning,

    A. Wright, E.-P. Damskägg, L. Juvela, and V . Välimäki, “Real-time guitar amplifier emulation with deep learning,” Applied Sciences , vol. 10, no. 3, p. 766, Jan. 2020. DOI: 10.3390/app10030766

    doi:10.3390/app10030766 2020

  8. [8]

    TensorFlow: A system for large- scale machine learning,

    M. Abadi, P. Barham, J. Chen, et al., “TensorFlow: A system for large- scale machine learning,” in Proc. USENIX Symp. Operating Syst. Des. and Implementation , Savannah, GA, USA, Nov. 2–4, 2016, pp. 265– 283

    work page 2016

Show all 39 references
  1. [9]

    PyTorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, et al., “PyTorch: An imperative style, high-performance deep learning library,” in Proc. Conf. Neural Inf. Process. Syst., Vancouver, Canada, Dec. 8–14, 2019, pp. 8024–8035

    2019

  2. [10]

    Interfacing real-time audio and file i/o,

    R. Bencina, “Interfacing real-time audio and file i/o,” in Proc. Aus- tralas. Comput. Music Conf. , Melbourne, Australia, Jul. 9–11, 2014, pp. 21–28

    2014

  3. [11]

    RTNeural: Fast neural inferencing for real-time sys- tems,

    J. Chowdhury, “RTNeural: Fast neural inferencing for real-time sys- tems,” 2021. arXiv: 2106.03037

    2021 arXiv

  4. [12]

    A comparison of deep learning inference engines for embedded real-time audio classification,

    D. Stefani, S. Peroni, and L. Turchet, “A comparison of deep learning inference engines for embedded real-time audio classification,” inProc. Int. Conf. Digit. Audio Effects , Vienna, Austria, Sep. 6–10, 2022, pp. 256–263

    2022

  5. [13]

    Schulz and V

    F. Schulz and V . Ackva, Anira - an architecture for neural network inference in real-time audio applications, version 0.1.2, Accessed: Sep. 14, 2024. [Online]. Available: https://github.com/anira-project/anira

    2024

  6. [14]

    Real-time embedded deep learning on elk audio OS,

    D. Stefani and L. Turchet, “Real-time embedded deep learning on elk audio OS,” in Proc. IEEE Int. Symp. Internet of Sounds , Pisa, Italy, Oct. 26–27, 2023, pp. 1–10

    2023

  7. [15]

    Pipeline for recording datasets and running neural networks on the bela em- bedded hardware platform,

    T. Pelinski, R. Diaz, A. L. B. Temprano, and A. McPherson, “Pipeline for recording datasets and running neural networks on the bela em- bedded hardware platform,” 2023. arXiv: 2306.11389

    2023 arXiv

  8. [16]

    14, 2024, 2015

    Kitware, Inc, CMake, the cross-platform, open-source build system , Accessed: Feb. 14, 2024, 2015. [Online]. Available: https : / / gitlab. kitware.com/cmake/cmake

    2024

  9. [17]

    Efficient neural networks for real-time modeling of analog dynamic range compression,

    C. J. Steinmetz and J. D. Reiss, “Efficient neural networks for real-time modeling of analog dynamic range compression,” in Proc. 152nd AES Convention, The Hague, The Netherlands, May 16–19, 2022, p. 10 596

    2022

  10. [18]

    24, 2024

    Tensorflow, TensorFlow lite, version 2.16.1, Accessed: Mar. 24, 2024. [Online]. Available: https://github.com/tensorflow/tensorflow/releases/ v2.16.1

    2024

  11. [19]

    24, 2024

    Microsoft, ONNX runtime , version 1.17.1, Accessed: Mar. 24, 2024. [Online]. Available: https : / / github . com / microsoft / onnxruntime / releases/tag/v1.17.1 (visited on 03/24/2024)

    2024

  12. [20]

    28, 2024

    PyTorch, LibTorch, version 2.2.2, Accessed: Mar. 28, 2024. [Online]. Available: https://github.com/pytorch/pytorch/releases/tag/v2.2.2

    2024

  13. [21]

    Carson, A

    A. Carson, A. Wright, J. Chowdhury, V . Välimäki, and S. Bilbao, Sample rate independent recurrent neural networks for audio effects processing, 2024. arXiv: 2406.06293

    2024 arXiv

  14. [22]

    Managing threads,

    A. Williams, “Managing threads,” in C++ concurrency in action: practical multithreading , Shelter Island, NY, USA: Manning, 2012, p. 30

    2012

  15. [23]

    Callback adaptation techniques,

    S. Letz, “Callback adaptation techniques,” GRAME - Computer Music Research Lab, hal-02158912, Nov. 1, 2001

    2001

  16. [24]

    28, 2024

    Google LLC, Google test , version v1.14.0, Accessed: Feb. 28, 2024. [Online]. Available: https://github.com/google/googletest/releases/tag/ v1.14.0

    2024

  17. [25]

    Google LLC, Google benchmark, version v1.8.3, Accessed: Feb. 28,

  18. [26]

    A review of neural network-based emulation of guitar amplifiers,

    T. Vanhatalo, P. Legrand, M. Desainte-Catherine, et al., “A review of neural network-based emulation of guitar amplifiers,”Applied Sciences, vol. 12, no. 12, p. 5894, Jun. 2022. DOI: 10.3390/app12125894

    2022 doi

  19. [27]

    Keith Bloemer, GuitarLSTM, Accessed: Feb. 2, 2024. [Online]. Avail- able: https://github.com/GuitarML/GuitarLSTM

    2024

  20. [28]

    ONNX runtime architecture,

    “ONNX runtime architecture,” ONNX Runtime. Accessed: Apr. 10,

  21. [29]

    Neural networks for real-time audio: Stateless LSTM,

    Keith Bloemer. “Neural networks for real-time audio: Stateless LSTM,” Medium. Accessed: Jan. 27, 2024. (May 5, 2021), [Online]. Available: https://towardsdatascience.com/neural-networks-for-real-time-audio- stateless-lstm-97ecd1e590b8

    2024

  22. [30]

    24, 2023), [Online]

    (Jan. 24, 2023), [Online]. Available: https : / / onnxruntime . ai / docs/reference/high-level-design.html

    2023

  23. [31]

    Trevelyan, A

    D. Trevelyan, A. Barker, and C. Apple, Realtime sanitizer, Accessed: Jul. 12, 2024. [Online]. Available: https : / / github . com / realtime - sanitizer/radsan

    2024

  24. [32]

    Address- Sanitizer: A fast address sanity checker,

    K. Serebryany, D. Bruening, A. Potapenko, and D. Vyukov, “Address- Sanitizer: A fast address sanity checker,” in Proc. USENIX Ann. Tech. Conf., Boston, MA, USA, Jun. 13–15, 2012, pp. 309–318

    2012

  25. [33]

    Grosjean and F

    P. Grosjean and F. Ibanez, Pastecs: Package for analysis of space-time ecological series, in collab. with M. Etienne, version 1.4.2, Accessed: May. 10, 2024. [Online]. Available: https : / / github. com / SciViews / pastecs

    2024

  26. [34]

    Schulz and V

    F. Schulz and V . Ackva, Anira-benchmark-evaluation: Statistical eval- uation of benchmarks made with the anira library , version 0.0.1, Accessed: Jul. 26, 2024. [Online]. Available: https://github.com/anira- project/anira-benchmark-evaluation

    2024

  27. [35]

    Fitting linear mixed- effects models using lme4,

    D. Bates, M. Mächler, B. Bolker, and S. Walker, “Fitting linear mixed- effects models using lme4,” J. Statistical Softw., vol. 67, no. 1, 2015. DOI: 10.18637/jss.v067.i01

    2015 doi

  28. [36]

    10, 2024, Vienna, Austria

    R Core Team, R: A language and environment for statistical computing, version 4.3.2, Accessed: May. 10, 2024, Vienna, Austria. [Online]. Available: https://www.R-project.org/

    2024

  29. [37]

    A simple sequentially rejective multiple test procedure,

    Sture Holm, “A simple sequentially rejective multiple test procedure,” Scand. J. Statist. , vol. 6, no. 2, pp. 65–70, 1979

    1979

  30. [38]

    Lenth, Emmeans: Estimated marginal means, aka least- squares means , version 1.10.1, Accessed: May

    Russell V . Lenth, Emmeans: Estimated marginal means, aka least- squares means , version 1.10.1, Accessed: May. 10, 2024. [Online]. Available: https://CRAN.R-project.org/package=emmeans

    2024

  31. [2024]

    Available: https : / / github

    [Online]. Available: https : / / github . com / google / benchmark / releases/tag/v1.8.3

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.