Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Twill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platforms

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A run-time scheduler that combines layer-level affinity mapping, migration, freezing, and DVFS can cut compound AI inference latency on edge hardware by up to 54 percent while staying within the power budget.

desk verdict Twill tackles a real scheduling gap with a sensible runtime design, but its headline latency claim is overstated and rests on unvalidated self-implemented baselines. read the letter →

arxiv 2507.00491 v1 pith:DBAT6AOZ submitted 2025-07-01 cs.MA cs.AIcs.CVcs.PF

Zain Taufique , Aman Vyas , Antonio Miele , Pasi Liljeberg , Anil Kanduri This is my paper
classification cs.MAcs.AIcs.CVcs.PF
keywords compoundAIsystemsheterogeneousmobileedgeinferenceschedulingDLAcompatibilitytaskmigrationfreezingDVFSpowercappingDNNandtransformerconcurrency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a compound AI system—a chain of DNNs, transformer encoders, and LLMs that arrive asynchronously—can be served on a power-limited mobile edge board without sacrificing latency. Its central claim is that a runtime controller that jointly tunes four actions (affinity-aware cluster mapping, cluster migration, priority-based task freezing, and DVFS) keeps concurrent inference requests from colliding on shared GPU and DLA clusters, and does so within the thermal power budget. If this is right, on-device compound AI services such as AR/VR assistants and interactive agents do not need cloud fallback or design-time workload assumptions. The paper supports the claim with deployments of five workload mixes on a Jetson Orin NX board, reporting per-mix execution-time reductions of 38, 54, 22, 37, and 31 percent against the three baseline schedulers it considers.

What carries the argument

The load-bearing mechanism is the Signature map produced by the Model Interpreter: a per-layer compatibility record that assigns GPU by default and adds DLA only when the layer precision, operation type, and parameter constraints match the accelerator's manual specification. This map lets the Controller know at request arrival time which cluster each model prefers, and the Controller's Decide phase turns that preference into one of four actuations: map to a free cluster, migrate a running task to its second-best cluster, freeze a lower-priority task to free its cluster, or freeze the newcomer. A linear DVFS model then raises GPU frequency when power headroom exists, keeping the board at or below the thermal design power while extracting extra performance.

What would settle it

Run the original implementations (or the original authors' released code) of the three baseline schedulers on the same five workload mixes and the same embedded board at the same 10 W power budget, and compare end-to-end execution time and power traces against Twill; if Twill fails to reproduce per-mix reductions of 38, 54, 22, 37, and 31 percent under identical conditions, the central claim is refuted.

Watch

Extended reading notes

Core claim

Twill is presented as an online heuristic scheduler, not an offline optimizer. On each inference request, a Model Interpreter parses the model's serialized description, checks every layer against a DLA compatibility matrix built from the hardware reference, and builds a Signature map that lists, per layer, whether it can run on the GPU, the DLA, or both. GPU is assumed compatible with everything; DLA is compatible only with layers whose precision, operation type, and convolution or fully-connected parameter ranges are supported. Because transformers rely on multi-head attention, layer normalization, and GELU activations that DLA does not implement without GPU fallback, the affinity signal effectively steers DNNs toward DLA and transformers or LLMs toward GPU. The Controller then runs an Analyze-Decide-Deploy loop: on a new application or a freed cluster, it maps the arriving task to its most-preferred free cluster; if that cluster is busy, it remaps the occupant to a suitable free cluster, or freezes a lower-priority occupant to make room, or freezes the newcomer; afterward it adjusts GPU frequency with a linear power model to use the remaining headroom up to the thermal design power. Over five mixes combining DNNs, vision and text transformers, and LLMs, the paper reports that Twill achieves the lowest end-to-end execution time, with per-mix reductions of 38, 54, 22, 37, and 31 percent and average reductions of 20, 19, and 34 percent against the three baseline schedulers, while only one baseline repeatedly violates the 10 W power budget.

Load-bearing premise

The comparison assumes that the authors' reimplementations of the three baseline schedulers faithfully reproduce the behavior of the original systems described in the cited papers; if any baseline is weaker than its original, the reported latency reductions are inflated.

Editorial extensions

If this is right

  • Interactive compound-AI services such as AR/VR assistants and on-device translation can run on power-limited edge hardware even when DNN, transformer, and LLM requests arrive asynchronously.
  • The controller's policy is independent of the specific model zoo, so new DNNs, transformers, or LLMs can be scheduled as long as their layers can be checked against the compatibility matrix.
  • Priority awareness lets prompt-driven transformer and LLM inference preempt batch DNN inference, matching the latency expectations of conversational interfaces.
  • Scaling GPU frequency up to the thermal design power means the latency gains come without violating the power budget, which is necessary for battery-powered and thermally constrained devices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the abstract's claim of 54 percent on average does not match the body's per-mix reductions of 38, 54, 22, 37, and 31 percent and its average reductions of 20, 19, and 34 percent against the three baselines; readers should quote the per-mix figures.
  • Beyond the paper, the same policy could be tried on boards where multiple kernels can share a cluster concurrently; Twill assumes at most one model per cluster at a time, and allowing co-residency would require extending the policy but might lower latency further.
  • Beyond the paper, learning per-layer execution costs online could replace the manual compatibility matrix and let the scheduler adapt to new accelerators that lack published operator tables.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Twill, an online runtime scheduler for compound AI (cAI) inference workloads on heterogeneous mobile edge platforms, specifically the Jetson Orin NX with GPU and DLA. Twill profiles each model's layer-cluster compatibility, maps and migrates models between GPU and DLA based on affinity, freezes lower-priority tasks under contention, and scales GPU frequency to respect a TDP power budget. The evaluation uses five workload mixes of DNNs, transformers, and LLMs and compares against reimplementations of Tango, MapFormer, and Band, reporting reductions in overall execution time of 38%, 54%, 22%, 37%, and 31% for Mix-1 to Mix-5, with per-baseline averages of 20%, 19%, and 34%. The abstract claims "54% on average," which is not supported by the body numbers.

Significance. If the claimed improvements hold, Twill addresses a real gap: prior edge schedulers focus on multi-DNN or transformer-only workloads and often assume design-time knowledge of request arrivals, while Twill handles dynamic arrival of mixed DNN, transformer, and LLM workloads. The work is evaluated on real hardware rather than in simulation, includes a DLA compatibility analysis for transformer fallback behavior, and uses contemporary models such as DeepSeek-R1 and Gemma-3. The main deliverable is an online heuristic system rather than a formal guarantee, so the strength of the empirical claim depends on baseline fidelity and measurement methodology.

major comments (4)
  1. [IV-A, IV-B] Section IV-A states that MapFormer, Tango, and Band were reimplemented by the authors (a transformer-based estimator, a Gymnasium-based latency model, and a model analyzer for subgraphs), but no validation is provided that these reimplementations reproduce the original systems' behavior. Because every latency improvement reported in Section IV-B is a relative improvement over these reimplementations, the central quantitative claim is only as strong as the baseline fidelity. Please either run the original implementations where available, provide quantitative validation against published latency or power results from the original papers, or make the reimplementation code available for independent checking.
  2. [Abstract, IV-B, V] The abstract says "reducing inference latency by 54% on average," but Section IV-B reports reductions of 38%, 54%, 22%, 37%, and 31% for Mix-1 to Mix-5 and average reductions of 20%, 19%, and 34% relative to Tango, MapFormer, and Band; the mean of the five per-mix figures is 36.4%. The conclusion correctly says "up to 54%." The abstract should be corrected, and the computation of any claimed average should be explicitly defined.
  3. [IV-B] Section IV-B reports single measured values for each mix and strategy, with no number of runs, variance, or error bars. The reported differences are plausible, but without repetition or confidence information it is difficult to assess whether they exceed run-to-run noise on the Orin NX platform. Please report means and standard deviations over multiple runs and state the number of repetitions for each configuration.
  4. [III-D (Algorithm 2)] Algorithm 2, Lines 40-42: the DVFS update computes freq_new = ((TDP - P_prec) / (P_curr - P_prec)) * get_freq(cluster). If P_curr equals P_prec, the denominator is zero; if P_curr is less than P_prec, the computed frequency is negative. No clamping or fallback is described, yet the abstract and Section IV claim the power budget is always honored. The control law needs a guard or clamp and a discussion of transient power excursions.
minor comments (5)
  1. [III-C] The subsection title "Comparability Matrix" should be "Compatibility Matrix."
  2. [IV-B] In Mix-3, the workload is defined as containing ViT-large, but the narrative discusses "ViT-base"; please correct the model name. In Mix-4, the text uses "VGG-1" where VGG-19 is intended.
  3. [II-B] Scenario-2 uses "unfreezed" and should say "unfrozen."
  4. [IV-B] The phrase "up to 38%, 54%, 22%, 37%, and 31%" is grammatically imprecise because "up to" should precede a single maximum value; consider "by 38%, 54%, ... for Mix-1 to Mix-5, respectively."
  5. [Figures 6-10] The captions for Figures 6 and 9-10 do not fully define axis units or the meaning of shaded regions; please specify units (ms or s, W) and any annotations.

Circularity Check

0 steps flagged · score 0.0 of 10

No formal circularity: Twill's latency reductions are measured on hardware; the abstract/body discrepancy and unvalidated baseline reimplementations are verification risks, not circular derivations.

full rationale

The paper's central claim is empirical: Twill actuates affinity mapping, migration, freezing, and DVFS on the Jetson Orin NX and reports lower measured latency and power than three baselines. No equation in the paper encodes the reported reductions, and no fitted parameter is renamed as a prediction. Algorithm 1 and Algorithm 2 are open design heuristics evaluated on a physical platform; the DVFS formula (Sec. III-D, lines 40-42) is a linear power-budget calculation, not a self-fulfilling definition. The only author self-citation is Tango [16], used as a comparison baseline; the paper states 'We used Gymnasium library [43] to implement Tango's latency estimation model' (Sec. IV-A). This is a self-evaluation fidelity risk because the reimplementation may not match the original system, but Twill's advantage is not deduced from Tango's code or results, so it is not a circular step. The abstract's '54% on average' is contradicted by the body's per-mix reductions of 38%, 54%, 22%, 37%, and 31% and by the stated averages of 20%, 19%, and 34% versus Tango, MapFormer, and Band (Sec. IV-B); this is an internal-consistency reporting error, not circular reasoning. No self-definitional reduction, fitted-input-as-prediction, uniqueness-imported-from-authors, or ansatz-smuggled-via-citation pattern appears. The paper is therefore not circular, though its quantitative claims would be strengthened by validating baseline reimplementations and correcting the abstract's average figure.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No numerical parameters were fitted to data. The hand-chosen elements are the application priority ordering, the workload arrival schedules used in the mixes, and the TDP budget of 10 W. The axioms above cover the main unstated technical and evaluation assumptions on which the central claim depends, including the accuracy of the compatibility matrix, the negligible cost of freezing, the linear DVFS model, and the fidelity of the self-implemented baselines.

assumptions (6)
  • domain assumption The DLA compatibility matrix derived from NVIDIA documentation is accurate for the tested models and correctly predicts per-layer GPU fallbacks.
    Algorithm 1 uses dla_matrix to decide whether each layer can run on DLA; if this matrix is wrong, the affinity-based mapping and migration decisions are invalid. See Section III-C.
  • domain assumption The GPU can execute all operation types and is the default preferred cluster for every layer.
    Algorithm 1 assigns GPU compatibility by default and all scheduling decisions inherit this. See Section III-C, Algorithm 1 lines 4-5.
  • ad hoc to paper Task freezing and unfreezing impose negligible overhead relative to inference execution time.
    The paper does not measure the cost of suspending and resuming a PyTorch inference. This assumption is load-bearing because freezing is a key knob in Scenarios 1-2 and Mixes 2-5. See Section III-D.
  • domain assumption The DVFS power model is linear between the measured power points.
    Algorithm 2 lines 41-42 compute the new frequency by linear interpolation using the previous and current power readings. This ignores nonlinear power-frequency behavior and temperature effects. See Section III-D, Algorithm 2.
  • ad hoc to paper The authors' reimplementations of Tango, MapFormer, and Band faithfully represent the original systems.
    Section IV-A says the authors implemented a transformer-based estimator for MapFormer, a Gymnasium-based latency model for Tango, and a clone of Band's model analyzer. No validation against original implementations is provided, so the comparison results rest on this assumption.
  • domain assumption User-defined application priorities are set as described and remain stable during each workload mix.
    The paper prioritizes transformers and LLMs over batch DNNs in its scenarios. This priority ordering drives the freeze and migrate decisions in Algorithm 2, and the paper does not study sensitivity to alternative priority assignments. See Section III-C and III-D.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Twill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platforms." pith.science (2026). https://pith.science/paper/DBAT6AOZ

@misc{pith2026250700491,
  author       = {Pith},
  title        = {Pith review of: Twill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platforms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DBAT6AOZ}},
  note         = {Machine review of arXiv:2507.00491}
}
read the original abstract

Compound AI (cAI) systems chain multiple AI models to solve complex problems. cAI systems are typically composed of deep neural networks (DNNs), transformers, and large language models (LLMs), exhibiting a high degree of computational diversity and dynamic workload variation. Deploying cAI services on mobile edge platforms poses a significant challenge in scheduling concurrent DNN-transformer inference tasks, which arrive dynamically in an unknown sequence. Existing mobile edge AI inference strategies manage multi-DNN or transformer-only workloads, relying on design-time profiling, and cannot handle concurrent inference of DNNs and transformers required by cAI systems. In this work, we address the challenge of scheduling cAI systems on heterogeneous mobile edge platforms. We present Twill, a run-time framework to handle concurrent inference requests of cAI workloads through task affinity-aware cluster mapping and migration, priority-aware task freezing/unfreezing, and DVFS, while minimizing inference latency within power budgets. We implement and deploy our Twill framework on the Nvidia Jetson Orin NX platform. We evaluate Twill against state-of-the-art edge AI inference techniques over contemporary DNNs and LLMs, reducing inference latency by 54% on average, while honoring power budgets.

Figures

Figures reproduced from arXiv: 2507.00491 by the authors.

Figure 1
Figure 1. Exemplar cAI system. (a) Task graph for cAI system chaining multiple [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Inference latency of cAI system across different execution strategies [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Various knob actuation scenarios, highlighting lack of priority and [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Twill Framework including Model Interpreter to analyze the model-cluster affinity and Controller module performing run-time control knobs actuation. exploration, and actuates the defined knobs to minimize the overall latency while guaranteeing a given power budget. As …
Figure 5
Figure 5. Figure 5: Supported operations on GPU and DLA clusters of Orin NX platform. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Inference latency and run-time power consumption of different workload mixes. The first row represents the application latency, and the second row [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Overall cAI system execution time for mix 1-5. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Waiting time of applications in the execution queue across mixes. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 10
Figure 10. Figure 10: Inference latency and run-time power consumption of [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 33 canonical work pages

  1. [1]

    The shift from models to compound ai systems,

    M. Zaharia, O. Khattab, L. Chen, J. Q. Davis, H. Miller, C. Potts, J. Zou, M. Carbin, J. Frankle, N. Rao, and A. Ghodsi, “The shift from models to compound ai systems,” https://bair.berkeley.edu/blog/2024/02/ 18/compound-ai-systems/, 2024

    work page 2024

  2. [2]

    Are more llm calls all you need? towards the scaling properties of compound ai systems,

    L. Chen, J. Q. Davis, B. Hanin, P. Bailis, I. Stoica, M. A. Zaharia, and J. Y . Zou, “Are more llm calls all you need? towards the scaling properties of compound ai systems,” Advances in Neural Information Processing Systems, vol. 37, pp. 45 767–45 790, 2024

    work page 2024

  3. [3]

    Optimizing model selection for compound ai systems,

    L. Chen, J. Q. Davis, B. Hanin, P. Bailis, M. Zaharia, J. Zou, and I. Stoica, “Optimizing model selection for compound ai systems,” arXiv preprint arXiv:2502.14815, 2025

    arXiv 2025

  4. [4]

    Can large language models really improve by self-critiquing their own plans?

    K. Valmeekam, M. Marquez, and S. Kambhampati, “Can large language models really improve by self-critiquing their own plans?”arXiv preprint arXiv:2310.08118, 2023

    arXiv 2023

  5. [5]

    Autogen: Enabling next-gen llm applications via multi-agent conversation,

    Q. Wu, G. Bansal, J. Zhang, Y . Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu et al. , “Autogen: Enabling next-gen llm applications via multi-agent conversation,” arXiv preprint arXiv:2308.08155 , 2023

    arXiv 2023

  6. [6]

    Drive as you speak: Enabling human-like interaction with large language models in au- tonomous vehicles,

    C. Cui, Y . Ma, X. Cao, W. Ye, and Z. Wang, “Drive as you speak: Enabling human-like interaction with large language models in au- tonomous vehicles,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 902–909

    2024

  7. [7]

    Langchain,

    H. Chase, “Langchain,” 2022, release date: 2022-10-17. [Online]. Available: https://github.com/langchain-ai/langchain

    work page 2022

  8. [8]

    Langbase: AI-Powered Multilingual Database,

    Langbase, “Langbase: AI-Powered Multilingual Database,” 2025. [Online]. Available: https://langbase.com/

    work page 2025

Show all 43 references
  1. [9]

    Autoscale: Energy efficiency optimization for stochas- tic edge inference using reinforcement learning,

    Y . G. Kim et al., “Autoscale: Energy efficiency optimization for stochas- tic edge inference using reinforcement learning,” Proc. of Int. Symp. on Microarchitecture, MICRO, pp. 1082–1096, 2020

    2020

  2. [10]

    Band: coordinated multi-dnn inference on heterogeneous mobile processors,

    J. Seong et al., “Band: coordinated multi-dnn inference on heterogeneous mobile processors,” in Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services , 2022

    2022

  3. [11]

    Automated exploration and implementation of distributed cnn inference at the edge,

    X. Guo et al., “Automated exploration and implementation of distributed cnn inference at the edge,” IEEE Internet of Things Journal , vol. 10, no. 7, pp. 5843–5858, April 2023

    2023

  4. [12]

    Apple intelligence,

    Apple Inc., “Apple intelligence,” 2024, accessed: 2025-04-21. [Online]. Available: https://www.apple.com/apple-intelligence/

    2024

  5. [13]

    Meta ray-ban smart glasses: Next generation smart eyewear,

    M. P. Inc., “Meta ray-ban smart glasses: Next generation smart eyewear,” Meta Technology Review , 2023. [Online]. Available: https: //www.ray-ban.com/usa/ray-ban-meta-ai-glasses

    2023

  6. [14]

    Apple vision pro: Spatial computing device,

    A. Inc., “Apple vision pro: Spatial computing device,” 2024. [Online]. Available: https://www.apple.com/apple-vision-pro/

    2024

  7. [15]

    Qualcomm launches its next-generation xr and ar platforms,

    Qualcomm, “Qualcomm launches its next-generation xr and ar platforms,” 2023, accessed: 2025-04-08. [On- line]. Available: https://www.qualcomm.com/news/releases/2023/09/ qualcomm-launches-its-next-generation-xr-and-ar-platforms--enabl

    2023

  8. [16]

    Tango: Low latency multi-dnn inference on heterogeneous edge platforms,

    Z. Taufique, A. Vyas, A. Miele, P. Liljeberg, and A. Kanduri, “Tango: Low latency multi-dnn inference on heterogeneous edge platforms,” in 2024 IEEE 42nd International Conference on Computer Design (ICCD), 2024, pp. 300–307

    2024

  9. [17]

    OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload,

    A. Karatzas and I. Anagnostopoulos, “OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload,” in Proc. of ACM/IEEE Design Automation Conf. (DAC) , 2023, pp. 1–6

    2023

  10. [18]

    MOC: Multi-Objective Mobile CPU-GPU Co-Optimization for Power-Efficient DNN Inference,

    Y . Wu, Y . Gong, Z. Zhan, G. Yuan, Y . Li, Q. Wang, C. Wu, and Y . Wang, “MOC: Multi-Objective Mobile CPU-GPU Co-Optimization for Power-Efficient DNN Inference,” in Proc. of ACM/IEEE Intl. Conf. on Computer Aided Design (ICCAD) , 2023, pp. 1–10

    2023

  11. [19]

    High-Throughput CNN Inference on Embedded ARM Big.LITTLE Multicore Processors,

    S. Wang, G. Ananthanarayanan, Y . Zeng, N. Goel, A. Pathania, and T. Mitra, “High-Throughput CNN Inference on Embedded ARM Big.LITTLE Multicore Processors,” IEEE Trans. on Computer-Aided Design of Integrated Circuits and Systems , vol. 39, no. 10, pp. 2254– 2267, 2019

    2019

  12. [20]

    Easter: Learning to split transformers at the edge robustly,

    X. Guo, Q. Jiang, Y . Shen, A. D. Pimentel, and T. Stefanov, “Easter: Learning to split transformers at the edge robustly,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 43, no. 11, pp. 3626–3637, 2024

    2024

  13. [21]

    Pipebert: High-throughput bert inference for arm big.little multi-core processors,

    H.-Y . Chang, S. H. Mozafari, C. Chen, J. J. Clark, B. H. Meyer, and W. J. Gross, “Pipebert: High-throughput bert inference for arm big.little multi-core processors,” J. Signal Process. Syst., vol. 95, p. 877–894, Oct. 2022

    2022

  14. [22]

    Shared memory-contention-aware con- current dnn execution for diversely heterogeneous system-on-chips,

    I. Dagli and M. E. Belviranli, “Shared memory-contention-aware con- current dnn execution for diversely heterogeneous system-on-chips,” in Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming. ACM, Feb. 2024, p. 243–256

    2024

  15. [23]

    Mapformer: Attention-based multi-dnn manager for throughout & power co-optimization on em- bedded devices,

    A. Karatzas and I. Anagnostopoulos, “Mapformer: Attention-based multi-dnn manager for throughout & power co-optimization on em- bedded devices,” in Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design , ser. ICCAD ’24. New York, NY , USA: Associat...

    2025

  16. [24]

    Rknn toolkit operator support list (v1.7.5),

    L. Rockchip Electronics Co., “Rknn toolkit operator support list (v1.7.5),” https://github.com/rockchip-linux/rknn-toolkit/blob/master/ docs/RKNN OP Support V1.7.5.md, 2023, accessed: 2025-04-19

    2023

  17. [25]

    Working with dla,

    N. Corporation, “Working with dla,” 2025. [On- line]. Available: https://docs.nvidia.com/deeplearning/tensorrt/latest/ inference-library/work-with-dla.html

    2025

  18. [26]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” CoRR, vol. abs/1409.1556, 2014

    2014 arXiv

  19. [27]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Pro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolo...

    2019

  20. [28]

    Nvidia jetson Orin,

    NVIDIA, “Nvidia jetson Orin,” https://www.nvidia.com/en-us/ autonomous-machines/embedded-systems/jetson-orin/, 2024

    2024

  21. [29]

    Interference-aware dnn serving on heterogeneous processors in edge systems,

    Y . Kim, I. Kim, K. Choi, J. Ahn, J. Park, and J. Huh, “Interference-aware dnn serving on heterogeneous processors in edge systems,” in2024 IEEE 42nd International Conference on Computer Design (ICCD) , 2024

    2024

  22. [30]

    Rankmap: Priority- aware multi-dnn manager for heterogeneous embedded devices,

    A. Karatzas, D. Stamoulis, and I. Anagnostopoulos, “Rankmap: Priority- aware multi-dnn manager for heterogeneous embedded devices,” 2024

    2024

  23. [31]

    Energy-aware scenario-based mapping of deep learning applications onto heterogeneous processors under real-time constraints,

    J. Kim and S. Ha, “Energy-aware scenario-based mapping of deep learning applications onto heterogeneous processors under real-time constraints,” IEEE Transactions on Computers, vol. 72, no. 6, pp. 1666– 1680, 2023

    2023

  24. [32]

    Axonn: Energy-aware execution of neural network inference on multi-accelerator heterogeneous socs,

    I. Dagli, A. Cieslewicz, J. McClurg, and M. E. Belviranli, “Axonn: Energy-aware execution of neural network inference on multi-accelerator heterogeneous socs,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, 2022, pp. 1069–1074

    2022

  25. [33]

    Onnx runtime,

    O. R. developers, “Onnx runtime,” https://onnxruntime.ai/, 2021

    2021

  26. [34]

    Gaussian error linear units (gelus),

    D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” 2023

    2023

  27. [35]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  28. [36]

    Efficientnet: Rethinking model scaling for convolutional neural networks,

    M. Tan and Q. V . Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in Proceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 97, 2019, pp. 6105–6114

    2019

  29. [37]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  30. [38]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    DeepSeek-AI, D. Guo, and et al, “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025

    2025

  31. [39]

    Pytorch installation guide,

    P. Team, “Pytorch installation guide,” https://pytorch.org/get-started/ locally/, 2024

    2024

  32. [40]

    Torchvision,

    “Torchvision,” https://pytorch.org/vision/stable/index.html, PyTorch Team, 2024

    2024

  33. [41]

    Hugging face: The ai community building the future of machine learning,

    H. Face, “Hugging face: The ai community building the future of machine learning,” 2025, accessed: 2025-04-18. [Online]. Available: https://huggingface.co/

    2025

  34. [42]

    Ollama: Run large language models locally,

    O. Inc., “Ollama: Run large language models locally,” 2025, accessed: 2025-04-18. [Online]. Available: https://ollama.com/

    2025

  35. [43]

    Gymnasium,

    M. Towers et al. , “Gymnasium,” 2023. [Online]. Available: https: //zenodo.org/record/8127025

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.