Pith. sign in

REVIEW 4 major objections 4 minor 46 references

Exploring the Dynamic Scheduling Space of Real-Time Generative AI Applications on Emerging Heterogeneous Systems

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For real-time generative AI on CPU-GPU-NPU chips, scheduling policy alone moves average deadline violation rates by 41.7 percentage points, while deadline-first and arrival-order policies fail in opposite directions.

desk verdict A useful first qualitative map of RTGen scheduling on a real NPU-GPU SoC, but the headline 41.7% number comes from an unvalidated simulator and is not reproducible from the paper as written. read the letter →

arxiv 2507.14715 v1 pith:TBLPLPCN submitted 2025-07-19 cs.LG

Rachid Karami , Rajeev Patwari , Hyoukjun Kwon , Ashish Sirasao This is my paper · ORCID
classification cs.LG
keywords real-timegenerativeAIRTGenworkloadsheterogeneousSoCNPU-GPUschedulingLLMprefillanddecodedeadlineviolationratetime-to-first-tokendynamicpolicies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies a new class of edge workload it calls RTGen: applications that run a large language model alongside real-time models such as segmentation, super-resolution, and object detection, all on one heterogeneous chip with CPU, GPU, and NPU. The authors profile each model on every backend of an AMD Ryzen AI system, build a simulator driven by that latency data, and test five scheduling policies on four realistic scenarios. Their central claim is that the scheduling decision alone changes outcomes by a large margin, up to a 41.7% difference in average deadline violation rates, and that standard policies fail in opposite ways: deadline-first scheduling starves the LLM, while arrival-order scheduling misses frame deadlines. They conclude that schedulers for these systems must be workload-aware and dynamic rather than static.

What carries the argument

The engine of the study is a Python-level runtime simulator whose inputs are a scenario's models, their input tensors, each model's target frame rate, and a scheduling policy. A latency database, populated by profiling every model on the CPU, GPU, and NPU of an AMD Ryzen AI 9 365 system in isolation (via the RyzenAI software stack, ONNX Runtime, and the ONNX Runtime Generative AI framework), is what the scheduler consults to estimate the cost of running each layer or model on each backend before dispatching it. The five evaluated policies are organized along four axes the paper defines — deadline awareness, dynamic hardware selection, heterogeneity awareness, and GenAI awareness — and the 'first token first' policy is the only one that combines all four: it protects the LLM's prefill by treating the first-token deadline as high priority, then falls back to deadline-first dynamic scheduling for the decode stage.

What would settle it

Run scenario D on a real Ryzen AI device with the RAG assistant, segmentation, object detection, and super-resolution models active concurrently under the FTF policy, and compare the measured deadline violation rate, time-to-first-token, and time-per-token against the simulated 20.8%, 1577.9 ms, and 339.9 ms. A cheaper check: measure a single LLM decode layer on the GPU while the NPU is simultaneously executing super-resolution inference and compare it with the isolated GPU latency; any material gap means the simulator's cost model misses the interference that real concurrent execution produces.

Watch

Extended reading notes

Core claim

On the evaluated Ryzen AI system, the authors find that no backend is uniformly best: the NPU wins for compute-bound work such as LLM prefill (about 3$\times$ faster than the GPU) and for CNN models such as super-resolution, while the GPU wins for memory-bandwidth-bound work such as LLM decode (about 7.5$\times$ faster than the NPU). Because the LLM's preferred backend flips between its prefill and decode stages, fixed ahead-of-time placement is structurally inadequate. Simulating four realistic multi-model scenarios, the paper shows that an earliest-deadline-first policy, with or without dynamic backend selection, lets the LLM be repeatedly preempted by closer-deadline tasks and starves it completely; a first-come-first-served policy preserves the LLM's standalone quality but pushes average frame-deadline violation rates to roughly 57.6% in the two busiest scenarios; and a GenAI-aware 'first token first' policy, which treats the first token as a high-priority deadline, restores standalone-level time-to-first-token while cutting deadline violations substantially (47.8% on average in scenarios C and D relative to FCFS), at the price of a 5.7$\times$–7.4$\times$ increase in time-per-token. The paper's central claim is that scheduling decisions of this magnitude are the rule, not the exception, for RTGen workloads on heterogeneous SoCs, so schedulers must be workload-aware, deadline-aware, and dynamic.

Load-bearing premise

The load-bearing premise is that every scheduled layer or model finishes in exactly its isolated profiled latency, with no slowdown from concurrent execution, no memory-system contention, and no preemption overhead, and this premise is never validated against an end-to-end run of the concurrent scenarios on the real hardware.

Editorial extensions

If this is right

  • In any scenario that couples an LLM with periodic real-time models, an earliest-deadline-first policy starves the LLM regardless of whether backend choice is static or dynamic, so deadline compliance alone cannot be the scheduling objective.
  • Arrival-order scheduling is not a safe fallback: it preserves LLM responsiveness but pushes average frame-deadline violation rates to about 57.6% in the two most loaded scenarios.
  • Treating the first token as a high-priority deadline (the FTF policy) restores standalone-level time-to-first-token while cutting average deadline violation rates in scenarios C and D by 47.8% relative to FCFS.
  • Dynamic backend selection relocates missed deadlines between models (e.g., from segmentation to object detection in scenario D) instead of removing them, so its benefit is scenario- and model-dependent.
  • Input sequence length changes scheduling outcomes directly: lengthening the prompt in scenario D from 32 to 2048 tokens raises the FTF deadline violation rate from 1.7% to 10.7%, because the longer prefill occupies the NPU for longer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural follow-up the paper's data motivates is pairing FTF's prefill protection with decode placement that keeps more decode layers on the GPU, the 7.5$\times$-faster backend, targeting the 5.7$\times$–7.4$\times$ time-per-token penalty the paper measures rather than accepting it.
  • Because all numbers come from a simulator that treats isolated profiled latencies as ground truth, the 41.7 percentage-point spread is best read as how much a scheduler alone can buy before hardware-level interference is counted; real-silicon measurements could compress or widen it.
  • The authors' observation that short prompts had 95.1% of their decode layers pushed to the NPU and suffered high time-per-token suggests a concrete design idea the study enables: use input length as an explicit signal in backend selection.
  • The same four-scenario methodology would transfer to multi-tenant LLM serving or mixture-of-experts models, where the preferred backend can shift at a finer granularity, testing whether the 'NPU for prefill, GPU for decode' pattern is a stable scheduling signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper defines a class of multi-model real-time generative AI (RTGen) workloads, constructs four application scenarios (AI assistant, gaming, game streaming, video conferencing), profiles the component models on an AMD Ryzen AI SoC, and evaluates five scheduling policies (FCFS-AOT, FCFS-DYN, EDF-AOT, EDF-DYN, FTF) using a Python-level runtime simulator. The central claim is that scheduling decisions significantly affect workload performance, quantified as a 41.7% average difference in deadline violation rates, and that workload-aware dynamic heterogeneous scheduling is therefore necessary for on-device RTGen. The paper additionally reports TTFT/TPT results, sequence-length sensitivity, and per-backend decode-layer dispatch percentages.

Significance. The workload characterization and the observation that the preferred backend shifts between LLM prefill and decode are genuinely useful and clearly presented. The taxonomy of scheduler features (deadline awareness, dynamic hardware selection, heterogeneity awareness, GenAI awareness) provides a clean framework for comparing policies. If the simulator were validated against real execution, the paper would be a valuable case study for the systems community. As it stands, the quantitative headline is not established because the simulator ignores inter-backend interference and scheduling overhead and is not validated against the real platform; the qualitative conclusion that scheduling matters is plausible and likely correct, but the specific numbers should be treated with caution.

major comments (4)
  1. [§4.2 and §5.2] The central quantitative claim rests entirely on the simulator described in Section 4.2, which assumes each scheduled layer or model executes at its isolated profiled latency with no cross-backend interference, no dispatch/launch overhead, and no preemption cost. The simulator is not validated against the Ryzen AI platform in Table 3. On an SoC where the NPU and GPU share DRAM bandwidth, and with frame deadlines as tight as 8.33 ms versus an LLM prefill layer of 98.62 ms, unmodeled contention or overhead of a few milliseconds can change which policy meets deadlines. The paper should either validate the simulator end-to-end against the real system for at least one scenario or provide a sensitivity analysis with explicit contention and overhead models.
  2. [Abstract and §5.2] The headline 41.7% average difference in deadline violation rates is not derivable from Table 5 as printed, and no definition of 'average difference' is given. For example, one natural aggregation of the max-minus-min violation rates across scenarios B, C, and D gives approximately 43.1 percentage points, not 41.7%. The paper should state the exact aggregation, report per-policy means and variances over repeated simulation runs, and include raw simulated deadline-violation counts so that the headline number can be reproduced.
  3. [Table 5] The EDF rows report a 0.0% deadline violation rate while showing '-' for TTFT and TPT, because the LLM is starved. Reporting only the deadline violation rate therefore makes EDF appear best on the real-time metric even though it fails the workload by not completing LLM inference. The paper should report an additional liveness or completion metric, such as the fraction of LLM requests that reach the first token, or present TTFT/TPT with a starvation penalty, so that the trade-off is not hidden.
  4. [Table 1 and §3.2] PointPainting [41] is a lidar-camera fusion method for 3D object detection, not a segmentation network for background blurring in video conferencing. Its profiled latency is unlikely to represent a real-time segmentation model, which undermines the realism of scenarios C and D and the interpretation of the Seg deadline-violation numbers. The authors should replace it with an actual segmentation model, such as DeepLabV3, BiSeNet, or MediaPipe Selfie Segmentation, or justify why PointPainting is representative of the intended background-blurring task.
minor comments (4)
  1. [§1 and §4.1] There are several typos, including 'artifical' in Section 1 and 'deploymnent' in Section 4.1; the manuscript should be proofread.
  2. [Table 5 and Figure 6] The paper does not specify the number of simulation repetitions, the simulated wall-clock duration, or the number of generated tokens used to compute the TTFT/TPT and violation-rate values; please add these experimental details to the methodology.
  3. [Related Work] The description of DREAM [15] as 'state-of-the-art' is reasonable, but since DREAM shares an author with the current paper, the relationship should be stated explicitly in the text.
  4. [§5.2.1] The claim that FTF 'achieves a TTFT that matches the standalone performance' is partly by construction, because FTF is designed to prioritize the first token; the paper should acknowledge this more directly, even though it does report the resulting TPT penalty.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: scheduling-impact results are simulator outputs from profiled latencies; self-citations are not load-bearing.

full rationale

The paper's derivation chain is empirical and self-contained: Section 4.1 profiles real models on the AMD Ryzen AI SoC, Section 4.2 stores the measured latencies in a database and uses a Python simulator to apply five scheduling policies (Table 4) to the request streams of four scenarios (Table 2). The headline 41.7% deadline-violation spread, the TTFT/TPT comparisons (Table 5), the sequence-length trends (Figure 6), and the decode-layer dispatch percentages (Figure 7) are all deterministic outputs of that simulator given the profiled isolation latencies and the policy definitions. No parameter is fitted to a target metric and then re-reported as a prediction. FTF's low TTFT is partly by design, because FTF is defined to prioritize first-token generation (Section 4.3), but the paper also reports the non-optimized consequences of that choice, including higher TPT and, in Scenario C, a higher deadline-violation rate than FCFS-DYN, so the evaluation is not a tautology. The only self-citations (DREAM and prior heterogeneous-accelerator work, [15,16,17]) appear as related-work context and are not used to justify any central claim. The principal weakness is external validity rather than circularity: the simulator assumes each scheduled layer completes at its isolated profiled latency, with no cross-backend interference, no dispatch/preemption overhead, and no end-to-end validation against the real SoC (Section 4.2, Section 5.2.1). If real concurrent NPU/GPU execution perturbs those latencies, the quantitative claims would change; this is a correctness and validation risk, not a circularity flaw.

Assumptions & free parameters 0 free parameters · 6 assumptions · 1 invented entities

The central claims rest on a simulator whose fidelity is unverified; key modeling choices (isolated latencies, zero preemption overhead, periodic arrivals, single-platform measurements) are load-bearing. There are no fitted free parameters; the latency database is measured data. The RTGen label is an invented taxonomy with no independent evidence.

assumptions (6)
  • domain assumption The simulator models each scheduled layer or model at its profiled mean latency, assuming no performance interference between backends or between concurrent layers.
    Section 4.2 describes a latency database used for cost estimation; no contention or interference model is mentioned, and the simulator is not validated against execution on Ryzen AI.
  • ad hoc to paper Preemption of a running layer is possible at any layer boundary at zero cost.
    EDF policies are described as preempting the LLM's prefill to schedule real-time layers (Section 5.2.1); no preemption primitives or overheads are modeled.
  • domain assumption Inference latencies are deterministic; no variance, tail latency, or thermal effects are modeled.
    Latency values in Table 5 and Figure 6 are reported without error bars; real-time scheduling depends on tail behavior.
  • domain assumption Requests arrive at fixed periodic inter-arrival times matching the target FPS.
    Section 4.2 says the inference generator issues requests based on each model's FPS; bursty or jittered arrivals are not modeled.
  • ad hoc to paper PointPainting is a valid segmentation model for background blurring and its measured latency represents a real-time segmentation network.
    Table 1 lists PointPainting [41] for background blurring, but PointPainting is a 3D object detection fusion method, not a standard segmentation network for this task.
  • domain assumption The measured latencies on AMD Ryzen AI 9 365 are representative of emerging heterogeneous SoCs.
    All conclusions are drawn from a single hardware platform (Table 3).
invented entities (1)
  • RTGen (real-time generative AI) workload class
    purpose: An umbrella category for multi-model workloads that combine LLM generation with real-time constraints, used to scope the study and define scenarios.
    This is a proposed taxonomy label, not a physical entity; it has no falsifiable handle outside the paper's own definitions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Dynamic Scheduling Space of Real-Time Generative AI Applications on Emerging Heterogeneous Systems." pith.science (2026). https://pith.science/paper/TBLPLPCN

@misc{pith2026250714715,
  author       = {Pith},
  title        = {Pith review of: Exploring the Dynamic Scheduling Space of Real-Time Generative AI Applications on Emerging Heterogeneous Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TBLPLPCN}},
  note         = {Machine review of arXiv:2507.14715}
}
read the original abstract

The integration of generative AI models, particularly large language models (LLMs), into real-time multi-model AI applications such as video conferencing and gaming is giving rise to a new class of workloads: real-time generative AI (RTGen). These workloads combine the compute intensity and dynamic execution patterns of generative models with the stringent latency and concurrency constraints of real-time inference. To meet the diverse demands of RTGen workloads, modern edge platforms increasingly adopt heterogeneous system-on-chip (SoC) architectures that integrate CPUs, GPUs, and NPUs. Despite the potential of heterogeneous SoC, the scheduling space complexity and performance implications of RTGen workloads on such platforms remain underexplored. In this work, we perform a comprehensive characterization of RTGen workloads on AMD's latest heterogeneous SoC, Ryzen AI. We construct realistic multi-model scenarios inspired by industry use cases and profile model performance across all available backends. Using this data, we evaluate five scheduling policies and their impact on both real-time metrics (e.g., deadline violation rate) and LLM performance (e.g., time-to-first-token and tokens-per-second). Our results show that scheduling decisions significantly affect workload performance (e.g., leading to a 41.7% difference in deadline violation rates on average), and highlight the need for scheduling strategies that are aware of workload dynamics and hardware heterogeneity. Our findings underscore the importance of workload-aware, dynamic heterogeneous scheduling in enabling high-performance, on-device RTGen applications.

Figures

Figures reproduced from arXiv: 2507.14715 by the authors.

Figure 1
Figure 1. Challenges in scheduling emerging real-time - generative workloads on heterogeneous systems. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of LLM architecture and generative inference. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. The impact of the hardware platform on the [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Comparing the latency of the (a) encoder, (b) [PITH_FULL_IMAGE:figures/full_fig_p003_5.png]
Figure 6
Figure 6. Figure 6: The impact of the input sequence length on the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The percentage of LLM layers processed on the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 35 canonical work pages

  1. [41]

    Pointpainting: Sequential fusion for 3d object detection

    Sourabh Vora, Alex H Lang, Bassam Helou, and Oscar Beijbom. Pointpainting: Sequential fusion for 3d object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4604–4612, 2020

    work page 2020

  2. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  3. [2]

    AMD FidelityFX Super Reso- lution

    Advanced Micro Devices, Inc. AMD FidelityFX Super Reso- lution. https://www.amd.com/en/products/graphics/technologies/ fidelityfx/super-resolution.html

    work page

  4. [3]

    AMD Radeon texttrademark RX ,7900 ,XTX graphics card

    Advanced Micro Devices, Inc. AMD Radeon texttrademark RX ,7900 ,XTX graphics card. https://www.amd.com/en/products/graphics/ desktops/radeon/7000-series/amd-radeon-rx-7900xtx.html

    work page

  5. [4]

    AMD Ryzen ™ AI – Windows PCs with AI Built In

    Advanced Micro Devices, Inc. AMD Ryzen ™ AI – Windows PCs with AI Built In. https://www.amd.com/en/products/processors/consumer/ ryzen-ai.html

    work page

  6. [5]

    RyzenAI-SW: AMD Ryzen ™ AI Software

    AMD. RyzenAI-SW: AMD Ryzen ™ AI Software. https://github.com/ amd/RyzenAI-SW

    work page

  7. [6]

    Apple reveals M3 Ultra,taking Apple silicon to a new extreme

    Apple Inc. Apple reveals M3 Ultra,taking Apple silicon to a new extreme. https://www.apple.com/newsroom/2025/03/ apple-reveals-m3-ultra-taking-apple-silicon-to-a-new-extreme/, Oct 2023

    work page 2025

  8. [7]

    Self-rag: Learning to retrieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations , 2023

    work page 2023

Show all 46 references
  1. [8]

    Map-and-conquer: Energy- efficient mapping of dynamic neural nets onto heterogeneous mpsocs

    Halima Bouzidi, Mohanad Odema, Hamza Ouarnoughi, Smail Niar, and Mohammad Abdullah Al Faruque. Map-and-conquer: Energy- efficient mapping of dynamic neural nets onto heterogeneous mpsocs. In 2023 60th ACM/IEEE Design Automation Conference (DAC) , pages 1–6. IEEE, 2023

    2023

  2. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, G Heigold, S Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference ...

    2020

  3. [10]

    Large language models and games: A survey and roadmap

    Roberto Gallotta, Graham Todd, Marvin Zammit, Sam Earle, Antonios Liapis, Julian Togelius, and Georgios N Yannakakis. Large language models and games: A survey and roadmap. IEEE Transactions on Games, 2024

    2024

  4. [11]

    Planaria: Dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks

    Soroush Ghodrati, Byung Hoon Ahn, Joon Kyung Kim, Sean Kinzer, Brahmendra Reddy Yatham, Navateja Alla, Hardik Sharma, Mo- hammad Alian, Eiman Ebrahimi, Nam Sung Kim, et al. Planaria: Dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks. In...

    2020

  5. [12]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024

    2024 arXiv

  6. [13]

    Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing

    Guseul Heo, Sangyeop Lee, Jaehong Cho, Hyunmin Choi, Sanghyeon Lee, Hyungkyu Ham, Gwangsun Kim, Divya Mahajan, and Jongse Park. Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing. In Proceedings of the 29th ACM International Conference on Architectural Sup...

    2024

  7. [14]

    Fiddler: Cpu-gpu orchestration for fast inference of mixture-of- experts models

    Keisuke Kamahori, Tian Tang, Yile Gu, Kan Zhu, and Baris Kasikci. Fiddler: Cpu-gpu orchestration for fast inference of mixture-of- experts models. arXiv preprint arXiv:2402.07033 , 2024

    2024 arXiv

  8. [15]

    Dream: A dynamic scheduler for dynamic real-time multi-model ml workloads

    Seah Kim, Hyoukjun Kwon, Jinook Song, Jihyuck Jo, Yu-Hsin Chen, Liangzhen Lai, and Vikas Chandra. Dream: A dynamic scheduler for dynamic real-time multi-model ml workloads. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languag...

    2023

  9. [16]

    Heterogeneous dataflow accelerators for multi-dnn workloads

    Hyoukjun Kwon, Liangzhen Lai, Michael Pellauer, Tushar Krishna, Yu- Hsin Chen, and Vikas Chandra. Heterogeneous dataflow accelerators for multi-dnn workloads. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , pages 71–83. IEEE, 2021

    2021

  10. [17]

    Xrbench: An extended reality (xr) machine learning benchmark suite for the metaverse

    Hyoukjun Kwon, Krishnakumar Nair, Jamin Seo, Jason Yik, Debabrata Mohapatra, Dongyuan Zhan, Jinook Song, Peter Capak, Peizhao Zhang, Peter Vajda, et al. Xrbench: An extended reality (xr) machine learning benchmark suite for the metaverse. Proceedings of Machine Learning and Sy...

    2023

  11. [18]

    {InfiniGen}: Efficient generative inference of large language models with dynamic {KV} cache management

    Wonbeom Lee, Jungi Lee, Junghwan Seo, and Jaewoong Sim. {InfiniGen}: Efficient generative inference of large language models with dynamic {KV} cache management. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24) , pages 155–172, 2024

    2024

  12. [19]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems...

    2020

  13. [20]

    Towards general text embeddings with multi- stage contrastive learning

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi- stage contrastive learning. arXiv preprint arXiv:2308.03281 , 2023

    2023 arXiv

  14. [21]

    Efficient heterogeneous video segmentation at the edge

    Jamie Menjay Lin, Siargey Pisarchyk, Juhyun Lee, David Tian, Tingbo Hou, Karthik Raveendran, Raman Sarokin, George Sung, Trent Tolley, and Matthias Grundmann. Efficient heterogeneous video segmentation at the edge. arXiv preprint arXiv:2208.11666 , 2022

    2022 arXiv

  15. [22]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems , 6:87–100, 2024

    2024

  16. [23]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012–10022, 2021

    2021

  17. [24]

    Mediapipe: A framework for building perception pipelines

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClana- han, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, et al. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172 , 2019

    1906 arXiv

  18. [25]

    AI Tools for Organizations | Microsoft Copilot

    Microsoft. AI Tools for Organizations | Microsoft Copilot. https: //www.microsoft.com/en-us/microsoft-copilot/organizations

  19. [26]

    Virtual Meeting Backgrounds and Background Blur | Teams

    Microsoft. Virtual Meeting Backgrounds and Background Blur | Teams. https://www.microsoft.com/en-us/microsoft-teams/ virtual-meeting-backgrounds

  20. [27]

    microsoft/onnxruntime -genai: Generative ai extensions for onnx runtime

    Microsoft. microsoft/onnxruntime -genai: Generative ai extensions for onnx runtime. https://github.com/microsoft/onnxruntime-genai, 2021

    2021

  21. [28]

    Enhancing Teams Video Quality with Super Resolu- tion

    Naba Kumar. Enhancing Teams Video Quality with Super Resolu- tion. https://techcommunity.microsoft.com/blog/microsoftteamsblog/ enhancing-teams-video-quality-with-super-resolution/4373307, Feb 2025

    2025

  22. [29]

    Vitis AI Execution Provider

    ONNX Runtime. Vitis AI Execution Provider. https://onnxruntime. ai/docs/execution-providers/Vitis-AI-ExecutionProvider.html

  23. [30]

    Windows - DirectML Execution Provider

    ONNX Runtime. Windows - DirectML Execution Provider. https://onnxruntime.ai/docs/execution-providers/ DirectML-ExecutionProvider.html

  24. [31]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology , pages 1–22, 2023

    2023

  25. [32]

    Laptops, Desktops & Tablets | Snapdragon

    Qualcomm Inc. Laptops, Desktops & Tablets | Snapdragon. https: //www.qualcomm.com/products/mobile/snapdragon/pcs

  26. [33]

    Best AI PCs: Exploring the Next Generation of Computing

    Qualcomm Technologies, Inc. Best AI PCs: Exploring the Next Generation of Computing. https://www.qualcomm.com/snapdragon/ laptops-and-tablets/best-ai-pcs

  27. [34]

    Softmap: Software-hardware co-design for integer-only softmax on associative processors

    Mariam Rakka, Jinhao Li, Guohao Dai, Ahmed Eltawil, Mohammed E Fouda, and Fadi Kurdahi. Softmap: Software-hardware co-design for integer-only softmax on associative processors. In 2025 Design, Automation & Test in Europe Conference (DATE) , pages 1–7. IEEE, 2025

    2025

  28. [35]

    Yolov3: An incremental improvement

    Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 , 2018

    2018 arXiv

  29. [36]

    Onnx runtime

    ONNX Runtime developers. Onnx runtime. https://onnxruntime.ai/, 2021

    2021

  30. [37]

    Flexgen: High-throughput generative inference of large language models with a single gpu

    Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher Ré, Ion Stoica, and Ce Zhang. Flexgen: High-throughput generative inference of large language models with a single gpu. In International Conference on Machine Learning , pag...

    2023

  31. [38]

    Introducing Streamlabs’ New Intelligent Streaming Assistant in Collaboration with NVIDIA and Inworld AI

    Streamlabs. Introducing Streamlabs’ New Intelligent Streaming Assistant in Collaboration with NVIDIA and Inworld AI. https://streamlabs.com/content-hub/post/ introducing-streamlabs-new-intelligent-streaming-assistant-in-collaboration-with-nvidia-and-inworld-ai, Jan 2025

    2025

  32. [39]

    Diffusion models are real-time game engines

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint arXiv:2408.14837, 2024

    2024 arXiv

  33. [40]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017

    2017

  34. [42]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 , 2022

    2022 arXiv

  35. [43]

    Image super-resolution using very deep residual channel attention networks

    Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European conference on computer vision (ECCV) , pages 286–301, 2018

    2018

  36. [44]

    Using blurred background – Zoom

    Zoom Video Communications, Inc. Using blurred background – Zoom. https://support.zoom.com/hc/en/article?id=zm_kb&sysparm_ article=KB0061066

  37. [45]

    Using gesture recognition – Zoom

    Zoom Video Communications, Inc. Using gesture recognition – Zoom. https://support.zoom.com/hc/en/article?id=zm_kb&sysparm_ article=KB0067842

  38. [46]

    Zoom AI Assistant

    Zoom Video Communications, Inc. Zoom AI Assistant. https://www. zoom.com/en/products/ai-assistant/

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.