Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A serving system that routes easy text-to-image queries to lightweight models reports up to 24% better response quality and 19-70% fewer latency violations.

desk verdict Solid systems paper with a genuine empirical win, but the optimizer's objective (max t) is not the FID the paper reports, so the causal story needs work. read the letter →

arxiv 2411.15381 v2 pith:LIDRZ6FR submitted 2024-11-22 cs.DC

Sohaib Ahmad , Qizheng Yang , Haoliang Wang , Ramesh K. Sitaraman , Hui Guan This is my paper · ORCID
classification cs.DC
keywords diffusionmodelservingcascadequery-awareroutingtext-to-imagegenerationresourceallocationlatencySLOmixedintegerlinearprogrammingdiscriminator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a text-to-image serving system can get both faster responses and better output quality by treating queries as easy or hard. Its design sends every prompt first to a cheap, fast diffusion model; a learned judge decides whether that image is good enough, and only the images judged not good enough go to a slow, high-quality model. A controller then tunes the judge's threshold, batch sizes, and the number of machines assigned to each model so that latency deadlines are met while quality is as high as demand allows. The paper reports that this beats serving systems that either fix one model or switch models based only on load, improving response quality by up to 24% and cutting missed latency deadlines by 19-70%. If right, this changes the usual accuracy-versus-cost tradeoff for generative image serving, because it makes prompt difficulty a first-class resource-allocation signal.

What carries the argument

The central mechanism is the diffusion model cascade with two components. First, the discriminator: a binary classifier trained offline to distinguish real images from images generated by both the light and heavy models; at serving time its softmax confidence measures how "real" the light model's output looks, and exceeding a threshold releases the image to the client. Second, the resource allocator: a mixed-integer linear program that maximizes the confidence threshold subject to per-query latency constraints (execution time plus a Little's-law queueing estimate) and throughput constraints (light-model workers must cover all demand, heavy-model workers must cover the deferred fraction $f(t)$), with $f(t)$ profiled as a function of the threshold. The MILP is re-solved periodically, so the threshold drops as demand rises and rises when the cluster is quiet.

What would settle it

Take a held-out prompt set the discriminator never saw, have it score each light-model image, and compare those scores with an independent per-image quality judgment such as human preference or PickScore; if the scores are uncorrelated, or if the images the cascade keeps are no better than the ones it defers, then the routing signal—not the MILP—is what would fail, and the reported quality gains over random routing should disappear.

Watch

Extended reading notes

Core claim

The central claim is that query-aware model scaling—building a cascade from available diffusion-model variants and routing each prompt by its difficulty—lets a serving system escape the standard quality-versus-throughput tradeoff. Concretely, the paper claims that for 20-40% of prompts a lightweight one-step model produces images equal to or better than a heavyweight model, and that a binary classifier trained to separate real photographs from generated images can identify those prompts through its softmax confidence at negligible runtime cost. On top of that, it claims that co-optimizing the confidence threshold, batch sizes, and model placement in a mixed-integer linear program, re-solved as demand changes, keeps latency under service-level objectives while maximizing quality; the reported consequence is up to 24% better response quality than load-only scaling and 19-70% fewer SLO violations than static allocation, with the cascade sometimes beating an all-heavy deployment on Frechet Inception Distance (FID).

Load-bearing premise

The load-bearing premise is that the discriminator's softmax confidence that a light-model image looks real is a trustworthy per-prompt signal of whether that image meets the quality bar, so easy queries can safely skip the heavy model.

Editorial extensions

If this is right

  • Under high demand, the controller can lower the confidence threshold, routing more queries to the light model and trading a little quality to keep latency within SLO; under low demand it raises the threshold and sends more work to the heavy model.
  • Operators can stop provisioning for peak demand alone, because the cascade absorbs load spikes by shifting work to lighter models instead of dropping or delaying queries.
  • The confidence threshold becomes a single tunable dial that converts an SLO requirement into an operating point on the quality-throughput frontier.
  • The same design extends to longer pipelines by placing a discriminator after each stage and adding each stage's threshold to the optimization, and the discriminator's overhead stays negligible relative to diffusion execution.
  • Since 20-40% of prompts are easy, a fixed heavy-only deployment wastes a large share of its compute on queries that a light model can serve as well or better.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the MILP objective is simply "maximize the threshold," the same cascade machinery can be re-targeted at cost, energy, or fairness objectives by swapping the objective function without changing the routing or latency constraints.
  • Editorial inference: the reported result that mixing light-model outputs can improve FID over all-heavy output suggests response diversity is part of the quality story; a testable extension is whether that effect persists across more model pairs or holds under per-image quality metrics.
  • Editorial inference: a router that predicts difficulty from the text prompt alone would avoid running the light model as a probe; since the paper leaves this open, a natural next experiment is training a regressor from text embeddings to the discriminator's confidence.
  • Editorial inference: the confidence threshold could double as an admission-control knob, so under extreme overload the system might drop only the hardest queries rather than shedding load uniformly, preserving SLOs for the rest.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents DiffServe, a serving system for text-to-image diffusion models that combines a cascaded light-heavy model architecture with a learned discriminator that routes 'easy' queries to a lightweight model and defers the rest to a heavier, higher-quality model. A resource manager periodically solves a mixed-integer optimization to choose the confidence threshold, batch sizes, and model placement in response to fluctuating demand. The system is evaluated on three diffusion model pairs, using both a simulator and a 16-GPU testbed, with synthetic and real-world traces. The reported results claim up to 24% improvement in response quality (lower FID) and 19-70% lower SLO violation rates compared to state-of-the-art serving systems.

Significance. If the claims hold, the paper makes a useful contribution to ML serving systems by showing that query-aware routing and dynamic resource allocation can jointly improve quality and latency for expensive generative models. The work is backed by an artifact with public code (GitHub and Zenodo DOI), a real testbed evaluation, simulator validation, and ablations of both the discriminator design and the resource allocator. The observation that existing metrics like PickScore and CLIP Score underperform a random router for this cascading task is an interesting and non-obvious result. However, the central optimization claim—that maximizing the confidence threshold maximizes response quality—is in tension with the paper's own evidence of non-monotonic FID behavior, and the MILP formulation is not fully specified. These issues need to be resolved before the attributed quality improvements can be accepted as caused by the proposed optimizer.

major comments (3)
  1. [§3.3, Eqs. (1)-(5)] The resource allocation problem is formulated as maximizing the confidence threshold t, with the claim that response quality is directly influenced by t. However, Figure 1a and the accompanying discussion in §2.2 explicitly state that 'FID gets worse as the latency increases in the end,' i.e., that response quality is non-monotonic in the deferral rate, which increases with t. Because f(t) in Eq. (3) is increasing in t, the feasible max-t solution may lie on the wrong side of the FID-quality curve, and the paper provides no experiment isolating the effect of t on FID while holding other allocation decisions fixed. As a result, the up-to-24% FID improvements in Figures 5 and 6 are not shown to be a consequence of the stated optimization objective. The authors should either replace the objective with one that directly models FID as a function of t (or the deferral rate) or provide evidence that, under the evaluated conditions, the feasible max-t solution indeed lies at or below the FID-minimizing deferral rate.
  2. [§3.3, Eqs. (1)-(5)] The optimization is described as a mixed integer linear program (MILP), but the formulation as given is not linear. The objective in Eq. (5) is linear in t, but constraint (3) contains f(t), a fitted and possibly non-linear function of the continuous variable t, and constraint (1) contains execution latency e(b) and queuing delay q(b), which are non-linear functions of integer batch sizes b1 and b2. The text does not describe the linearization, piecewise approximation, or any reformulation used to make the problem solvable by Gurobi. Without this detail, the claim that the problem is a MILP is not verifiable, and the reported ~10 ms solver time cannot be reproduced or trusted. Please provide the exact reformulation or specify the approximation method for f, e, and q.
  3. [§3.2, Figure 3] The discriminator's softmax confidence is used as a per-query signal for whether the lightweight model's output meets the quality bar, but the evaluation metric is the distribution-level FID. The paper provides empirical evidence that the confidence score separates some queries (Figure 1b), but no calibration analysis (e.g., reliability diagrams or expected calibration error) and no held-out generalization study across datasets or prompt distributions are presented. The end-to-end quality gains could plausibly arise from distributional mixing of light and heavy model outputs rather than from accurate per-query deferral—a hypothesis the authors themselves raise in §2.2. To support the causal claim that confidence-based routing improves FID, the authors should provide direct per-query validation showing that high-confidence images from the light model are actually of comparable quality to heavy-model outputs, and low-confidence images are not, using a per-prompt quality metric on held-out data.
minor comments (6)
  1. [§3.1] Typo: 'test-to-image' should be 'text-to-image'.
  2. [§4.1] In the Evaluation Metrics paragraph, 'We access system performance' should be 'We assess system performance'.
  3. [Figure 4] The legend uses 'DiffServe' while the rest of the paper uses 'DIFFSERVE'; please keep the capitalization consistent.
  4. [Abstract and §4.3] The abstract reports 'up to 24% improvement' while §4.3 reports 'up to 23.4%' for the same trace; please reconcile the numbers or clarify that the abstract rounds the value.
  5. [Figures 5 and 8] The y-axis labeled 'FID' over time is not defined: it is unclear whether FID is computed on a rolling window of prompts, on the full 5K-prompt set for each time step, or on aggregated outputs per interval. Please specify the computation procedure.
  6. [§4.4] The discriminator training details (number of epochs, learning rate, train/validation split, regularization) are not given, which makes the discriminator ablations difficult to reproduce; please add these to the artifact appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DiffServe's quality gains are empirical comparisons against external baselines, not derived predictions.

full rationale

DiffServe does not derive its headline FID improvements from a first-principles model; it measures them in a simulator and on a testbed against external baselines (Clipper-Light, Clipper-Heavy, Proteus, DiffServe-Static). The discriminator is trained on a binary real-vs-generated classification task, and its softmax confidence is used as a routing signal. That confidence is not defined in terms of FID, and the reported FID scores are computed externally from generated images versus ground-truth distributions. The routing assumption is therefore an empirical design choice validated by comparison with random routing and metric-based discriminators, not a tautology. The MILP maximizes the confidence threshold t using a profiled deferral function f(t), but f(t) is a system input used to enforce throughput constraints, not the claimed response-quality prediction; the quality improvements are observed after running the system, not derived from the optimization. Even the non-monotonicity of FID in latency noted in Section 2.2 would be a correctness or modeling-fidelity concern about whether maximizing t optimizes FID, not a circularity. The paper cites Proteus, prior work by overlapping authors, as a baseline and as background, but no load-bearing claim depends on an unverified self-cited theorem or on a fitted parameter being renamed as a prediction. Profiling and evaluation on the same datasets and hardware is standard practice for serving-system papers and does not make the empirical comparison circular.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The system is an engineering contribution whose 'derivation' is a resource allocation model; it depends on fitted per-system performance profiles (execution latency, throughput, deferral fraction) rather than on new physical or mathematical entities. The main load-bearing assumptions are empirical: easy queries exist, discriminator confidence is a good quality proxy, and the MILP's linearization of f(t) is valid.

free parameters (2)
  • Deferral fraction f(t) = profiled table/function, updated online
    Used in Eq. 3 as D*f(t); initialized offline and updated during serving, so the relation between threshold and deferral rate is an empirical fit.
  • Over-provisioning factor lambda = 1.05
    Chosen by default in Section 3.3 to inflate demand estimate in Eq. 4; no sensitivity analysis is reported.
assumptions (6)
  • domain assumption Easy queries exist: for 20-40% of prompts the lightweight model produces images comparable to or better than the heavyweight model.
    Motivated by Figure 1b using PickScore and discriminator confidence; if this distribution is not representative, cascade gains vanish.
  • domain assumption The discriminator's softmax confidence is a valid per-query proxy for whether light-model output meets the quality bar.
    Section 3.2: routing is based on this confidence; no calibration analysis is provided.
  • standard math Little's law gives accurate queueing delay estimates from current queue length and arrival rate.
    Section 3.3 uses W=L/lambda to estimate queuing delays; assumes steady-state conditions that may not hold during demand spikes.
  • domain assumption Execution latency and throughput can be profiled and treated as deterministic functions of batch size.
    Section 3.3 and Section 4.1: models are profiled offline; variance on real GPUs is acknowledged in Section 4.3 but not modeled in the MILP.
  • ad hoc to paper f(t) is monotonic and can be embedded in a linear optimization without loss.
    Eq. 3 includes f(t), a nonlinear fitted function of optimization variable t; the paper does not specify the linearization scheme used to justify the MILP formulation.
  • domain assumption Higher confidence threshold t monotonically improves response quality.
    The MILP objective maximizes t (Eq. 5), yet Section 2.1 reports FID worsens when too many queries are deferred to the heavy model, so the objective and metric may be misaligned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling." pith.science (2026). https://pith.science/paper/LIDRZ6FR

@misc{pith2026241115381,
  author       = {Pith},
  title        = {Pith review of: DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LIDRZ6FR}},
  note         = {Machine review of arXiv:2411.15381}
}
read the original abstract

Text-to-image generation using diffusion models has gained increasing popularity due to their ability to produce high-quality, realistic images based on text prompts. However, efficiently serving these models is challenging due to their computation-intensive nature and the variation in query demands. In this paper, we aim to address both problems simultaneously through query-aware model scaling. The core idea is to construct model cascades so that easy queries can be processed by more lightweight diffusion models without compromising image generation quality. Based on this concept, we develop an end-to-end text-to-image diffusion model serving system, DiffServe, which automatically constructs model cascades from available diffusion model variants and allocates resources dynamically in response to demand fluctuations. Our empirical evaluations demonstrate that DiffServe achieves up to 24% improvement in response quality while maintaining 19-70% lower latency violation rates compared to state-of-the-art model serving systems.

Figures

Figures reproduced from arXiv: 2411.15381 by the authors.

Figure 1
Figure 1. (a) The quality-latency trade-offs of systems serving independent diffusion models and diffusion model cascades with different discriminator designs with batch size one. The top panel uses diffusion model cascades built with SDv1.5 as the heavyweight model (H) and SD-Turbo as the lightweight model (L). The bottom panel uses SDXS as the lightweight model instead. Lower latency is achieved by using lighter models or t… view at source ↗
Figure 2
Figure 2. shows the system architecture. It has separate data and control paths. In the data path, clients send queries to the system’s Load Balancer, which routes them to suitable workers and returns generated images. In the control path, the Controller periodically re-allocates resources depending on the system runtime statistics collected from the workers. Controller. The Controller manages the resources in the system. It … view at source ↗
Figure 3
Figure 3. Training and inference paths of the discriminator. ‘Real’ refers to images sourced from real-world high-quality datasets. ‘Fake’ refers to the generated images from diffusion models. GLM: Generated images from Lightweight diffusion Model; GHM: Generated images from Heavyweight diffusion Model. sion model to determine whether deferral to a heavyweight model is necessary. The discriminator must be accurate in quality … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Performance comparison on static trace. DIFFSERVE offers Pareto optimal between FID and SLO violations (lower left curve). 0 5 10 15 20 25 30 35 40 Demand (QPS) 16 17 18 19 20 21 22 23 FID 0 50 100 150 200 250 300 350 Time (sec) 0.0 0.2 0.4 0.6 0.8 1.0 SLO Violation Ra…
Figure 6
Figure 6. Figure 6: Comparison of approaches on the testbed. Across both Cascade 2 and 3, DIFFSERVE achieves an 6%-24% reduction in av￾erage FID compared to all baselines, except Clipper-Heavy which yields significantly high SLO violations. DIFFSERVE reduces the average SLO violation by u…
Figure 5
Figure 5. Figure 5: Performance comparison on real-world trace for Cascade 1. DIFFSERVE improves quality by up to 23.4% over baselines while maintaining low SLO violations. During peak, it offers similar or better quality than static approaches with significantly lower SLO violations (fro…
Figure 7
Figure 7. Figure 7: Discriminator comparison. EfficientNet trained with ground truth images achieves the lowest FID given latency require￾ments, outperforming all other approaches across both cascades. mentioned before, Clipper-Heavy suffers from significantly high SLO violations at this …
Figure 8
Figure 8. Figure 8: Performance of the resource allocation algorithm in DIFF￾SERVE and alternative approaches. DIFFSERVE reduces SLO vio￾lations by 20% and improves quality by up to 19%. w GT: using Vision Transformer (Dosovitskiy et al., 2021) (i.e., ViT b 16) trained with ground truth i…
Figure 9
Figure 9. Figure 9: Effect of SLO on performance for Cascade 1. DIFF￾SERVE guarantees low SLO violations and high quality over a broad range of SLO values. the discriminator in DIFFSERVE, leading to improved ac￾curacy in confidence estimation and overall performance within the cascading a…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Xema: Efficient Diffusion Serving through Fine-Grained Memory Management and Auto-Configuration

    cs.DC 2026-07 conditional novelty 6.0 of 10

    Trace-guided fine-grained memory control and offline joint planning raise diffusion serving SLO attainment by up to 3.7× while cutting configuration search from hours to minutes.

Reference graph

Works this paper leans on

52 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://archive.org/details/archiveteam-twitter-stream-2018-04, 2018

    Twitter streaming traces. https://archive.org/details/archiveteam-twitter-stream-2018-04, 2018

    work page 2018

  2. [2]

    build, train, and deploy machine learning models at scale

    Amazon sagemaker. build, train, and deploy machine learning models at scale. https://aws.amazon.com/sagemaker/, 2020. Accessed: 2021-06-23

    work page 2020

  3. [3]

    https://developer.nvidia.com/nvidia-triton-inference-server, 2022

    Triton inference server. https://developer.nvidia.com/nvidia-triton-inference-server, 2022

    work page 2022

  4. [4]

    Adobe firefly

    Adobe. Adobe firefly. https://www.adobe.com/products/firefly.html, 2024. Accessed: 2024-10-07

    work page 2024

  5. [5]

    D., Williams, T., Sitaraman, R

    Ahmad, S., Guan, H., Friedman, B. D., Williams, T., Sitaraman, R. K., and Woo, T. Proteus: A high-throughput inference-serving system with accuracy scaling. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, pp.\ 318--334, 2024

    work page 2024

  6. [6]

    Batch: Machine learning inference serving on serverless platforms with adaptive batching

    Ali, A., Pinciroli, R., Yan, F., and Smirni, E. Batch: Machine learning inference serving on serverless platforms with adaptive batching. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp.\ 1--15. IEEE, 2020

    work page 2020

  7. [7]

    Adaptive neural networks for efficient inference

    Bolukbasi, T., Wang, J., Dekel, O., and Saligrama, V. Adaptive neural networks for efficient inference. In International Conference on Machine Learning, pp.\ 527--536. PMLR, 2017

    work page 2017

  8. [8]

    Frugalgpt: How to use large language models while reducing cost and improving performance, 2023

    Chen, L., Zaharia, M., and Zou, J. Frugalgpt: How to use large language models while reducing cost and improving performance, 2023. URL https://arxiv.org/abs/2305.05176

    arXiv 2023

Show all 52 references
  1. [9]

    J., Gonzalez, J

    Crankshaw, D., Wang, X., Zhou, G., Franklin, M. J., Gonzalez, J. E., and Stoica, I. Clipper: A low-latency online prediction serving system. In 14th USENIX Symposium on Networked Systems Design and Implementation ( NSDI 17) , pp.\ 613--627, Boston, MA, March 2017 a . USENIX As...

    2017

  2. [10]

    J., Gonzalez, J

    Crankshaw, D., Wang, X., Zhou, G., Franklin, M. J., Gonzalez, J. E., and Stoica, I. Clipper: A low-latency online prediction serving system. In NSDI, volume 17, pp.\ 613--627, 2017 b

    2017

  3. [11]

    Inferline: latency-aware provisioning and scaling for prediction serving pipelines

    Crankshaw, D., Sela, G.-E., Mo, X., Zumar, C., Stoica, I., Gonzalez, J., and Tumanov, A. Inferline: latency-aware provisioning and scaling for prediction serving pipelines. In Proceedings of the 11th ACM Symposium on Cloud Computing, pp.\ 477--491, 2020

    2020

  4. [12]

    Ding, D., Mallick, A., Wang, C., Sim, R., Mukherjee, S., Ruhle, V., Lakshmanan, L. V. S., and Awadallah, A. H. Hybrid llm: Cost-efficient and quality-aware query routing, 2024. URL https://arxiv.org/abs/2404.14618

    2024 arXiv

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. URL https://arxiv.org/ab...

    2021 arXiv

  6. [14]

    Serving \ DNNs \ like clockwork: Performance predictability from the bottom up

    Gujarati, A., Karimi, R., Alzayat, S., Hao, W., Kaufmann, A., Vigfusson, Y., and Mace, J. Serving \ DNNs \ like clockwork: Performance predictability from the bottom up. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pp.\ 443--462, 2020

    2020

  7. [15]

    R., Mishra, C

    Gunasekaran, J. R., Mishra, C. S., Thinakaran, P., Sharma, B., Kandemir, M. T., and Das, C. R. Cocktail: A multidimensional optimization for model serving in cloud. In USENIX NSDI, pp.\ 1041--1057, 2022

    2022

  8. [16]

    Sommelier: Curating dnn models for the masses

    Guo, P., Hu, B., and Hu, W. Sommelier: Curating dnn models for the masses. In Proceedings of the 2022 International Conference on Management of Data, pp.\ 1876--1890, 2022

    2022

  9. [17]

    S., Menon, A

    Gupta, N., Narasimhan, H., Jitkrittum, W., Rawat, A. S., Menon, A. K., and Kumar, S. Language model cascades: Token-level uncertainty and beyond, 2024. URL https://arxiv.org/abs/2404.10136

    2024 arXiv

  10. [18]

    URL https://www.gurobi.com/

    Gurobi, 2024. URL https://www.gurobi.com/

    2024

  11. [19]

    Deep residual learning for image recognition, 2015

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition, 2015. URL https://arxiv.org/abs/1512.03385

    2015 arXiv

  12. [20]

    CLIPS core: A reference-free evaluation metric for image captioning

    Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., and Choi, Y. CLIPS core: A reference-free evaluation metric for image captioning. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pr...

    2021 doi

  13. [21]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 6629–6640, ...

    2017

  14. [22]

    Scrooge: A cost-effective deep learning inference system

    Hu, Y., Ghosh, R., and Govindan, R. Scrooge: A cost-effective deep learning inference system. In Proceedings of the ACM Symposium on Cloud Computing, pp.\ 624--638, 2021

    2021

  15. [23]

    Hugging face models hub, 2024

    Hugging Face . Hugging face models hub, 2024. URL https://huggingface.co/models. Accessed: 2024-10-30

    2024

  16. [24]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation, 2023

    Kirstain, Y., Polyak, A., Singer, U., Matiana, S., Penna, J., and Levy, O. Pick-a-pic: An open dataset of user preferences for text-to-image generation, 2023. URL https://arxiv.org/abs/2305.01569

    2023 arXiv

  17. [25]

    Willump: A statistically-aware end-to-end optimizer for machine learning inference, 2020

    Kraft, P., Kang, D., Narayanan, D., Palkar, S., Bailis, P., and Zaharia, M. Willump: A statistically-aware end-to-end optimizer for machine learning inference, 2020. URL https://arxiv.org/abs/1906.01974

    2020 arXiv

  18. [26]

    Cascadebert: Accelerating inference of pre-trained language models via calibrated complete models cascade, 2021

    Li, L., Lin, Y., Chen, D., Ren, S., Li, P., Zhou, J., and Sun, X. Cascadebert: Accelerating inference of pre-trained language models via calibrated complete models cascade, 2021. URL https://arxiv.org/abs/2012.14682

    2021 arXiv

  19. [27]

    Sdxl-lightning: Progressive adversarial diffusion distillation, 2024

    Lin, S., Wang, A., and Yang, X. Sdxl-lightning: Progressive adversarial diffusion distillation, 2024. URL https://arxiv.org/abs/2402.13929

    2024 arXiv

  20. [28]

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp....

    2014

  21. [29]

    Midjourney

    Midjourney. Midjourney. https://www.midjourney.com/home, 2024. Accessed: 2024-10-07

    2024

  22. [30]

    Tensorflow-serving: Flexible, high-performance ml serving

    Olston, C., Li, F., Harmsen, J., Soyke, J., Gorovoy, K., Lao, L., Fiedel, N., Ramesh, S., and Rajashekhar, V. Tensorflow-serving: Flexible, high-performance ml serving. In Workshop on ML Systems at NIPS 2017, 2017

    2017

  23. [31]

    Pytorch: An imperative style, high-performance deep learning library, 2019

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An impera...

    2019 arXiv

  24. [32]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., M \"u ller, J., Penna, J., and Rombach, R. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  25. [33]

    Torchserve

    PyTorch Authors . Torchserve. https://pytorch.org/serve/, 2023. Accessed: 2024-09-26

    2023

  26. [34]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 10684--10695, June 2022

    2022

  27. [35]

    J., and Kozyrakis, C

    Romero, F., Li, Q., Yadwadkar, N. J., and Kozyrakis, C. \ INFaaS \ : Automated model-less inference serving. In 2021 USENIX Annual Technical Conference (USENIX ATC 21), pp.\ 397--411, 2021 a

    2021

  28. [36]

    J., and Kozyrakis, C

    Romero, F., Zhao, M., Yadwadkar, N. J., and Kozyrakis, C. Llama: A heterogeneous & serverless framework for auto-tuning video analytics pipelines. In Proceedings of the ACM Symposium on Cloud Computing, SoCC '21, pp.\ 1–17, New York, NY, USA, 2021 b . Association for Computing...

    2021

  29. [37]

    Adversarial diffusion distillation, 2023

    Sauer, A., Lorenz, D., Blattmann, A., and Rombach, R. Adversarial diffusion distillation, 2023. URL https://arxiv.org/abs/2311.17042

    2023 arXiv

  30. [38]

    Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider

    Shahrad, M., Fonseca, R., Goiri, \'I ., Chaudhry, G., Batum, P., Cooke, J., Laureano, E., Tresness, C., Russinovich, M., and Bianchini, R. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. In 2020 USENIX Annual Technical C...

    2020

  31. [39]

    Fast video classification via adaptive cascading of deep models

    Shen, H., Han, S., Philipose, M., and Krishnamurthy, A. Fast video classification via adaptive cascading of deep models. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 2197--2205, 2017. doi:10.1109/CVPR.2017.236

    2017 doi

  32. [40]

    Nexus: A gpu cluster engine for accelerating dnn-based video analysis

    Shen, H., Chen, L., Jin, Y., Zhao, L., Kong, B., Philipose, M., Krishnamurthy, A., and Sundaram, R. Nexus: A gpu cluster engine for accelerating dnn-based video analysis. In Proceedings of the 27th ACM Symposium on Operating Systems Principles, pp.\ 322--337, 2019

    2019

  33. [41]

    F., Thompson, J

    Shortle, J. F., Thompson, J. M., Gross, D., and Harris, C. M. Fundamentals of queueing theory, volume 399. John Wiley & Sons, 2018

    2018

  34. [42]

    Sdxs: Real-time one-step latent diffusion models with image conditions, 2024

    Song, Y., Sun, Z., and Yin, X. Sdxs: Real-time one-step latent diffusion models with image conditions, 2024. URL https://arxiv.org/abs/2403.16627

    2024 arXiv

  35. [43]

    Tensorflow serving

    TensorFlow Authors . Tensorflow serving. https://www.tensorflow.org/tfx/guide/serving, 2023. Accessed: 2024-09-26

    2023

  36. [44]

    and Jones, M

    Viola, P. and Jones, M. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, volume 1, pp.\ I--I. Ieee, 2001

    2001

  37. [45]

    Diffusers: State-of-the-art diffusion models

    von Platen, P., Patil, S., Lozhkov, A., Cuenca, P., Lambert, N., Rasul, K., Davaadorj, M., Nair, D., Paul, S., Berman, W., Xu, Y., Liu, S., and Wolf, T. Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers, 2022

    2022

  38. [46]

    K., and Ooi, B

    Wang, W., Wang, S., Gao, J., Zhang, M., Chen, G., Ng, T. K., and Ooi, B. C. Rafiki: Machine learning as an analytics service system. arXiv preprint arXiv:1804.06087, 2018

    2018 arXiv

  39. [47]

    Tabi: An efficient multi-level inference system for large language models

    Wang, Y., Chen, K., Tan, H., and Guo, K. Tabi: An efficient multi-level inference system for large language models. In Proceedings of the Eighteenth European Conference on Computer Systems, EuroSys '23, pp.\ 233–248, New York, NY, USA, 2023. Association for Computing Machinery...

    2023

  40. [48]

    J., Montoya, E., Munechika, D., Yang, H., Hoover, B., and Chau, D

    Wang, Z. J., Montoya, E., Munechika, D., Yang, H., Hoover, B., and Chau, D. H. DiffusionDB : A large-scale prompt gallery dataset for text-to-image generative models. arXiv:2210.14896 [cs], 2022. URL https://arxiv.org/abs/2210.14896

    2022 arXiv

  41. [49]

    Bartscore: Evaluating generated text as text generation

    Yuan, W., Neubig, G., and Liu, P. Bartscore: Evaluating generated text as text generation. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp.\ 27263--27277. Curran Associates, In...

    2021

  42. [50]

    Zhang, H., Ananthanarayanan, G., Bodik, P., Philipose, M., Bahl, P., and Freedman, M. J. Live video analytics at scale with approximation and Delay-Tolerance . In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17), pp.\ 377--392, Boston, MA, March 2...

    2017

  43. [51]

    \ Model-Switching \ : Dealing with fluctuating workloads in \ Machine-Learning-as-a-Service \ systems

    Zhang, J., Elnikety, S., Zarar, S., Gupta, A., and Garg, S. \ Model-Switching \ : Dealing with fluctuating workloads in \ Machine-Learning-as-a-Service \ systems. In 12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 20), 2020

    2020

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.