pith. sign in

arxiv: 2605.16401 · v1 · pith:UNWFMARDnew · submitted 2026-05-12 · 💻 cs.CV · cs.LG

CADS: Conformal Adaptive Decision System for Cost-Efficient Image Classification

Turkoglu Mikael , Bary Tim , Thielens Vincent , Dausort Manon , Macq Beno\^it This is my paper

Pith reviewed 2026-05-20 21:47 UTC · model grok-4.3

classification 💻 cs.CV cs.LG
keywords conformal predictionadaptive inferencemodel cascadeimage classificationuncertainty quantificationcost-efficient inferenceresource optimization
0
0 comments X

The pith

CADS uses conformal prediction to route images through model cascades for lower cost and maintained accuracy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces CADS as a sequential multi-model algorithm that applies conformal prediction to estimate uncertainty in each image at runtime. Samples judged simple are sent to lightweight Scout models while uncertain ones escalate to high-capacity Oracle models. This setup targets the cost-accuracy trade-off by avoiding heavy computation on routine cases. A sympathetic reader would care because it tackles the waste of resources when powerful models process every input in domains such as clinical imaging.

Core claim

CADS provides a mathematically grounded framework for balancing the cost-accuracy dilemma that dynamically routes samples through a model cascade, ranging from lightweight Scout models to high-capacity Oracle architectures. It leverages conformal prediction to quantify image uncertainty at runtime and was validated on two datasets, demonstrating superior efficiency and accuracy at a computational cost up to 12 times lower than heavy-model inference.

What carries the argument

Conformal prediction to quantify image uncertainty at runtime for dynamic routing decisions across a model cascade.

If this is right

  • High-capacity models are reserved for complex samples while routine images use lightweight ones.
  • Average inference cost drops substantially without loss of diagnostic reliability.
  • The approach supports deployment where compute budgets are limited.
  • Environmental impact of repeated heavy-model use is reduced.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The routing logic could be tested on additional image datasets or other data types to check broader applicability.
  • Pairing CADS with existing model compression methods might produce further efficiency gains.
  • Design of future model families may shift toward versions explicitly built to work together in cascades.

Load-bearing premise

Conformal prediction supplies sufficiently calibrated uncertainty estimates at runtime to make correct routing decisions that preserve overall accuracy.

What would settle it

Direct measurement on the two validation datasets showing that CADS accuracy falls below the Oracle model alone or that the claimed cost reduction is not realized.

Figures

Figures reproduced from arXiv: 2605.16401 by Bary Tim, Dausort Manon, Macq Beno\^it, Thielens Vincent, Turkoglu Mikael.

Figure 1
Figure 1. Figure 1: Performance evaluation on PathMNIST and CIFAR-100 showing a significant cost reduction as well as surpassing the top-performing individual expert by leveraging model complementarity [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Proportion of expert calls in CADS according to the three groups for PathMNIST and CIFAR-100. The system prioritizes Scouts at low budgets, progressively escalating to Specialists and Oracles as computational constraints are relaxed. system granularly routes complex samples to heavier ar￾chitectures to resolve ambiguities. This adaptive tran￾sition optimizes the cost-accuracy trade-off, maintaining high re… view at source ↗
read the original abstract

While high-capacity AI models have advanced state-of-the-art performance, their practical deployment is often hindered by high inference costs, environmental impact, and a "one-size-fits-all" approach that ignores varying sample complexity. In clinical settings for instance, the waste of computational resources on routine cases is a significant barrier to sustainable AI. In this paper, we introduce the Conformal Adaptive Decision System (CADS), a sequential multi-model algorithm designed to optimize resource allocation by efficiently sampling models based on the estimated data complexity. CADS leverages conformal prediction to quantify image uncertainty at runtime. CADS provides a mathematically grounded framework for balancing the cost-accuracy dilemma that dynamically routes samples through a model cascade, ranging from lightweight "Scout" models to high-capacity "Oracle" architectures. Validated on two datasets, CADS demonstrated superior efficiency and accuracy at a computational cost that can be up to 12 times lower than heavy-model inference. By accurately routing samples based on real-time complexity, CADS ensures high diagnostic reliability while drastically reducing the economic and environmental footprint of AI.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces CADS, a sequential multi-model cascade that uses conformal prediction to estimate runtime uncertainty and route image samples from lightweight Scout models to high-capacity Oracle models. The central claim is that this yields up to 12× lower computational cost than always using the heavy model while preserving or improving overall accuracy, with validation reported on two datasets.

Significance. If the routing decisions preserve accuracy without hidden degradation, the approach could meaningfully reduce inference cost and environmental impact for variable-complexity tasks such as clinical imaging. The reliance on standard conformal-prediction validity is a positive, as it avoids self-referential fitting; however, the absence of quantitative controls makes it difficult to judge whether the reported efficiency gains are robust.

major comments (2)
  1. [Abstract / Experiments] Abstract and Experiments section: the claim of 'up to 12 times lower' cost with 'superior efficiency and accuracy' is stated without error bars, baseline comparisons (e.g., always-Oracles, random routing, or fixed-threshold cascades), dataset details, or statistical tests. This leaves the central efficiency claim without visible quantitative support.
  2. [Method] Method / Routing subsection: conformal prediction supplies marginal coverage, yet the manuscript provides no ablation or measurement showing that the nonconformity score reliably identifies samples on which the Scout model would err. Without an accuracy comparison to the always-Oracle baseline on the routed subset, it is unclear whether the reported cost reduction preserves the accuracy claim.
minor comments (2)
  1. [Method] Add explicit definitions and values for the uncertainty threshold used for escalation; currently it appears as a free parameter without sensitivity analysis.
  2. [Experiments] Clarify the exact architectures and training regimes of the Scout and Oracle models, and state the two datasets by name with standard references.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments on our manuscript. We address each major comment below and have outlined the specific revisions we will incorporate to strengthen the quantitative support and methodological clarity of the work.

read point-by-point responses

  1. Referee: [Abstract / Experiments] Abstract and Experiments section: the claim of 'up to 12 times lower' cost with 'superior efficiency and accuracy' is stated without error bars, baseline comparisons (e.g., always-Oracles, random routing, or fixed-threshold cascades), dataset details, or statistical tests. This leaves the central efficiency claim without visible quantitative support.

    Authors: We agree that the central efficiency claims would benefit from additional quantitative controls and statistical grounding. In the revised manuscript we will expand the Experiments section to report all cost and accuracy metrics with error bars (standard deviation across multiple runs or cross-validation folds), include direct comparisons against always-Oracle, random routing, and fixed-threshold cascade baselines, provide fuller dataset descriptions (including sample counts, class distributions, and preprocessing), and add statistical significance tests (e.g., paired t-tests or Wilcoxon signed-rank tests with p-values). The Abstract will be updated to reference these supporting analyses while preserving the original performance summary. revision: yes

  2. Referee: [Method] Method / Routing subsection: conformal prediction supplies marginal coverage, yet the manuscript provides no ablation or measurement showing that the nonconformity score reliably identifies samples on which the Scout model would err. Without an accuracy comparison to the always-Oracle baseline on the routed subset, it is unclear whether the reported cost reduction preserves the accuracy claim.

    Authors: The referee correctly notes that an explicit link between the nonconformity score and Scout-model errors is not currently demonstrated. In the revision we will add an ablation in the Method / Routing subsection that reports (i) the Scout model’s error rate on the subset of samples routed to the Oracle versus the subset retained by the Scout, and (ii) the end-to-end accuracy of CADS compared with the always-Oracle baseline restricted to the routed subset. This analysis will quantify how well the conformal score identifies difficult samples and will confirm that accuracy is maintained on the routed portion while realizing the reported cost savings. revision: yes

Circularity Check

0 steps flagged

No circularity in CADS derivation chain

full rationale

The paper's central construction applies standard conformal prediction to produce runtime uncertainty scores that drive routing decisions across a Scout-to-Oracle model cascade. Conformal prediction validity is an externally established property drawn from prior literature, not derived or fitted inside this work. Efficiency and accuracy claims rest on empirical validation across two datasets rather than any self-definitional loop, fitted parameter renamed as prediction, or load-bearing self-citation. No equation reduces the reported routing or cost savings to the inputs by construction; the framework remains independent of its own outputs.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim depends on the validity of conformal prediction for producing actionable uncertainty scores and on the existence of a useful separation between easy and hard images that the routing can exploit. No explicit free parameters or invented entities are named in the abstract, but threshold selection for escalation is implicitly required.

free parameters (1)
  • uncertainty threshold for escalation
    A decision threshold must be chosen or fitted to determine when to route from Scout to Oracle; its value directly controls the cost-accuracy trade-off.
axioms (1)
  • domain assumption Conformal prediction yields valid and useful uncertainty quantification for image classification at inference time
    The routing logic rests on this standard property of conformal methods being reliable enough to guide model selection without accuracy loss.

pith-pipeline@v0.9.0 · 5730 in / 1308 out tokens · 112181 ms · 2026-05-20T21:47:04.074795+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 7 internal anchors

  1. [1]

    This ”bigger is bet- ter” paradigm has resulted in a dramatic rise in com- pute requirements and storage footprints [ 2]

    INTRODUCTION The unprecedented success of artificial intelligence has been largely driven by ”scaling laws”, where increasing model complexity and parameters consistently lead to state-of-the-art performance [ 1]. This ”bigger is bet- ter” paradigm has resulted in a dramatic rise in com- pute requirements and storage footprints [ 2]. However, this traject...

    work page

  2. [2]

    CADS: Conformal Adaptive Decision System for Cost-Efficient Image Classification

    RELATED WORKS The pursuit of computational efficiency has led to the de- velopment of architectures that move away from static, ”dense” computation toward conditional computation, where only specific parts of a model are activated per sample. The multi-expert paradigm, originally intro- duced by Jacobs et al. [ 7], has seen a massive resurgence arXiv:2605....

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  3. [3]

    METHODOLOGY In this section, we introduce a formal description of the CADS method and its optimization framework, followed by the pool of models and the datasets used for valida- tion. 3.1. CADS: Conformal Adaptive Decision System To answer the tension existing between lightweight mod- els (less-compute intensive but less accurate for complex cases) and h...

    work page

  4. [4]

    The set size acts as a proxy for difficulty: a singleton implies certainty, while sets containing three or more classes trigger further expert consultation

    Conformal prediction: instead of relying on soft la- bels, CADS constructs a prediction set C(x). The set size acts as a proxy for difficulty: a singleton implies certainty, while sets containing three or more classes trigger further expert consultation

    work page

  5. [5]

    Complementarity analysis: the cascade of model in- ferences is adaptive, if a model exhibits uncertainty between specific classes, the system dynamically se- lects the expert historically most proficient for that specific confusion

    work page

  6. [6]

    Two-level weighted ensemble: predictions are aggre- gated using weights assigned both globally (based on general accuracy) and locally (prioritizing experts with statistical strength in the specific suspected class). 3.1.1. Problem Setting Given K pre-trained experts E = {k1, . . . , kK} with as- sociated costs {g1, . . . , gK} and accuracies, the objecti...

    work page

  7. [7]

    Global: measured across the entire calibration dataset: Comp(A, B) = P[ˆy(B) = y | ˆy(A) ̸= y] (4)

    work page

  8. [8]

    Pair-wise: specifically targeting confusion between a class pair (c1, c2): Compc1,c2 (A, B) = P ( ˆy(B) = y | y, ˆy(A) = c1, c2 ) (5) These scores are precomputed on a calibration set to cap- ture fine-grained failure modes, allowing the system to selectively invoke experts based on their historical per- formance in resolving specific type of uncertaintie...

    work page

  9. [9]

    Consensus boost αboosted: if more than 80% of con- sulted experts agree on the same class, the required confidence is lowered by a factor δ multiplied by the number of expert consulted (up to a limit δmax), acknowledging that multi-expert agreement reduces risk: αboosted = αbase − min(δ · (|used| − 1), δmax) (9)

    work page

  10. [10]

    Class difficulty adjustment: the threshold is further refined based on the historical difficulty dc∗ of the consensus class precomputed during calibration: αfinal = min(αboosted + (dc∗ − 0.5) · 0.1, 0.98) (10) The cascade then terminates if all available experts have been consulted or if the following conditions are si- multaneously satisfied: a minimum num...

    work page

  11. [11]

    Models are categorized into three groups based on their cost and complexity

    RESULTS The experimental evaluation of the CADS framework demonstrates significant performance gains across the Table 1. Models are categorized into three groups based on their cost and complexity. ”Scout” handles easy sam- ples, while ”Oracle” serves for high-uncertainty cases. Model Ref Param. (M) GFLOPs Scout MobileNetV3 Small [ 16] 2.5 0.01 EfficientNe...

    work page

  12. [12]

    To validate the conformal routing, we compared the APS conformal routing against standard uncertainty measures, including Max Softmax, Entropy, and Margin

    ABLATION STUDIES We conducted three primary ablation studies to vali- date the structural choices of the CADS framework and demonstrate the necessity of its individual components. To validate the conformal routing, we compared the APS conformal routing against standard uncertainty measures, including Max Softmax, Entropy, and Margin. On CIF AR-100 at a 9 ...

    work page

  13. [13]

    By leveraging conformal prediction, this approach uses lightweight models to quantify the uncertainty and complexity of an image

    CONCLUSION In this paper, we introduce CADS as an advanced multi- model methodology. By leveraging conformal prediction, this approach uses lightweight models to quantify the uncertainty and complexity of an image. The presented methods allows the system to intelligently route difficult samples to larger models only when statistically neces- sary, based on...

    work page

  14. [14]

    Scaling Laws for Neural Language Models

    Jared Kaplan et al., “Scaling laws for neural lan- guage models,” arXiv:2001.08361, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  15. [15]

    Compute trends across three eras of machine learning,

    Jaime Sevilla et al., “Compute trends across three eras of machine learning,” in International joint con- ference on neural networks (IJCNN). IEEE, 2022, pp. 1–8

    work page 2022

  16. [16]

    Unraveling the hidden environmen- tal impacts of ai solutions for environment life cycle assessment of ai solutions,

    Ligozat et al., “Unraveling the hidden environmen- tal impacts of ai solutions for environment life cycle assessment of ai solutions,” Sustainability, vol. 14, 2022

    work page 2022

  17. [17]

    Temporal quality degradation in ai models,

    D Vela et al., “Temporal quality degradation in ai models,” Scientific reports, vol. 12, no. 1, pp. 11654, 2022

    work page 2022

  18. [18]

    Estimating the difficulty of medical classification tasks using 3d image datasets,

    T Thornblad et al., “Estimating the difficulty of medical classification tasks using 3d image datasets,” in Annual Intern. Conf. of the IEEE En- gineering in Medicine and Biology Society., 2025, vol. 2025, pp. 1–7

    work page 2025

  19. [19]

    No need for learning to defer? a training free deferral frame- work to multiple experts through conformal predic- tion,

    Tim Bary, Benoît Macq, and Louis Petit, “No need for learning to defer? a training free deferral frame- work to multiple experts through conformal predic- tion,” arXiv preprint arXiv:2509.12573, 2025

    work page arXiv 2025

  20. [20]

    Adaptive mixtures of local ex- perts,

    R Jacobs et al., “Adaptive mixtures of local ex- perts,” Neural computation, vol. 3, no. 1, pp. 79–87, 1991

    work page 1991

  21. [21]

    Mixtral of Experts

    Albert Q Jiang et al., “Mixtral of experts,” arXiv preprint arXiv:2401.04088, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  22. [22]

    DeepSeek-V3 Technical Report

    Aixin Liu et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    Branchynet: Fast in- ference via early exiting from deep neural networks,

    Surat Teerapittayanon et al., “Branchynet: Fast in- ference via early exiting from deep neural networks,” in 23rd international conference on pattern recogni- tion (ICPR). IEEE, 2016, pp. 2464–2469

    work page 2016

  24. [24]

    Msdnet for medical image fusion,

    Song et al., “Msdnet for medical image fusion,” in Int. conf. on image and graphics. Springer, 2019, p. 278

    work page 2019

  25. [25]

    Towards inference efficient deep ensemble learning,

    Ziyue Li et al., “Towards inference efficient deep ensemble learning,” in Proceedings of the AAAI Conf. on Artificial Intelligence, 2023, vol. 37, pp. 8711–8719

    work page 2023

  26. [26]

    Adaptive neural networks for efficient inference,

    Tolga Bolukbasi et al., “Adaptive neural networks for efficient inference,” in International conference on machine learning. PMLR, 2017, pp. 527–536

    work page 2017

  27. [27]

    A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

    Anastasios N Angelopoulos and Stephen Bates, “A gentle introduction to conformal prediction and distribution-free uncertainty quantification,” arXiv preprint arXiv:2107.07511, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  28. [28]

    Optuna: A next-generation hyperparameter optimization framework,

    Takuya Akiba et al., “Optuna: A next-generation hyperparameter optimization framework,” in Pro- ceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Min- ing, 2019

    work page 2019

  29. [29]

    Searching for mobilenetv3,

    Andrew Howard et al., “Searching for mobilenetv3,” in Proceedings of the IEEE international conf. on computer vision, 2019, pp. 1314–1324

    work page 2019

  30. [30]

    Efficientnet: Rethinking model scaling for convolutional neural networks,

    Mingxing Tan et al., “Efficientnet: Rethinking model scaling for convolutional neural networks,” in Int. conf. on ML. PMLR, 2019, pp. 6105–6114

    work page 2019

  31. [31]

    Ghostnet: More features from cheap operations,

    Kai Han et al., “Ghostnet: More features from cheap operations,” in Proceedings of the IEEE conf. on computer vision and pattern recognition, 2020, pp. 1580–1589

    work page 2020

  32. [32]

    MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer

    Sachin Mehta et al., “Mobilevit: light-weight, general-purpose, and mobile-friendly vision trans- former,” arXiv preprint arXiv:2110.02178, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  33. [33]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders,

    Sanghyun Woo et al., “Convnext v2: Co-designing and scaling convnets with masked autoencoders,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2023, pp. 16133– 16142

    work page 2023

  34. [34]

    Eva-02: A visual representation for neon genesis,

    Q Sun et al., “Eva-02: A visual representation for neon genesis,” arXiv preprint arXiv:2303.11331, 2023

    work page arXiv 2023

  35. [35]

    Efficientnetv2: Smaller models and faster training,

    Mingxing Tan and Quoc Le, “Efficientnetv2: Smaller models and faster training,” in Interna- tional conference on machine learning. PMLR, 2021, pp. 10096–10106

    work page 2021

  36. [36]

    Swin transformer v2: Scaling up ca- pacity and resolution,

    Liu et al., “Swin transformer v2: Scaling up ca- pacity and resolution,” in Proceed. of the IEEE on computer vision and pattern recognition, 2022, pp. 12009–12019

    work page 2022

  37. [37]

    DINOv2: Learning Robust Visual Features without Supervision

    Maxime Oquab et al., “Dinov2: Learning robust visual features without supervision,” arXiv preprint arXiv:2304.07193, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    Maxvit: Multi-axis vision transformer,

    Tu et al., “Maxvit: Multi-axis vision transformer,” in Eu. conf. on comp. vision. Springer, 2022, pp. 459–479

    work page 2022

  39. [39]

    Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical im- age classification,

    Jiancheng Yang et al., “Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical im- age classification,” Scientific Data, vol. 10, no. 1, pp. 41, 2023

    work page 2023

  40. [40]

    Learning multiple layers of features from tiny images.,

    Alex Krizhevsky et al., “Learning multiple layers of features from tiny images.,” 2009

    work page 2009