Pith. sign in

REVIEW 3 major objections 5 minor 18 references

The paper claims that rigorous AI evaluation can be run as composable microservices without losing the formal statistical guarantees of conformal prediction, drift detection, and fairness monitoring.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 08:46 UTC pith:PTEH3N5V

load-bearing objection Competent systems integration with a clean conformal multi-split experiment, but the open-source claim is unverifiable without a code release. the 3 major comments →

arxiv 2607.21623 v1 pith:PTEH3N5V submitted 2026-07-04 cs.LG

Cloud-Native Evaluation-as-a-Service: A Microservices Architecture for Scalable AI Monitoring with Conformal Guarantees

Lei Yang This is my paper
classification cs.LG
keywords conformal predictionmicroservicesdrift detectionfairness monitoringcalibration assessmentDAG orchestrationcoverage guaranteecloud-native AI evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that rigorous AI evaluation methods can be operationalized as six stateless microservices on container-orchestration infrastructure: conformal prediction with finite-sample coverage guarantees, calibration assessment, drift detection via random-Fourier-feature MMD, fairness monitoring with bootstrap intervals, a DAG orchestrator, and a result store. The author validates that empirical conformal coverage stays within 1.4 percentage points of the nominal target across 50 random calibration/test splits, that drift detection achieves full detection power at the median kernel bandwidth with well-calibrated false positives, and that fairness monitoring flags a real demographic-parity gap of 0.33 on a standard income dataset. The reason a reader would care: if true, production AI systems can obtain continuous, composable evaluation with formal statistical backing instead of offline benchmarks or black-box commercial monitors. The paper positions the architecture as the first open-source platform combining conformal-prediction-as-a-service, microservice decomposition, and DAG orchestration.

Core claim

On its own terms, the central claim is that the statistical guarantees of conformal prediction survive deployment as a stateless service: with the finite-sample-corrected APS quantile q̂ = Quantile(s_1,...,s_n; ⌈(n+1)(1−α)⌉/n), empirical coverage across K=50 random splits of 500 real language-model answers tracks the 1−α target, with Wilson 95% confidence intervals containing the target at every α level and mean coverage within 1.4 points. The same service stack detects multivariate shift: RFF-approximated MMD agrees with exact MMD on 768-dimensional embeddings, achieving 100% power for mild and severe drift at the median-heuristic bandwidth with Type I error between 5–8.5%. Fairness monitor

What carries the argument

The load-bearing mechanism is the decomposition of each evaluation method into a stateless, HTTP-served microservice (conformal prediction with Adaptive Prediction Sets and the ⌈(n+1)(1−α)⌉/n quantile correction; four ECE estimators; KS/PSI plus RFF-approximated Maximum Mean Discrepancy for drift; demographic parity and equalized odds with bootstrap confidence intervals), orchestrated by a lightweight DAG executor that manages retries, backpressure, and idempotent caching. The RFF approximation reduces MMD from O(n²D) to O(n·D·n_rff), making high-dimensional drift detection practical at about 500 milliseconds; the conformal quantile correction is what transfers the finite-sample coverage gua

Load-bearing premise

The strongest claim—production-ready, guarantee-carrying AI monitoring—rests on the assumption that measurements taken in single-process benchmarks (sub-2 ms p99) and on one language-model sample plus one tabular dataset transfer to real multi-pod container deployments with network overhead, autoscaling delays, and other data distributions.

What would settle it

Deploy the six microservices on a real multi-node container cluster, run the same 50-split conformal evaluation on live MMLU-style queries through the HTTP endpoints, and measure end-to-end coverage and p99 latency under actual network hops and autoscaling. If coverage deviates from the 1−α target by more than the finite-sample bound 1/(n_cal+1) or the end-to-end p99 latency exceeds the reported sub-2 ms, the central claim is falsified. Alternatively, a single counterexample dataset where conformal coverage fails away from the marginal guarantee across random splits would weaken the generality

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Production deployments can run conformal prediction as a service and trust the marginal coverage guarantee across recalibration splits, not just a single split.
  • Drift detection on high-dimensional embeddings can run periodically in about 500 milliseconds with well-calibrated Type I error, making multivariate shift monitoring practical.
  • Fairness violations can be detected on real data with bootstrap uncertainty and stable sequential alerts, moving beyond synthetic fairness checks.
  • Evaluation pipelines can be composed as DAGs with retry, idempotency, and backpressure, enabling automated, auditable continuous evaluation.
  • The sub-2 ms p99 latency of core services at batch size 100 supports integration into per-batch monitoring loops.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's single-process latency and multiprocessing scaling are proxies; a real container-orchestration deployment with network hops and autoscaling delays will likely increase end-to-end latency, so the sub-2 ms figure should be read as a component-level bound, not a system-level one.
  • The conformal guarantee is marginal over the calibration set; the paper's 50-split analysis suggests stability in practice, but production users should still recalibrate on each new data batch and monitor coverage drift over time.
  • The fairness findings on the standard income dataset with an education-based classifier show that even simple thresholds reveal demographic gaps; applying the same service to a production model's actual outputs would be a direct test of the architecture's value.
  • The RFF-MMD hyperparameter sweep implies an operational prescription: use the median-heuristic bandwidth and at least 200 permutations to keep Type I error below 0.085 while retaining full power—this is a directly transferable rule of thumb.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EAAS, a cloud-native reference architecture that packages AI evaluation methods as six stateless Kubernetes microservices: conformal prediction with APS scores, multi-variant calibration assessment, KS/PSI and RFF-approximated MMD drift detection, fairness monitoring with bootstrap CIs, a DAG orchestrator, and a result storage API. The empirical sections report: (i) conformal coverage close to nominal across K=50 random calibration/test splits on 500 real GPT-4O-MINI MMLU logits, with Wilson CIs containing the target; (ii) bounded impact of simulated top-20 logprob imputation; (iii) RFF-MMD drift detection with full power at the median-heuristic bandwidth and well-calibrated Type I error across broad hyperparameter sweeps; (iv) fairness monitoring on UCI Adult revealing a demographic-parity gap of 0.33 by race; and (v) sub-2 ms p99 latency for conformal/calibration services at batch size 100. The authors claim this is the first open-source microservices architecture combining conformal prediction, microservice decomposition, and DAG orchestration.

Significance. If the implementation were publicly available and the deployment claims validated, EAAS would be a useful integration of rigorous statistical methods — conformal prediction, calibration assessment, multivariate drift detection, and fairness monitoring — into composable cloud-native services. The paper's empirical core is careful in several respects: the K=50 multi-split coverage experiment (Table 6) is a clean demonstration of the marginal conformal guarantee; the RFF-MMD sensitivity ablations (Tables 9, Figures 8–9) systematically probe bandwidth, number of random features, and permutation count; and the fairness analysis uses bootstrap intervals on real data. The main value is as a reference architecture and as evidence that these methods can be exposed as low-latency services. However, the central 'first open-source' contribution is not verifiable from the manuscript: no code repository, Docker/Helm charts, or artifact URL is provided, and the scaling results are a multiprocessing simulation rather than a real Kubernetes deployment. These gaps materially limit the contribution's current significance.

major comments (3)
  1. [§1, Contributions and Table 13] The central claim is that EAAS is 'the first open-source microservices architecture' that operationalizes conformal prediction, drift detection, fairness, and calibration, and Table 13 marks 'Open Source' with a ✓. Yet the full text provides no repository URL, no Docker/Helm charts, no code listing, and no artifact link. The 'implement' part of 'we design, implement, and empirically validate' is therefore unverifiable and the novelty claim reduces to a paper design. This is load-bearing for the contribution. The authors must either provide a publicly accessible, versioned code repository with the microservices, or substantially rephrase the contribution as a design/reference architecture and remove the 'Open Source' checkmark.
  2. [§4.7 and Limitation 1] The horizontal-scaling evidence is a multiprocessing simulation, not a real multi-pod Kubernetes deployment. The text correctly acknowledges this in Limitation 1, but the abstract and conclusion still describe EAAS as a 'scalable' architecture and Figure 13 shows 'projected throughput scales linearly' with no empirical measurement of network overhead (~0.1–1 ms per hop) or HPA scale-out times (30–60 s), both named in Limitation 1. Since scalability is part of the paper's central claim, the manuscript should either report a real deployment experiment on Kubernetes (even a small cluster) or consistently present the scaling result as a single-process projection that has not been validated in the target deployment environment.
  3. [§4.5, P2B] The headline fairness result — DP gap = 0.33 by race with stable alerts — is obtained using an education-based threshold classifier on UCI Adult, not a production ML model or even a standard trained classifier. Limitation 5 acknowledges this, but the abstract and conclusion present the result as validating fairness monitoring on real data. The gap value and alert behavior are properties of this proxy classifier, and it is not established that they would transfer to a deployed model. The authors should either run the fairness service with a real trained classifier (e.g., a logistic regression or gradient-boosted model on Adult) or explicitly scope the claim in the abstract and conclusion to a threshold classifier.
minor comments (5)
  1. [Throughout] The acronym is inconsistent: 'EAAS' appears in the title and most of the text, but the Abstract and Section 3.1 use 'EaaS'. Please standardize.
  2. [Table 5, §4.2] At α = 0.05 the single-split empirical coverage is 0.973, 2.3 percentage points above the 0.95 target. The multi-split mean in Table 6 (0.954, std 0.021) is reassuring, but the single-split point should be acknowledged as somewhat high; the current text says only that it is 'within statistical fluctuation.'
  3. [§4.9, Table 13] The 'Open Source' row for EAAS should either be removed or cite the repository. As written, it is an assertion without supporting evidence.
  4. [§3.2, Eq. (3)] The RFF feature definition uses z(x)=√(2/D_rff)·cos(Wx+b), but standard RFF normalization is typically √(2/D_rff) only if W is scaled appropriately. Please clarify the exact scaling of W and b, since this affects the MMD estimator bias.
  5. [§5, Limitations] Limitation 7 states that bootstrap CIs assume independent samples; this is a fair and useful caveat, but it applies to the fairness batch-stability analysis in §4.5, not just to the general method. Please note explicitly that the reported CI widths for the sequential batches inherit this assumption.

Circularity Check

0 steps flagged

No significant circularity: empirical validations are independent checks of external algorithms; the unsupported 'open-source' claim is a verifiability issue, not a circular derivation.

full rationale

This is a systems and empirical-validation paper, not a derivational paper. The load-bearing quantitative claims are checks of externally established algorithms against data: conformal coverage uses the Angelopoulos-Bates finite-sample quantile (Eq. 2) and is evaluated on held-out calibration/test splits, including K=50 random splits with Wilson CIs; RFF-MMD uses Rahimi-Recht random features with the standard median-heuristic bandwidth chosen before evaluation, and the sensitivity sweep reports degraded power at non-default settings; fairness disparities are direct sample statistics with bootstrap CIs, not fitted outputs. No 'prediction' is constructed from a parameter fitted to the same target quantity. The paper contains no load-bearing self-citation chain: the cited conformal, drift, and fairness results are external, published theorems or benchmark datasets. The 'first open-source' claim is unsupported because no repository or artifact URL appears anywhere in the manuscript, and Table 13 marks 'Open Source' as a feature; however, that is a factual-support/verifiability problem, not a circular derivation. The scaling and latency numbers are explicitly limited to single-process/multiprocessing benchmarks (Limitations 1), which weakens production claims but does not create circularity. Accordingly, the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The system introduces no new theoretical entities; the free parameters are operational hyperparameters and dataset choices. The most consequential hand-set choices are the RFF bandwidth, the minimum-sample threshold, and the alert threshold, because the fairness and drift headline numbers depend on them. All statistical foundations are standard and were not invented for this paper.

free parameters (6)
  • RFF-MMD bandwidth sigma = 1.0 x sigma_median via median heuristic on 500 pairwise distances
    Chosen by standard heuristic; central to MMD power/Type I claims (Sec 3.2 Eq 3; Sec 4.4 P2A).
  • Number of random features n_rff = 500
    Default used in drift-detection validation; affects approximation variance and cost (Sec 4.4).
  • Permutation count for MMD p-values = 200
    Used for p-values; the paper shows 50 permutations inflate Type I error and 200 are needed (Sec 4.4 P2A).
  • Minimum-sample threshold for fairness groups = default 30; 200 and 500 used in experiments
    Suppresses small subgroups; changes the race DP gap from 0.330 to 0.288 at threshold 500 (Sec 4.5).
  • Fairness alert threshold = 0.10 disparity gap
    Hand-set; determines zero alerts by sex vs high-severity alerts by race (Sec 3.2, Sec 4.5).
  • Education-based threshold classifier on UCI Adult = not specified numerically
    Proxy classifier used for fairness monitoring; external validity is limited and acknowledged (Sec 4.5, Limitation 5).
axioms (6)
  • standard math Finite-sample split conformal guarantee with corrected quantile q = Quantile(s; ceil((n+1)(1-alpha))/n)
    Invoked in Eq. 2; foundational for all coverage claims (Sec 3.2).
  • standard math Random Fourier features approximate the Gaussian kernel with W ~ Normal(0, sigma^-2 I)
    Eq. 3; basis for RFF-MMD drift detection.
  • domain assumption Bootstrap percentile intervals are valid for fairness metric CIs under independent sampling
    Used in Sec 4.5; the paper acknowledges temporal correlations in production data could violate this (Limitation 7).
  • domain assumption GPT-4o-mini top-20 logprobs from the OpenAI API are sufficient for MMLU answer-token conformal scoring
    Sec 3.3 and P1B; verified only for 500 questions, not generally.
  • domain assumption Synthetic Gaussian shift scenarios model realistic production drift
    Sec 3.3 and Sec 4.4; power claims are tied to shift magnitudes mu=0.5/2.0.
  • domain assumption UCI Adult with an education-based threshold classifier is a valid fairness monitoring testbed
    Sec 4.5; the paper acknowledges the Ding et al. critique and the proxy-classifier limitation.

pith-pipeline@v1.3.0-alltime-deepseek · 15345 in / 15810 out tokens · 162505 ms · 2026-08-02T08:46:45.835624+00:00 · methodology

0 comments
read the original abstract

We present EaaS, a cloud-native reference architecture that operationalizes AI evaluation methods as six stateless Kubernetes microservices: conformal prediction with finite-sample-corrected Adaptive Prediction Sets, calibration assessment, drift detection via RFF-approximated Maximum Mean Discrepancy, fairness monitoring with bootstrap confidence intervals, a DAG-based pipeline orchestrator, and a result storage API. We validate four key methodological concerns. First, empirical coverage is consistent with the marginal conformal guarantee across K=50 random calibration/test splits, with mean coverage within 1.4 percentage points of the nominal target. Second, all four MMLU answer tokens appear in the top-20 logprobs with 0% imputation needed, and simulated imputation at 10% produces less than 1.5% coverage impact. Third, RFF-MMD achieves 100% detection power for mild and severe drift at the median heuristic bandwidth, with Type I error between 5-8.5%. Fourth, fairness monitoring on the UCI Adult Income dataset reveals significant demographic parity disparities by race (DP gap=0.33) with stable alerts across sequential batches. Conformal prediction and calibration services achieve sub-2ms p99 latency at batch size 100; RFF-MMD requires ~500ms suited for periodic batch monitoring. A comparison with four open-source tools suggests that, to the best of our knowledge, no current platform combines conformal-prediction-as-a-service, microservice decomposition, and DAG-based orchestration.

Figures

Figures reproduced from arXiv: 2607.21623 by Lei Yang.

Figure 1
Figure 1. Figure 1: EaaS reference architecture. Six stateless microservices are orchestrated via a DAG [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Conformal prediction coverage validation. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Coverage distribution across 50 random calibration/test splits on real GPT-4 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Logprob imputation sensitivity. Coverage and mean set size as a function of imputation [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: ECE estimator comparison. (a) All four estimators correctly distinguish lower- from higher-miscalibration outputs; debiased ECE removes finite-sample bias. (b) Fixed-bin ECE varies with bin count while adaptive and debiased ECE remain stable. Bin sensitivity. Fixed ECE varies by up to 0.02 across M = 5 to 50 bins, while adaptive and debiased ECE remain stable. This stability makes adaptive ECE preferable f… view at source ↗
Figure 6
Figure 6. Figure 6: Drift detection results. (a) Univariate KS+PSI detection across drift magnitudes. (b) RFF￾MMD approximation quality vs. number of random features. (c) ROC curves for KS and MMD drift detection. RFF-MMD sensitivity ablations (P2A). We systematically evaluate three hyperparameter dimen￾sions. (i) nrff sweep: the RFF estimate standard deviation decreases from 0.004 (nrff=50) to 0.001 (nrff=1000), showing that… view at source ↗
Figure 7
Figure 7. Figure 7: RFF-MMD approximation variance vs. nrff. Estimate variance decreases with nrff; drift￾detection decisions are consistent with exact MMD across all values tested [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Type I error and power vs. bandwidth multiplier ( [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: p-value calibration QQ plots under the null (no drift). With ≥ 200 permutations, p-values are well-calibrated (close to the diagonal). With 50 permutations, discretization inflates the rejection rate. 4.5 Fairness Monitoring [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Fairness monitoring results on synthetic data. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Fairness monitoring on real UCI ADULT INCOME data. (a) Demographic parity by sex and race with bootstrap 95% CIs. (b) Minimum-sample threshold effect. (c) Batch stability of DP gap and alerts over 10 sequential batches. 4.6 Service Latency [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Service latency benchmarks. (a) Conformal prediction latency scales linearly with batch size. (b) Per-service p99 latency comparison. (c) KS drift detection scaling with number of features. tions requires 458–590 ms (p50) depending on nrff, making it suitable for periodic batch evaluation rather than per-inference monitoring. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Horizontal scaling. (a) Per-request latency remains constant across replica counts. (b) Projected throughput scales linearly; observed multiprocessing throughput is limited by IPC serialization, not service compute. The low aggregate RPS in the multiprocessing simulation (41–151 RPS for 2–8 replicas) is an arti￾fact of Python’s ProcessPoolExecutor IPC serialization overhead (NumPy array pickling), not a s… view at source ↗
Figure 14
Figure 14. Figure 14: Pipeline orchestrator validation. (a) Full pipeline DAG execution (22.0 ms total). (b) Idempotency cache provides 157× speedup. (c) Backpressure via semaphore controls concur￾rency. The individual node latencies are: conformal 10.72 ms, metrics 2.17 ms, drift 5.21 ms, fairness 2.74 ms, and aggregate 0.02 ms. Retry behavior is verified with 2 retries before success on a tran￾sient failure. The idempotency … view at source ↗
Figure 15
Figure 15. Figure 15: Feature comparison heatmap of EAAS against existing monitoring and evaluation tools across 12 capability dimensions [PITH_FULL_IMAGE:figures/full_fig_p017_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 14 linked inside Pith

  1. [1]

    Detection of data drift and outliers affecting machine learning model performance over time.arXiv preprint arXiv:2012.09258,

    Samuel Ackerman, Eitan Farchi, Orna Raz, et al. Detection of data drift and outliers affecting machine learning model performance over time.arXiv preprint arXiv:2012.09258,

    Pith/arXiv arXiv 2012

  2. [5]

    On the computational-statistical tradeoff in kernel two-sample testing with random fourier features.arXiv preprint arXiv:2602.09787,

    Ikjun Choi and Ilmun Kim. On the computational-statistical tradeoff in kernel two-sample testing with random fourier features.arXiv preprint arXiv:2602.09787,

    arXiv

  3. [9]

    Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain

    arXiv:2205.02302. Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. Verified uncertainty calibration. InAdvances in Neural Information Processing Systems,

    Pith/arXiv arXiv

  4. [11]

    National Institute of Standards and Technology

    arXiv:2211.09110. National Institute of Standards and Technology. Artificial intelligence risk management framework (AI RMF 1.0).NIST AI 100-1,

    Pith/arXiv arXiv

  5. [12]

    Victor Quach, Adam Fisch, Tal Schuster, et al

    arXiv:2509.14294. Victor Quach, Adam Fisch, Tal Schuster, et al. Conformal language modeling with vocabulary- adaptive prediction sets.arXiv preprint arXiv:2307.02598,

    arXiv

  6. [13]

    Liangjie Su et al

    arXiv:2401.00588. Liangjie Su et al. LofreeCP: Logit-free conformal prediction for token-level uncertainty in LLMs. arXiv preprint arXiv:2408.05558,

    Pith/arXiv arXiv

  7. [14]

    TrustLLM: Trustworthiness in large lan- guage models.arXiv preprint arXiv:2401.05561,

    Lichao Sun, Yue Huang, Haoran Wang, Siyue Wu, et al. TrustLLM: Trustworthiness in large lan- guage models.arXiv preprint arXiv:2401.05561,

    Pith/arXiv arXiv

  8. [15]

    Microservice resource allocation with latency-based measurements under concur- rent load.arXiv preprint arXiv:2404.15835,

    Stefan Ursa et al. Microservice resource allocation with latency-based measurements under concur- rent load.arXiv preprint arXiv:2404.15835,

    Pith/arXiv arXiv

  9. [16]

    Partial Wasserstein and MMD distances for outlier and drift detection.arXiv preprint arXiv:2106.12893,

    Thomas Viehmann. Partial Wasserstein and MMD distances for outlier and drift detection.arXiv preprint arXiv:2106.12893,

    Pith/arXiv arXiv

  10. [18]

    We summarize the key endpoints below

    21 A API Specification Each EAAS service exposes a FastAPI auto-generated OpenAPI specification atGET /docs. We summarize the key endpoints below. Conformal Prediction Service (port 8001).POST /calibrateaccepts softmax probabilities, labels, significance levelα, method (APS or simple), and a randomize flag, returning the calibration thresholdˆqand adjuste...

    2023

  11. [1995]

    Training-conditional coverage for distribution-free predictive inference.arXiv preprint arXiv:2205.03647,

    Michael Bian and Rina Foygel Barber. Training-conditional coverage for distribution-free predictive inference.arXiv preprint arXiv:2205.03647,

    Pith/arXiv arXiv

  12. [2000]

    Small sample beta correction for conformal prediction.arXiv preprint arXiv:2503.04894,

    Ricardo Zwart. Small sample beta correction for conformal prediction.arXiv preprint arXiv:2503.04894,

    arXiv

  13. [2017]

    MMDEW: Maximum mean discrepancy on exponential windows for online change detection.arXiv preprint arXiv:2205.12706,

    Florian Kalinke and Zolt´an Szab´o. MMDEW: Maximum mean discrepancy on exponential windows for online change detection.arXiv preprint arXiv:2205.12706,

    Pith/arXiv arXiv

  14. [2019]

    Conformal prediction with large language models for multi-choice question answering.arXiv preprint arXiv:2305.18404,

    Bhawesh Kumar, Charlie Lu, Gauri Gupta, et al. Conformal prediction with large language models for multi-choice question answering.arXiv preprint arXiv:2305.18404,

    Pith/arXiv arXiv

  15. [2020]

    Amazon SageMaker model monitor.arXiv preprint arXiv:2111.13657,

    Amazon Web Services. Amazon SageMaker model monitor.arXiv preprint arXiv:2111.13657,

    Pith/arXiv arXiv

  16. [2023]

    Yoav Benjamini and Yosef Hochberg

    arXiv:2107.07511. Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing.Journal of the Royal Statistical Society: Series B, 57(1):289–300,

    Pith/arXiv arXiv

  17. [2024]

    Gianluca Detommaso, Alberto Gasparin, Michele Forber, et al

    arXiv:2406.06955. Gianluca Detommaso, Alberto Gasparin, Michele Forber, et al. Fortuna: A library for uncertainty quantification.arXiv preprint arXiv:2302.04019,

    Pith/arXiv arXiv

  18. [2025]

    Dominik Kreuzberger, Niklas K ¨uhl, and Sebastian Hirschl

    arXiv:2506.20657. Dominik Kreuzberger, Niklas K ¨uhl, and Sebastian Hirschl. MLOps: Overview, definition, and architecture.IEEE Access,

    Pith/arXiv arXiv