Ensuring Reliability via Hyperparameter Selection: Review and Advances

Amirmohammad Farzaneh; Osvaldo Simeone

REVIEW 3 major objections 3 minor 32 references

Ensuring Reliability via Hyperparameter Selection: Review and Advances

T0 review · 3 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Statistical testing turns hyperparameter selection into a reliability-certified procedure.

desk verdict A useful but flawed survey of LTT-based hyperparameter selection whose central null hypothesis is defined backwards; with the sign error fixed it would be a solid tutorial, but as written the guarantees do not follow. read the letter →

arxiv 2502.04206 v1 pith:7MQ4DLT4 submitted 2025-02-06 cs.LG cs.ITmath.IT

Amirmohammad Farzaneh , Osvaldo Simeone This is my paper · ORCID

classification cs.LGcs.ITmath.IT

keywords hyperparameterselectionLearn-Then-Testmultiplehypothesistestingfamily-wiseerrorratefalsediscoverye-valuesriskcontrolreliability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hyperparameter selection is usually a heuristic search, but this paper reviews a statistical reformulation: treat each candidate hyperparameter as a null hypothesis to be tested and return only the configurations that pass a multiple-testing filter. The Learn-Then-Test (LTT) framework promises that the returned set contains no unreliable configuration with probability at least $1-\delta$ (family-wise error control), or only a controlled fraction of unreliable ones (false discovery rate control). The paper shows how this guarantee extends to quantile risks, mutual information risks, multi-objective constraints, prior-knowledge graphs, and adaptive testing with e-values. If right, practitioners can certify hyperparameters for safety-critical deployments rather than rely on empirical validation alone.

What carries the argument

The central object is the multiple hypothesis testing structure: each hyperparameter $\lambda$ is attached to a null hypothesis $H_\lambda : R(\lambda) < \alpha$, and a valid $p$-value (e.g., the Hoeffding bound in (3) for average risk) or $e$-value (e.g., the product betting score in (13)) is computed from held-out data. A multiple-testing algorithm then decides which nulls to reject, yielding a subset $\hat\Lambda_Z$ with a family-wise error rate bound like (6) or a false discovery rate bound like (9). The extensions reviewed—quantile and mutual-information risk $p$-values, Pareto testing, reliability graphs with DAGGER, and adaptive $e$-processes—all plug into this same testing pipeline, replacing the risk measure or the multiple-testing control.

What would settle it

Feed an unbounded risk function into the Hoeffding-based p-value (3) on i.i.d. data and count the fraction of selected configurations whose true risk exceeds $\alpha$; if the empirical family-wise error exceeds $\delta$ at the nominal level, the tested bound is invalid and the guarantee collapses.

Watch

Extended reading notes

Core claim

Learn-Then-Test (LTT) reframes hyperparameter selection as a multiple hypothesis testing problem: each candidate $\lambda$ is assigned a null hypothesis $H_\lambda : R(\lambda) < \alpha$ and a valid $p$-value or $e$-value computed from a held-out calibration set $Z$, and a multiple-testing algorithm rejects the nulls that deem hyperparameters unreliable, yielding a subset $\hat\Lambda_Z$ guaranteed by (6) to exclude every unreliable configuration with probability at least $1-\delta$. The paper reviews how this same pipeline extends to quantile risks and mutual-information risks (via dedicated $p$-values), to Pareto-front pruning for multi-objective constraints, to reliability graphs that inject prior knowledge with FDR control, and to adaptive e-value-based testing that can stop early without losing anytime-valid guarantees. The review indicates that the whole framework achieves its goal by ensuring that the selected subset $\hat\Lambda_Z$ satisfies the FWER bound (6) or its FDR analogue (9), regardless of the risk measure plugged in.

Load-bearing premise

The framework's guarantees are only as strong as the validity of the p-value or e-value bounds computed from i.i.d. held-out data, so any risk measure without a valid bound—or any calibration set that is not i.i.d.—would void the claimed family-wise or false-discovery control.

Editorial extensions

If this is right

If the LTT reformulation is correct, hyperparameter tuning can move from picking the best-looking validation score to certifying that the chosen configuration meets a risk target with probability at least $1-\delta$.
The extension to quantile and mutual-information risks means safety-performance metrics like 90th-percentile latency or information-theoretic sufficiency can be controlled with the same guarantees, not just averages.
Multi-objective Pareto testing lets an engineer search over trade-offs while still guaranteeing that no selected configuration violates any constraint.
Reliability graphs incorporate prior knowledge (e.g., energy-use ordering in telecom) and enable FDR control, reducing conservatism compared to Bonferroni correction.
Adaptive e-value testing allows data to be collected sequentially and stopping when evidence is sufficient, cutting evaluation cost without losing anytime-valid guarantees.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the authors only gesture at: applying the same testing pipeline to contextual or instance-dependent risk functions (they mention contextual MHT as future work) could certify reliability for personalized models.
The validity of the whole scheme hinges on the calibration data being i.i.d.; in non-stationary deployment, the guarantee may degrade—a testable extension would be to use distribution-free or anytime-valid bounds for drifting data.
The review's confidence-interval duality (Eq. 8) suggests LTT-style control could be composed with conformal prediction to deliver end-to-end reliability guarantees for predictive systems.
One could empirically compare the conservativeness of FWER versus FDR variants on the same candidate set to decide which guarantee is economically appropriate for a given deployment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a review/tutorial, not a new research contribution. There are no new theorems, algorithms, or experiments. What it does well is organize the growing LTT literature into a single picture: the core framework from Angelopoulos et al., plus extensions to quantile risk, mutual information, multi-objective control, reliability graphs, and adaptive e-value testing. The comparison table in Section I is genuinely helpful, and the expository structure makes it easy to see how the extensions relate. A practitioner looking for a map of this niche would get value from the paper.

The problem is in the formal core. Section II defines the null hypothesis as H_λ: R(λ) < α, calling that state unreliable. But then the p-value in Eq. (3) is derived from Hoeffding and is small when the empirical risk is below α, which is only valid if the null is R(λ) ≥ α. The guarantee in Eq. (6) also concerns R(λ) ≤ α, and the aLTT section in Eq. (13) explicitly uses H: R(λ) > α. So the paper contradicts itself about what the null is. This is not cosmetic. A reader following Eq. (1) literally would compute p-values that are valid for the opposite hypothesis, and the advertised FWER/FDR control would not follow. The stress-test note is right: this is a load-bearing internal inconsistency.

The soft spots beyond that are milder. The title says “Advances,” but the paper is a review and the extensions are mostly the authors’ own prior work, described without independent reproduction. That is normal for a tutorial, but it does limit critical distance. Figure 2 uses a different calibration data set without reporting it, and there is no code or data release. None of these would sink a well-formed survey, but they matter given the paper’s purpose.

Who is this for? Someone who wants a compact overview of LTT and its variants, and who is willing to check the original papers before relying on the formulas. As written, I would not trust the equations without fixing the null definition first. A corrected version would deserve a serious referee; the current one does too, but with a clear request for major revision.

Recommendation: engage with it, but only after asking the authors to fix the null hypothesis in Eq. (1) and to reconcile the strict/non-strict thresholds. It should not be accepted as is.

Referee Report

3 major / 3 minor

Summary. This tutorial-style paper reviews the Learn-Then-Test (LTT) framework for hyperparameter selection, in which candidate hyperparameters are cast as multiple hypothesis tests and p-values or e-values computed on a held-out calibration set are fed into a multiple-hypothesis-testing procedure. The paper claims that the output subset of hyperparameters comes with formal FWER or FDR guarantees on population risk measures. It surveys extensions to quantile risk, mutual-information risk, multi-objective Pareto testing, side information encoded as reliability graphs, and adaptive e-value-based testing, and it illustrates the methods with communication-system applications. The central claim is that practitioners can certify hyperparameter choices with statistical reliability guarantees rather than relying only on empirical validation.

Significance. If the formal inconsistencies identified below are corrected, this would be a useful compact introduction to an emerging area, with a clear tabular comparison and concrete engineering examples. The paper's strengths are its organizing Table I, the reproduction of the core LTT equations, and the connection to wireless-system use cases. However, the central formal guarantee is not currently supported by the paper's own definitions: the null hypothesis in Eq. (1) has the opposite direction from the p-value it is paired with and from the guarantees stated in Eqs. (6) and (9). Because this sign error propagates to the reviewed extensions, the manuscript needs substantial revision before its main claim can be accepted. The tutorial value is real, but the load-bearing formalism must be made internally consistent.

major comments (3)

[Section II, Eq. (1)] The null hypothesis is defined as Hλ: R(λ) < α and is called "unreliable". This is the reverse of the standard LTT convention and is inconsistent with the paper's own subsequent equations. Under the stated null, rejecting Hλ should be interpreted as evidence that R(λ) ≥ α, i.e., that λ is unreliable, but the text says rejecting means deeming λ reliable. Furthermore, the p-value in Eq. (3) is small when the empirical risk R̂(λ|Z) is below α; for the null Hλ: R(λ) ≥ α, Hoeffding's inequality gives P_{H0}(pλ ≤ u) ≤ u, so Eq. (3) is a valid p-value only for that null, not for Hλ: R(λ) < α. As a result, the FWER/FDR guarantees in Section IV do not follow from the stated testing procedure. This is the load-bearing formal claim of the paper and must be corrected, e.g., by defining Hλ: R(λ) ≥ α, and propagated consistently throughout.
[Section IV, Eqs. (6) and (9)] Even after fixing Eq. (1), the threshold conventions are inconsistent. Eq. (6) states Pr(R(λ) ≤ α for all λ ∈ Λ̂Z) ≥ 1 − δ, while Eq. (9) defines an unreliable hyperparameter as one with R(λ) > α, and Eq. (13) tests Hλi: R(λi) > α. The paper should adopt a single convention for "reliable" (e.g., R(λ) ≤ α) and use it consistently in Eq. (1), Eq. (6), Eq. (9), and Eq. (13). With the current mixture of strict and non-strict inequalities, the claimed control of "no unreliable configurations" is ambiguous.
[Section III, QLTT and IB-MHT] The paper asserts that the works cited as [2] and [3] provide valid p-values for quantile and mutual-information risks, but it does not state the direction of the null hypothesis for these p-values or the validity inequality that they satisfy. Because the same sign error in Eq. (1) propagates to these extensions, the reader cannot verify that the FWER guarantee holds for QLTT and IB-MHT. Please include, at least for each risk measure, the null being tested and the p-value validity condition (P_{H0}(p ≤ u) ≤ u), or explicitly state that this is inherited from the cited works.

minor comments (3)

[Section II, Eq. (3)] The notation in Eq. (3) is inconsistent: the exponent uses |Z| in the second occurrence of the equation in the manuscript but |Ẑ| in the first occurrence, while the data set is defined as Z. Please make the notation uniform.
[Figures 1 and 4] The block diagram for LTT-based hyperparameter selection appears twice, once as Fig. 1 and once as Fig. 4, with identical content. One of the two figures should be removed.
[Section III, Eq. (5)] The sentence following Eq. (5), "controlling the q-quantile risk ... guarantees that the worst-case risk among the top-q fraction of test instances is no smaller than α," is confusing and appears to describe the complementary fraction. Please rephrase it in terms of the fraction 1 − q of instances whose risk lies below α.

Circularity Check

1 steps flagged · score 5.0 of 10

Eq. (1) prints the null as R(λ) < α, but the adjoining gloss, the Hoeffding p-value in Eq. (3), and the guarantees in Eqs. (6)/(9)/(13) only cohere with the opposite event R(λ) ≥ α; as written, 'reliable' and 'unreliable' coincide by definition.

self definitional [Section II, Eq. (1) and its gloss; Section III, Eq. (3); Section IV, Eq. (6); Section VII, Eq. (13)]
"Formally, the null hypothesis Hλ is defined as Hλ : R(λ) < α, (1)... the guarantee that it contains no, or few, unreliable hyperparameters λ violating the condition (1), i.e., R(λ) ≥ α... a valid p-value for the null hypothesis Hλ in (1) can be calculated by using Hoeffding's inequality [1] p avg λ (Z) = e^{−2|Z|(α− ˆR avg (λ|Z))^2_+} , (3)"

Eq. (1) defines the null 'λ is unreliable' as R(λ) < α, while two sentences later 'unreliable' is defined as 'violating the condition (1), i.e., R(λ) ≥ α' — the event and its complement — so the headline guarantee 'contains no unreliable hyperparameters' is settled by definition rather than by the MHT machinery. The Hoeffding p-value in (3) is super-uniform only under the null R(λ) ≥ α; under R(λ) = 0 < α it rejects with probability close to one (P(p ≤ u) ≈ 1 > u), so it is not valid for the null stated in (1). Eq. (6) then promises R(λ) ≤ α, a third event, while Eq. (9) and Eq. (13) switch back to R(λ) > α. No consistent reading of (1), (3), (6) supports the advertised FWER/FDR guarantee; the formal claim rests on the definitional flip-flop, not on a derivation.

full rationale

This paper is a review: the core LTT framework is externally sourced from [1] (Angelopoulos et al.), the extensions (QLTT [2], IB-MHT [3], RG-PT [5], aLTT [6]) are the authors' own prior work, and the paper contains no fitted parameters, no empirical predictions, and no uniqueness claims, so none of the fit-based circularity patterns apply. Heavy self-citation is the normal structure of a survey written by the method's developers; the validity of the quantile and mutual-information p-values is forwarded from [2] and [3] without restating their assumptions, which is a completeness and correctness risk for the Section IV guarantees but not circularity, since the cited works carry published or arXiv proofs. However, the formal derivation chain of the central claim is broken internally. Eq. (1) states the null as Hλ: R(λ) < α, yet the same paragraph defines 'unreliable' as violating (1), i.e., R(λ) ≥ α; rejection of (1) is said to mean 'reliable', so by the paper's own definitions the reliable and unreliable events coincide. The Hoeffding p-value in Eq. (3) is super-uniform only under the null R(λ) ≥ α, so it is not valid for the null as printed. Eq. (6) states the FWER guarantee as R(λ) ≤ α, while Eq. (9) and Eq. (13) use R(λ) > α. No consistent reading of Eqs. (1), (3), and (6) yields the advertised guarantee; the claim is an artifact of the definitions rather than a derivation. The fix is to state Hλ: R(λ) ≥ α and keep strictness consistent; the intended framework is real, external, and sound, which is why this is scored as a partial, definition-level circularity rather than a complete collapse.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper contributes no fitted parameters and no new entities. Its claims rest on inherited statistical validity results: i.i.d. calibration data, valid p-values or e-values per risk measure, and standard multiple-testing error control. These are reasonable domain assumptions for the cited methods, but they are assumptions, not established inside this paper.

assumptions (3)

domain assumption Calibration data Z are i.i.d. from the true unknown distribution PZ.
Stated in Section II as a required input; validity of Hoeffding bounds and p-values depends on this.
domain assumption For each risk measure, a valid p-value or e-value can be computed (Hoeffding for average risk, specialized bounds for quantile and mutual information risks).
The paper does not prove validity; it cites [2] and [3]. The entire LTT mechanism rests on this assumption.
standard math The multiple hypothesis testing algorithms (Bonferroni, FST, DAGGER) control their stated error rates, FWER or FDR.
Standard results from [19], [24], [26], and [31] are used without proof.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Ensuring Reliability via Hyperparameter Selection: Review and Advances." pith.science (2026). https://pith.science/paper/7MQ4DLT4

@misc{pith2026250204206,
  author       = {Pith},
  title        = {Pith review of: Ensuring Reliability via Hyperparameter Selection: Review and Advances},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MQ4DLT4}},
  note         = {Machine review of arXiv:2502.04206}
}

read the original abstract

Hyperparameter selection is a critical step in the deployment of artificial intelligence (AI) models, particularly in the current era of foundational, pre-trained, models. By framing hyperparameter selection as a multiple hypothesis testing problem, recent research has shown that it is possible to provide statistical guarantees on population risk measures attained by the selected hyperparameter. This paper reviews the Learn-Then-Test (LTT) framework, which formalizes this approach, and explores several extensions tailored to engineering-relevant scenarios. These extensions encompass different risk measures and statistical guarantees, multi-objective optimization, the incorporation of prior knowledge and dependency structures into the hyperparameter selection process, as well as adaptivity. The paper also includes illustrative applications for communication systems.

Figures

Figures reproduced from arXiv: 2502.04206 by the authors.

**Figure 4.** Block diagram for LTT-based hyperparameter selection methods. Fig. 1: Block diagram for LTT-based hyperparameter selection methods. [PITH_FULL_IMAGE:figures/full_fig_p001_4.png] view at source ↗

**Figure 2.** Packet delay distribution for LTT and QLTT in a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** An example of prior information on the relative [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Comparison of (a) Bonferroni method, (b) FST, and [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 23 canonical work pages

[2]

Quantile learn-then-test: Quantile-based risk control for hyperparameter optimization,

A. Farzaneh, S. Park, and O. Simeone, “Quantile learn-then-test: Quantile-based risk control for hyperparameter optimization,” IEEE Signal Processing Letters , 2024

work page 2024
[3]

Statistically Valid Information Bottleneck via Multiple Hypothesis Testing

A. Farzaneh and O. Simeone, “Statistically valid information bottleneck via multiple hypothesis testing,” arXiv preprint arXiv:2409.07325, 2024

work page Pith review arXiv 2024
[1]

Learn then test: Calibrating predictive algorithms to achieve risk control,

A. N. Angelopoulos, S. Bates, E. J. Cand `es, M. I. Jordan, and L. Lei, “Learn then test: Calibrating predictive algorithms to achieve risk control,” arXiv preprint arXiv:2110.01052 , 2021

arXiv 2021
[4]

Efficiently controlling multiple risks with Pareto testing,

B. Laufer-Goldshtein, A. Fisch, R. Barzilay, and T. S. Jaakkola, “Efficiently controlling multiple risks with Pareto testing,” in Proc. International Conference on Learning Representations , 2023

work page 2023
[5]

Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs

A. Farzaneh and O. Simeone, “Multi-objective hyperparameter se- lection via hypothesis testing on reliability graphs,” arXiv preprint arXiv:2501.13018, 2025

work page Pith review arXiv 2025
[6]

Adaptive learn-then-test: Statisti- cally valid and efficient hyperparameter selection,

M. Zecchin and O. Simeone, “Adaptive learn-then-test: Statisti- cally valid and efficient hyperparameter selection,” arXiv preprint arXiv:2409.15844, 2024

arXiv 2024
[7]

On hyperparameter optimization of machine learning algorithms: Theory and practice,

L. Yang and A. Shami, “On hyperparameter optimization of machine learning algorithms: Theory and practice,” Neurocomputing, vol. 415, pp. 295–316, 2020

2020
[8]

Random search for hyper-parameter opti- mization.,

J. Bergstra and Y . Bengio, “Random search for hyper-parameter opti- mization.,” Journal of machine learning research , vol. 13, no. 2, 2012

work page 2012

Show all 32 references

[9]

Practical bayesian optimiza- tion of machine learning algorithms,

J. Snoek, H. Larochelle, and R. P. Adams, “Practical bayesian optimiza- tion of machine learning algorithms,” Advances in neural information processing systems, vol. 25, 2012

2012
[10]

Achieving the promise of artificial intelligence in health and medicine: Building a foundation for the future,

V . J. Dzau, M. H. Laitner, A. Temple, and T. H. Nguyen, “Achieving the promise of artificial intelligence in health and medicine: Building a foundation for the future,” 2023

2023
[11]

What, indeed, is an achievable provable guarantee for learning-enabled safety-critical systems,

S. Bensalem, C.-H. Cheng, W. Huang, X. Huang, C. Wu, and X. Zhao, “What, indeed, is an achievable provable guarantee for learning-enabled safety-critical systems,” in International Conference on Bridging the Gap between AI and Reality , pp. 55–76, Springer, 2023

2023
[12]

Confident adaptive language modeling,

T. Schuster, A. Fisch, J. Gupta, M. Dehghani, D. Bahri, V . Tran, Y . Tay, and D. Metzler, “Confident adaptive language modeling,” Advances in Neural Information Processing Systems, vol. 35, pp. 17456–17472, 2022

2022
[13]

Image-to-image regres- sion with distribution-free uncertainty quantification and applications in imaging,

A. N. Angelopoulos, A. P. Kohli, S. Bates, M. Jordan, J. Malik, T. Alshaabi, S. Upadhyayula, and Y . Romano, “Image-to-image regres- sion with distribution-free uncertainty quantification and applications in imaging,” in International Conference on Machine Learning , pp. 717– ...

2022
[14]

Learning to broadcast for ultra-reliable communication with differential quality of service via the conditional value at risk,

R. Karasik, O. Simeone, H. Jang, and S. S. Shitz, “Learning to broadcast for ultra-reliable communication with differential quality of service via the conditional value at risk,” IEEE Transactions on Communications , vol. 70, no. 12, pp. 8060–8074, 2022

2022
[15]

Multi-objective optimization for power efficient full-duplex wireless communication systems,

Y . Sun, D. W. K. Ng, and R. Schober, “Multi-objective optimization for power efficient full-duplex wireless communication systems,” in 2015 IEEE Global Communications Conference (GLOBECOM) , pp. 1– 6, IEEE, 2015

2015
[16]

Adaptive and flexible model-based ai for deep receivers in dynamic channels,

T. Raviv, S. Park, O. Simeone, Y . C. Eldar, and N. Shlezinger, “Adaptive and flexible model-based ai for deep receivers in dynamic channels,” IEEE Wireless Communications, 2024

2024
[17]

What if we had used a different app? reliable counterfactual kpi analysis in wireless systems,

Q. Hou, S. Park, M. Zecchin, Y . Cai, G. Yu, and O. Simeone, “What if we had used a different app? reliable counterfactual kpi analysis in wireless systems,” IEEE Transactions on Cognitive Communications and Networking, 2025

2025
[18]

Hypothesis testing with e-values,

A. Ramdas and R. Wang, “Hypothesis testing with e-values,” arXiv preprint arXiv:2410.23614, 2024

2024 arXiv
[19]

J. A. Rice, Mathematical Statistics and Data Analysis. Belmont, CA: Duxbury Press., third ed., 2006

2006
[20]

A review of uncertainty quantification in deep learning: Techniques, applications and challenges,

M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya,et al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information fusion, vol. 76, pp. 243–297, 2021

2021
[21]

An information-theoretic approach to generalization theory,

B. Rodr ´ıguez-G´alvez, R. Thobaben, and M. Skoglund, “An information-theoretic approach to generalization theory,” arXiv preprint arXiv:2408.13275, 2024

2024 arXiv
[22]

Wireless Suite: A collection of problems in wireless telecommunications,

A. Valcarce, “Wireless Suite: A collection of problems in wireless telecommunications,” https://github.com/nokia/wireless-suite, 2020

2020
[23]

The information bottleneck method,

N. Tishby, F. Pereira, and W. Bialek, “The information bottleneck method,” in Proc. Allerton Conference on Communication, Control and Computation, 2001

2001
[24]

Multiple testing in clinical trials,

P. Bauer, “Multiple testing in clinical trials,” Statistics in Medicine , vol. 10, no. 6, pp. 871–890, 1991

1991
[25]

Semi-supervised risk con- trol via prediction-powered inference,

B.-S. Einbinder, L. Ringel, and Y . Romano, “Semi-supervised risk con- trol via prediction-powered inference,”arXiv preprint arXiv:2412.11174, 2024

2024 arXiv
[26]

Controlling the false discovery rate: a practical and powerful approach to multiple testing,

Y . Benjamini and Y . Hochberg, “Controlling the false discovery rate: a practical and powerful approach to multiple testing,” Journal of the Royal statistical society: series B (Methodological) , vol. 57, no. 1, pp. 289–300, 1995

1995
[27]

AI-Enabled Radio Resource Alloca- tion in 5G for URLLC and eMBB Users,

M. Elsayed and M. Erol-Kantarci, “AI-Enabled Radio Resource Alloca- tion in 5G for URLLC and eMBB Users,” in 2019 IEEE 2nd 5G World Forum (5GWF), pp. 590–595, 2019

2019
[28]

Dynamic resource allocation schemes for embb and urllc services in 5g wireless networks,

X. Han, K. Xiao, R. Liu, X. Liu, G. C. Alexandropoulos, and S. Jin, “Dynamic resource allocation schemes for embb and urllc services in 5g wireless networks,” Intelligent and Converged Networks , vol. 3, no. 2, pp. 145–160, 2022

2022
[29]

Intelligent energy saving solution of 5g base station based on artificial intelligence technologies,

T. Rumeng, W. Tong, S. Ying, and H. Yanpu, “Intelligent energy saving solution of 5g base station based on artificial intelligence technologies,” in 2021 IEEE International Joint EMC/SI/PI and EMC Europe Sympo- sium, pp. 739–742, 2021

2021
[30]

C. M. Bishop and N. M. Nasrabadi, Pattern recognition and machine learning, vol. 4. Springer, 2006

2006
[31]

A sequential algorithm for false discovery rate control on directed acyclic graphs,

A. Ramdas, J. Chen, M. J. Wainwright, and M. I. Jordan, “A sequential algorithm for false discovery rate control on directed acyclic graphs,” Biometrika, vol. 106, no. 1, pp. 69–86, 2019

2019
[32]

Online multiple testing with e-values,

Z. Xu and A. Ramdas, “Online multiple testing with e-values,” in Inter- national Conference on Artificial Intelligence and Statistics , pp. 3997– 4005, PMLR, 2024

2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [2]

Quantile learn-then-test: Quantile-based risk control for hyperparameter optimization,

A. Farzaneh, S. Park, and O. Simeone, “Quantile learn-then-test: Quantile-based risk control for hyperparameter optimization,” IEEE Signal Processing Letters , 2024

work page 2024

[2] [3]

Statistically Valid Information Bottleneck via Multiple Hypothesis Testing

A. Farzaneh and O. Simeone, “Statistically valid information bottleneck via multiple hypothesis testing,” arXiv preprint arXiv:2409.07325, 2024

work page Pith review arXiv 2024

[3] [1]

Learn then test: Calibrating predictive algorithms to achieve risk control,

A. N. Angelopoulos, S. Bates, E. J. Cand `es, M. I. Jordan, and L. Lei, “Learn then test: Calibrating predictive algorithms to achieve risk control,” arXiv preprint arXiv:2110.01052 , 2021

arXiv 2021

[4] [4]

Efficiently controlling multiple risks with Pareto testing,

B. Laufer-Goldshtein, A. Fisch, R. Barzilay, and T. S. Jaakkola, “Efficiently controlling multiple risks with Pareto testing,” in Proc. International Conference on Learning Representations , 2023

work page 2023

[5] [5]

Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs

A. Farzaneh and O. Simeone, “Multi-objective hyperparameter se- lection via hypothesis testing on reliability graphs,” arXiv preprint arXiv:2501.13018, 2025

work page Pith review arXiv 2025

[6] [6]

Adaptive learn-then-test: Statisti- cally valid and efficient hyperparameter selection,

M. Zecchin and O. Simeone, “Adaptive learn-then-test: Statisti- cally valid and efficient hyperparameter selection,” arXiv preprint arXiv:2409.15844, 2024

arXiv 2024

[7] [7]

On hyperparameter optimization of machine learning algorithms: Theory and practice,

L. Yang and A. Shami, “On hyperparameter optimization of machine learning algorithms: Theory and practice,” Neurocomputing, vol. 415, pp. 295–316, 2020

2020

[8] [8]

Random search for hyper-parameter opti- mization.,

J. Bergstra and Y . Bengio, “Random search for hyper-parameter opti- mization.,” Journal of machine learning research , vol. 13, no. 2, 2012

work page 2012