How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification

Julia Machnio; Mads Nielsen; Mostafa Mehdipour Ghazi

REVIEW 4 major objections 4 minor 21 references

How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ALDA predicts the deployment-optimal active-learning method from a 15–30% pilot of the annotation budget and can cut label costs by up to 82% relative to a poor strategy choice.

desk verdict ALDA is a sensible decision layer, but the evaluation compares curve fits to curve fits; the real test against observed label costs is missing. read the letter →

arxiv 2608.03511 v1 pith:56KUI7QF submitted 2026-08-04 cs.CV cs.AIcs.LG

Julia Machnio , Mads Nielsen , Mostafa Mehdipour Ghazi This is my paper

classification cs.CVcs.AIcs.LG

keywords ActivelearningMedicalimageclassificationAnnotationcostpredictionLearning-curveextrapolationDeploymentwindowThresholdsensitivityPilot-basedmethodselectionALDA

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ALDA reframes active-learning (AL) method choice in medical imaging as a prospective deployment decision: before the full annotation budget is spent, which sampling strategy will reach the clinician's accuracy target with the fewest expert labels? The paper's central claim is that a parametric learning curve fitted to the first 15–30% of each candidate strategy's trajectory lets ALDA screen out methods that cannot reach the target, predict the absolute number of labels each feasible strategy needs, and recommend the most threshold-stable option among near-cheapest strategies. If correct, a clinical team can commit to an AL strategy from a short pilot instead of discovering after full annotation that the chosen method was expensive. Across brain MRI, skin lesions, fetal ultrasound, and breast ultrasound, ALDA's pick matches the full-trajectory oracle from roughly 20% of the budget onward and saves up to 82% of labels relative to the worst feasible choice. The paper contributes a decision layer, not a new sampling heuristic.

What carries the argument

The load-bearing machinery is the parametric AL learning curve of Eq. (1), with parameters A_max (predicted accuracy ceiling), δ (per-episode gain), α (seed-set shift), and β (diminishing-returns exponent). Fitting this curve to a 15–30% pilot lets ALDA invert it in closed form (Eq. 2) to turn the clinical target τ into a predicted label budget B_abs(τ). The deployment window W (Eq. 3) then measures how that budget changes under threshold revisions, and the cost-tolerance set C_η (Eq. 5) defines which methods count as cost-competitive; the final pick minimizes W within C_η. The framework converts a trajectory forecast into an actionable, risk-aware choice.

What would settle it

Run the full AL pipeline on a held-out medical dataset, record for each method the actual first episode where test accuracy reaches τ, and compare with ALDA's predicted B_abs from a 30% pilot; the central claim is wrong if on a substantial share of methods the predicted and observed label counts diverge by more than one acquisition batch, or if a method predicted feasible never reaches τ.

Watch

Extended reading notes

Core claim

On the paper's own terms, ALDA's central result is that method selection in active learning can be made before the full budget is spent. It adopts a four-parameter learning curve with an asymptotic ceiling, fits the curve to each candidate method's pilot trajectory, and rejects as infeasible any method whose estimated ceiling falls below the clinical threshold. For feasible methods it inverts the fitted curve to obtain B_abs(τ), the predicted number of expert annotations (rounded up to the next acquisition batch) needed to reach τ, and defines the deployment window W as the change in B_abs when the threshold is raised or lowered by Δτ. The recommendation is the feasible method with smallest

Load-bearing premise

The parametric learning curve fitted to just 15–30% of a method's trajectory extrapolates accurately to the full trajectory for every candidate method and dataset, so the predicted label counts B_abs and deployment windows W are trustworthy; the paper never checks these predictions against the observed episode where accuracy first crosses the target, and the oracle it compares with is itself a full-trajectory curve fit.

Editorial extensions

If this is right

A clinical team can run a pilot with 15–30% of the intended budget, apply ALDA, and then spend the remaining budget on the recommended AL strategy, avoiding the cost of discovering a bad choice after full annotation.
Methods predicted never to reach the target are flagged before the budget is committed, preventing wasted labeling on infeasible strategies.
The cheapest feasible method is usually also the most threshold-stable on these datasets, so in practice optimizing for B_abs tends to minimize sensitivity to target revisions.
ALDA's recommendation is stable across window widths and across nearby thresholds, but it does shift with target level, reflecting genuine differences in AL learning dynamics rather than selection instability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors do not check B_abs predictions against the actually observed first episode where accuracy crosses τ; a direct test of their predicted label counts on held-out runs would reveal whether the parametric curve's extrapolation holds, especially for methods with late gains.
Because ALDA is agnostic to the acquisition function it selects over, the same pilot-plus-inversion decision layer could transfer to other costly-annotation settings (e.g., document or satellite-image classification) where target thresholds are specified in advance.
The paper fixes the cost tolerance η at 5% and does not test its sensitivity; teams with rigid label budgets could re-run selection with smaller η, which may change the recommendation when the cheapest and near-cheapest methods have very different W.
The observed correlation between low B_abs and low W suggests that, on these datasets, choosing for cost also chooses for stability; if that correlation fails on a new domain, the tiebreak will matter more and selecting on W alone could reject a clearly cheaper method.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: ALDA answers a real deployment question — which AL strategy to commit to before you spend the full labeling budget — and the decision layer (feasibility, absolute cost, deployment window, risk-aware tiebreak) is genuinely new relative to the PALM curve it builds on. But the evaluation does not, as of now, demonstrate that the predicted costs are correct. The pilot-based estimates are compared against a full-trajectory fit of the same parametric curve, not against the observed label count where accuracy first crosses the target. That makes the headline savings a fit-to-fit result, and the 14-percentage-point Amax miss for ProbCover on ISIC2019 (Table 2) shows the curve can be badly off for at least one method, which would flip the feasibility screen.

On the plus side: the paper is clearly written, the protocol is reproducible (code released), and the authors are careful to distinguish ALDA from a new sampling heuristic. The risk-aware selection rule — prefer the narrowest deployment window among near-optimal costs — is a sensible way to handle threshold uncertainty, and the sensitivity analyses in the appendix for window width and threshold level are useful.

Where it falls short: First, the central metric B_abs is an inversion of the fitted curve (Eq. 2), and the oracle in Sec. 4.3 is the same kind of fit on full data. You need to check B_abs against the actual first crossing episode. The ProbCover/ISIC cell suggests this check would fail at least sometimes. Second, B_abs and W are reported without error bars; three seeds are averaged but we never see the spread. Third, the 'up to 82%' savings is versus the most expensive feasible method; against Random, where the comparison is reported, savings are 21–59%, still meaningful but a different headline. Fourth, the claim that the pilot phase has 'no extra labeling cost' needs a careful defense: each method selects different samples, so to get method-specific trajectories you either need to label the union of their pilot selections (which is likely more than 30% of the budget) or you need to assume the orderings do not affect accuracy, which is false.

The tau thresholds are heuristic per dataset, though the appendix shows the recommendations are often stable across a range. That's a minor point.

Who should read it: anyone working on AL evaluation or learning-curve extrapolation. The framework is a good candidate for the AL selection toolbox, but the empirical claims need to be redone against observed crossings and with reasonable baselines (always-Random, and maybe a nonparametric curve). I'd send it to peer review — the idea deserves referee time — but I wouldn't accept it as is.

Referee Report

4 major / 4 minor

Summary. The paper proposes ALDA, a decision layer on top of existing active-learning sampling strategies. It fits the PALM parametric learning curve (Eq. 1) to a short pilot prefix of each candidate strategy, screens strategies as infeasible when the fitted ceiling is below a clinical threshold τ, computes the absolute annotation cost B_abs (Eq. 2) and a deployment window W (Eq. 3), and recommends the strategy minimizing W among those within a cost non-inferiority band (Algorithm 1). On four medical imaging datasets, the authors report that 30% pilot fits identify the same method as a full-trajectory oracle and that following ALDA reduces annotation cost by up to 82% relative to the most expensive feasible alternative.

Significance. The framework addresses a real and under-served deployment question: choosing an AL strategy before the full annotation budget is spent. The deployment-window concept is a sensible addition, and the paper is clearly written with code made available. If the pilot extrapolation were shown to be reliable, ALDA would be a practical contribution. However, the current experiments validate the pipeline against another curve fit rather than against observed annotation costs, and several load-bearing quantities are reported without uncertainty. The central empirical claim is therefore not yet demonstrated.

major comments (4)

[Section 4.3, Figure 2] The evaluation is fit-to-fit. The oracle is defined as the method minimizing B_abs computed from full-trajectory PALM fits, not the observed number of labels at which accuracy first reaches τ. Therefore the claimed savings in the Abstract and Figure 1(B) are not savings measured in real labels; they are consistency between two fits of the same parametric curve. The authors should report the observed crossing point B_obs = min{B : observed test accuracy ≥ τ} for every method/dataset and compare predicted B_abs, the ALDA selection, and the savings against this quantity.
[Table 2, ISIC2019/ProbCover row] The table contains direct evidence that the PALM curve can severely underestimate the ceiling from a 30% pilot: A_ach = 68.7% while the pilot-fitted A_max = 54.7%, a 14 pp gap. For any τ between 54.7 and 68.7, ALDA would falsely screen out a method that actually reaches the target. This is a load-bearing failure of the feasibility screen and B_abs inversion. The paper should systematically compare pilot-fitted A_max with observed final accuracy across all cells, not only where it happens to support the authors' screening decision.
[Table 2 / Section 4.1] B_abs and W are presented as point estimates with no uncertainty quantification, even though each method–dataset pair is run over three seeds (Section 4.1). The decision rule in Eq. (6) can switch on small differences, e.g., BUSI Entropy (209) vs. Margin (208), and the tie-break depends on W, whose variance is not reported. The authors should provide standard errors or confidence intervals for B_abs and W, or at least per-seed values.
[Section 5, first paragraph] The paper claims that ALDA's pilot phase 'only requires annotating a pilot subset of 20–30% of the intended budget' and that each candidate method induces an ordering over the pilot samples 'at no extra labeling cost.' The experiments, however, fit to prefixes of full individually-run trajectories (Section 4.1), not to a single shared pilot subset. Under a shared-pilot protocol the acquisition pool is different and the PALM curves may not match the full-trajectory curves. Please either clarify and validate the shared-pilot protocol or temper the claim: as written, the label-cost advantage of the pilot phase is unsupported.

minor comments (4)

[Abstract vs. Section 5] The pilot fraction is stated as 15–30% in the Abstract and Section 4.3, but as 20–30% in Section 5. Please make this consistent.
[Eq. (2) and surrounding text] The notation r·s is not defined precisely, and 'rounding up to the nearest multiple of Δb' is ambiguous when the left-hand side is already scaled by b. Please define the ceiling operation explicitly and reconcile it with Eq. (1).
[Table 2] The combined column header Aach{Amax is unclear; it should be explained in the caption that the first number is observed final accuracy and the second is the 30%-pilot-fitted asymptotic estimate.
[Algorithm 1] The algorithm does not specify what to return when no feasible method exists. A deployment user needs an explicit fallback (e.g., 'report all methods infeasible and stop').

Circularity Check

3 steps flagged · score 6.0 of 10

Fit-to-fit evaluation: pilot B_abs is a deterministic inversion of the pilot-fitted PALM curve, and the oracle is defined from the same curve family; observed crossing points are never checked.

fitted input called prediction [Section 3.3, Eq. (2); Tables 2 and 3]
"For each feasible method, the absolute annotation cost is the smallest budget at which predicted performance reaches τ: B_abs(τ)=min{B∈Z>0 : A^m(B)≥τ}. Inverting Eq. (1) gives a closed-form expression: B_abs(τ)=b·⌈((log(1−τ/A_max)/log(1−δ))^{1/β}−α)⌉_Δb."

The headline 'predicts the number of expert annotations needed' is implemented as a closed-form inversion of the pilot-fitted PALM parameters. The paper never checks B_abs against the observed label count where test accuracy first reaches τ; Table 3 only reports final accuracy after exhausting the pool, not the crossing point. Thus B_abs is a deterministic transformation of the fitted θ_m, so the claimed label-cost prediction reduces to the fitted curve values. If the curve is misspecified, the predicted cost is wrong by construction, and no observed data are used to correct it.
self definitional [Section 4.3, Figure 2]
"Figure 2 shows the method selected at each pilot fraction together with the full-curve oracle, defined as the method minimizing B_abs(τ) alone (no W tiebreak) as the strongest baseline, since it is the absolute lower bound on annotation cost."

The evaluation target ('deployment-optimal method') is itself defined by fitting the same PALM equation to the full trajectory and applying Eq. (2). Therefore the pilot-vs-oracle comparison in Figure 2 only checks whether the parametric curve fitted on 10–30% prefixes extrapolates to the same curve fitted on 100% of the data. It does not compare against an independently observed annotation cost. Calling the oracle the 'absolute lower bound' is misleading because it is a fitted estimate, not an empirically observed lower bound. The reported savings (up to 82%) are differences between two fitted values, so they measure fit-to-fit consistency, not actual labels saved.

1 more flagged steps

self citation load bearing [Section 2, 'Learning-curve prediction'; Section 3.1, Eq. (1)]
"PALM [4] proposed a parametric AL curve that can be fit from partial trajectories and used to forecast final performance. We adopt this model as the curve-fitting backbone of ALDA."

Reference [4] is the authors' own prior work (Machnio, Nielsen, Mehdipour Ghazi). The central extrapolation from a 15–30% pilot to full AL trajectories rests entirely on the PALM functional form of Eq. (1), and the paper provides no external validation that this form captures AL learning dynamics, especially late phase transitions. Its only supporting evidence is the fit-to-fit agreement in Section 4.3, which uses the same model family for both pilot and oracle. A misspecification of the PALM ansatz would bias pilot and oracle identically, so the self-citation is load-bearing and unverified within this paper.

full rationale

ALDA's decision layer (feasibility, cost, window, risk-aware selection) is a reasonable framework, and fitting a curve to a pilot is not circular per se. However, the paper's central claim — predicting the number of expert annotations needed and reducing annotation costs — is operationalized entirely through Eq. (2), a closed-form inversion of the pilot-fitted PALM curve. The evaluation never validates B_abs against the observed episode where accuracy first crosses τ. Instead, Section 4.3 defines the oracle as the method minimizing B_abs computed from full-trajectory PALM fits, so the pilot oracle comparison is fit-to-fit. Table 2 exposes a concrete misspecification: ProbCover on ISIC2019 has actual final accuracy 68.7% but pilot-fitted A_max of 54.7%, a 14 pp underestimate; the same misspecification would corrupt both pilot and oracle cost estimates. The 82% savings figures are therefore differences between fitted curve inversions, not measured label savings. Additionally, the PALM backbone is cited from the authors' own prior work without independent validation, making the self-citation load-bearing. These issues make the headline prediction substantially circular relative to its evaluation, though the framework itself is not entirely vacuous, so a score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central cost prediction B_abs is obtained by inverting a four-parameter curve fit to pilot data (Eq. 2), so the four curve parameters per method are the main free parameters. The evaluation thresholds and decision hyperparameters are hand-chosen. No new physical entities are introduced.

free parameters (4)

PALM curve parameters (A_max, delta, alpha, beta) = Fitted per method and dataset via L-BFGS-B; not reported in table
These four parameters per method determine B_abs and W through Eqs. (2) and (3); the central prediction is a deterministic function of them.
Clinical threshold tau = 85% (Fetal), 95% (BRISC), 70% (ISIC), 75% (BUSI)
Hand-chosen per dataset with no stated clinical source; Table 5 shows recommendations change with tau, so the main results depend on these operating points.
Threshold uncertainty Delta_tau = 5 percentage points
Default for the main results; the ablation shows W scales with it, and selections are stable for a fixed tau.
Cost non-inferiority margin eta = 0.05
Defines the near-optimal set C_eta; on BRISC it includes Entropy (880) and BADGE (895), and the W tiebreak flips the choice.

assumptions (4)

domain assumption The parametric curve in Eq. (1) accurately models AL accuracy trajectories for all methods and datasets.
Adopted from the authors' prior PALM work with no theoretical guarantee; B_abs and W are direct inversions of this curve, so a wrong functional form breaks the central claim.
domain assumption A 15-30% pilot prefix is representative of the full AL trajectory for every candidate method.
The prospective claim depends on extrapolation from short prefixes; the experiments test this only against full-trajectory fits, not against observed crossing labels.
domain assumption Test accuracy is the right clinical performance target.
tau is defined on accuracy even for class-imbalanced datasets; macro-F1 appears only in the appendix and is not used in the selection pipeline.
ad hoc to paper The non-inferiority plus deployment-window rule is a better selection rule than picking the nominal cheapest method.
eta and Delta_tau are hand-set, and no evidence is provided that the W tiebreak improves real deployment outcomes over simply picking minimal B_abs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification." pith.science (2026). https://pith.science/paper/56KUI7QF

@misc{pith2026260803511,
  author       = {Pith},
  title        = {Pith review of: How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56KUI7QF}},
  note         = {Machine review of arXiv:2608.03511}
}

read the original abstract

Active learning (AL) promises to reduce the cost of medical imaging projects by lowering the number of clinical labels required. However, practical deployment requires committing to a sampling strategy before the full annotation budget is spent, and choosing the wrong strategy can increase rather than decrease costs. We propose Active-Learning Deployment Advisor (ALDA), a deployment-oriented framework for AL method selection under clinical performance constraints. Given a short pilot phase, ALDA fits a parametric learning-curve model to each candidate strategy, estimates whether that strategy is expected to reach a required clinical performance target, and predicts the number of expert annotations needed to do so. In addition to absolute annotation cost, ALDA introduces a deployment window that quantifies the sensitivity of this cost estimate to uncertainty in the clinical threshold. The final recommendation follows a risk-aware rule: among strategies with near-optimal predicted cost, ALDA prefers the strategy with the narrowest deployment window, the most robust to threshold revisions. Experiments on four medical imaging classification domains show that ALDA predicts the deployment-optimal method from a pilot of 15-30% of the intended budget and reduces annotation costs by up to 82% compared with a poor strategy choice. Rather than introducing a new sampling heuristic, ALDA provides a practical decision layer that answers a deployment-critical question: how many labels are enough?

Figures

Figures reproduced from arXiv: 2608.03511 by the authors.

**Figure 1.** (A) Schematic of ALDA deployment quantities. The clinical target τ defines the required operating point; the shaded band denotes threshold uncertainty ˘∆τ . Method C is infeasible because its predicted ceiling lies below τ . Among feasible methods, B B abs ă B A abs and WB ă WA, so method B is more efficient and less risky. (B) Labelingcost savings of the ALDA recommendation relative to the most expensive feasible … view at source ↗

**Figure 2.** Pilot-based ALDA predictions from partial AL trajectories. For each dataset, the parametric curve is fitted to increasing pilot fractions (10%, 15%, 20%, 25%, 30%) and used to extrapolate the label budget required to reach the target threshold τ . The black curve denotes the oracle full-trajectory fit for the method selected on the complete trajectory; colored dashed curves show the methods selected from each pilot … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 20 canonical work pages

[1]

In: International Conference on Machine Learning, PMLR (2022) 8175–8195

Hacohen, G., Dekel, A., Weinshall, D.: Active learning on a budget: Opposite strategies suit high and low budgets. In: International Conference on Machine Learning, PMLR (2022) 8175–8195

work page 2022
[2]

A Mechanism-Driven Theory of Phase Transitions in Active Learning

Machnio, J., Nielsen, M., Mehdipour Ghazi, M.: A mechanism-driven theory of phase transitions in active learning. arXiv preprint arXiv:2607.00144 (2026)

work page Pith review arXiv 2026
[3]

arXiv preprint arXiv:2203.13450 (2022)

Zhan, X., Wang, Q., Huang, K.h., Xiong, H., Dou, D., Chan, A.B.: A comparative survey of deep active learning. arXiv preprint arXiv:2203.13450 (2022)

arXiv 2022
[4]

In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision

Machnio, J., Nielsen, M., Mehdipour Ghazi, M.: To label or not to label: PALM-a predictive model for evaluating sample efficiency in active learning models. In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision. (2025) 4039–4048

work page 2025
[5]

IEEE Transactions on Circuits and Systems for Video Technology27(12) (2016) 2591–2600

Wang, K., Zhang, D., Li, Y., Zhang, R., Lin, L.: Cost-effective active learning for deep image classification. IEEE Transactions on Circuits and Systems for Video Technology27(12) (2016) 2591–2600

work page 2016
[6]

In: International Conference on Learning Representations

Sener, O., Savarese, S.: Active learning for convolutional neural networks: A core- set approach. In: International Conference on Learning Representations. (2018)

work page 2018
[7]

In: International Con- ference on Learning Representations

Ash, J.T., Zhang, C., Krishnamurthy, A., Langford, J., Agarwal, A.: Deep batch active learning by diverse, uncertain gradient lower bounds. In: International Con- ference on Learning Representations. (2020)

work page 2020
[8]

Advances in Neural Information Processing Systems35(2022) 22354– 22367

Yehuda, O., Dekel, A., Hacohen, G., Weinshall, D.: Active learning through a cov- ering lens. Advances in Neural Information Processing Systems35(2022) 22354– 22367

work page 2022

Show all 21 references

[9]

In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion, Springer (2017) 399–407

Yang, L., Zhang, Y., Chen, J., Zhang, S., Chen, D.Z.: Suggestive annotation: A deep active learning framework for biomedical image segmentation. In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion, Springer (2017) 399–407

2017
[10]

Medical Image Analysis63(2020) 101693

Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J.N., Wu, Z., Ding, X.: Embracing imperfect datasets: A review of deep learning solutions for medical image segmen- tation. Medical Image Analysis63(2020) 101693

2020
[11]

In: 2018 17th IEEE International ConferenceonMachineLearningand Applications(ICMLA),IEEE(2018)481–488

Smailagic, A., Costa, P., Noh, H.Y., Walawalkar, D., Khandelwal, K., Galdran, A., Mirshekari, M., Fagert, J., Xu, S., Zhang, P., et al.: Medal: Accurate and robust deep active learning for medical image analysis. In: 2018 17th IEEE International ConferenceonMachineLearningand ...

2018
[12]

Journal of Computational Biology10(2) (2003) 119–142

Mukherjee, S., Tamayo, P., Rogers, S., Rifkin, R., Engle, A., Campbell, C., Golub, T.R., Mesirov, J.P.: Estimating dataset size requirements for classifying dna mi- croarray data. Journal of Computational Biology10(2) (2003) 119–142

2003
[13]

BMC Medical Informatics and Decision Making12(1) (2012) 8 10 J

Figueroa, R.L., Zeng-Treitler, Q., Kandula, S., Ngo, L.H.: Predicting sample size required for classification performance. BMC Medical Informatics and Decision Making12(1) (2012) 8 10 J. Machnio et al

2012
[14]

In: Proceedings of theThirteenthConferenceonComputationalNaturalLanguageLearning(CoNLL- 2009)

Bloodgood, M., Vijay-Shanker, K.: A method for stopping active learning based on stabilizing predictions and the need for user-adjustable stopping. In: Proceedings of theThirteenthConferenceonComputationalNaturalLanguageLearning(CoNLL- 2009). (2009) 39–47

2009
[15]

IEEE Transactions on Audio, Speech, and Language Processing18(6) (2009) 1323–1331

Zhu, J., Wang, H., Tsou, B.K., Ma, M.: Active learning with sampling by uncer- tainty and density for data annotations. IEEE Transactions on Audio, Speech, and Language Processing18(6) (2009) 1323–1331

2009
[16]

Scientific Data (2026)

Fateh,A.,Rezvani,Y.,Moayedi,S.,Rezvani,S.,Fateh,F.,Fateh,M.,Abolghasemi, V.: BRISC: Annotated dataset for brain tumor segmentation and classification. Scientific Data (2026)

2026
[17]

MethodsX 7(2020) 100864

Gessert, N., Nielsen, M., Shaikh, M., Werner, R., Schlaefer, A.: Skin lesion classifi- cation using ensembles of multi-resolution EfficientNets with meta data. MethodsX 7(2020) 100864

2020
[18]

Scientific Reports10(1) (2020) 10200

Burgos-Artizzu, X.P., Coronado-Gutiérrez, D., Valenzuela-Alcaraz, B., Bonet- Carne, E., Eixarch, E., Crispi, F., Gratacós, E.: Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes. Scientific Reports10(1) (2020) 10200

2020
[19]

Data in Brief28(2020) 104863

Al-Dhabyani, W., Gomaa, M., Khaled, H., Fahmy, A.: Dataset of breast ultrasound images. Data in Brief28(2020) 104863

2020
[20]

In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

He,K.,Zhang,X.,Ren,S.,Sun,J.: Deepresiduallearningforimagerecognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (2016) 770–778

2016
[21]

In: International Conference on Learning Represen- tations

Bae, W., Sutherland, D., Oliveira, G.: Uncertainty herding: One active learning method for all label budgets. In: International Conference on Learning Represen- tations. Volume 2025. (2025) 788–805 A Experimental Setup and Reproducibility Details Initialization.We evaluate ALD...

2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

In: International Conference on Machine Learning, PMLR (2022) 8175–8195

Hacohen, G., Dekel, A., Weinshall, D.: Active learning on a budget: Opposite strategies suit high and low budgets. In: International Conference on Machine Learning, PMLR (2022) 8175–8195

work page 2022

[2] [2]

A Mechanism-Driven Theory of Phase Transitions in Active Learning

Machnio, J., Nielsen, M., Mehdipour Ghazi, M.: A mechanism-driven theory of phase transitions in active learning. arXiv preprint arXiv:2607.00144 (2026)

work page Pith review arXiv 2026

[3] [3]

arXiv preprint arXiv:2203.13450 (2022)

Zhan, X., Wang, Q., Huang, K.h., Xiong, H., Dou, D., Chan, A.B.: A comparative survey of deep active learning. arXiv preprint arXiv:2203.13450 (2022)

arXiv 2022

[4] [4]

In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision

Machnio, J., Nielsen, M., Mehdipour Ghazi, M.: To label or not to label: PALM-a predictive model for evaluating sample efficiency in active learning models. In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision. (2025) 4039–4048

work page 2025

[5] [5]

IEEE Transactions on Circuits and Systems for Video Technology27(12) (2016) 2591–2600

Wang, K., Zhang, D., Li, Y., Zhang, R., Lin, L.: Cost-effective active learning for deep image classification. IEEE Transactions on Circuits and Systems for Video Technology27(12) (2016) 2591–2600

work page 2016

[6] [6]

In: International Conference on Learning Representations

Sener, O., Savarese, S.: Active learning for convolutional neural networks: A core- set approach. In: International Conference on Learning Representations. (2018)

work page 2018

[7] [7]

In: International Con- ference on Learning Representations

Ash, J.T., Zhang, C., Krishnamurthy, A., Langford, J., Agarwal, A.: Deep batch active learning by diverse, uncertain gradient lower bounds. In: International Con- ference on Learning Representations. (2020)

work page 2020

[8] [8]

Advances in Neural Information Processing Systems35(2022) 22354– 22367

Yehuda, O., Dekel, A., Hacohen, G., Weinshall, D.: Active learning through a cov- ering lens. Advances in Neural Information Processing Systems35(2022) 22354– 22367

work page 2022