Epistemic Uncertainty Sampling

Eyke H\"ullermeier; S\'ebastien Destercke; Vu-Linh Nguyen

REVIEW 3 major objections 4 minor 23 references

Epistemic Uncertainty Sampling

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Active learners should query instances with high epistemic uncertainty—ignorance about the model—not high aleatoric uncertainty from noise.

desk verdict A competent and honest application of a known epistemic/aleatoric split to uncertainty sampling, with a real limitation: the key measure can vanish under model misspecification. read the letter →

arxiv 1909.00218 v1 pith:JS7GDTOE submitted 2019-08-31 cs.LG stat.ML

Vu-Linh Nguyen , Sébastien Destercke , Eyke Hüllermeier This is my paper · ORCID

classification cs.LGstat.ML

keywords activelearninguncertaintysamplingepistemicaleatoriclikelihood-basedplausibilityParzenwindowclassifierdecisiontreelogisticregression

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Active learning is about spending labels where they will teach the most. This paper argues that standard uncertainty sampling conflates two different things: epistemic uncertainty, which comes from not knowing the right model and can be reduced by more data, and aleatoric uncertainty, which comes from inherent randomness in the labels and cannot. The paper's conjecture is that, in uncertainty sampling, an instance's usefulness is better reflected by its epistemic component, and it instantiates the query rule $x^* = \arg\max_{x\in U} u_e(x)$ using likelihood-based plausibility scores. Experiments with Parzen-window classifiers, decision trees, and logistic regression on eight datasets support the conjecture, with epistemic uncertainty sampling performing best and aleatoric-only sampling worst. If the conjecture holds, active learners can become more label-efficient simply by asking whether the uncertainty they see is reducible before querying.

What carries the argument

The machinery is the normalized likelihood $\pi_H(h)=L(h)/L(h_{ml})$ of a model $h$ given the training data, combined with the two degrees of support $\pi(1\mid x)=\sup_{h\in H}\min[\pi_H(h), 2h(x)-1]$ and $\pi(0\mid x)=\sup_{h\in H}\min[\pi_H(h), 1-2h(x)]$. These support degrees turn training data into a plausibility ranking over hypotheses and then split total uncertainty into $u_e$ and $u_a$. Everything else is computation: for Parzen windows and decision-tree leaves, constant hypotheses per region reduce the supports to one-dimensional scalar maximizations solved with Brent's method; for logistic regression, the supports are found by maximizing a strictly concave regularized log-likelihood over parallel hyperplanes, with the level $\alpha$ discretized and solved by Newton-CG or sequential least squares.

What would settle it

Run the comparison on a synthetic binary problem where one region has clean labels and another has labels flipped with probability $p$, for several values of $p$. If instances with high $u_e$ are not the ones whose addition reduces test error fastest, or if the accuracy advantage of the $u_e$ rule fails to grow with $p$, the central conjecture is false.

Watch

Extended reading notes

Core claim

The paper's central claim is that in uncertainty sampling the utility of an unlabeled instance is better captured by its epistemic uncertainty than by its aleatoric uncertainty, so the learner should query $x^* = \arg\max_{x\in U} u_e(x)$ instead of maximizing predictive entropy. With $u_e(x)=\min[\pi(1\mid x),\pi(0\mid x)]$ and $u_a(x)=1-\max[\pi(1\mid x),\pi(0\mid x)]$, the first is high when plausible models disagree strongly about which class is more likely, while the second is high when every plausible model puts the class probability near one half. The paper derives these degrees of support from normalized likelihoods, instantiates the query rule for local learners and logistic regression, and presents experiments in which epistemic uncertainty sampling achieves the best accuracy and aleatoric sampling the worst, with standard entropy and random sampling in between. The paper interprets the smaller gains under logistic regression as evidence that the method's advantage is tied to how long the hypothesis space retains genuine model uncertainty.

Load-bearing premise

The method assumes that the numbers it computes from the training data truly separate 'uncertainty that more data can fix' from 'uncertainty that is just noise', rather than merely reflecting the arbitrary choice of model family.

Editorial extensions

If this is right

Replacing entropy or least-confidence with $u_e(x)$ as the utility score should give an active learner comparable or better accuracy at the same label budget, since the experiments show the epistemic rule dominating standard uncertainty sampling.
Aleatoric-only sampling is a cautionary baseline: querying the instances where all plausible models agree on a near-random class probability spends labels where the label noise itself is the problem.
The difference between epistemic and aleatoric scores is a practical diagnostic: a large gap means the learner's uncertainty is mostly reducible, while a small gap means further labels are unlikely to sharpen the model.
For algorithms with strong inductive bias, such as linear logistic regression, epistemic uncertainty can be exhausted quickly, so epistemic sampling converges to standard uncertainty sampling; the paper's generic construction means the rule can be attached to any hypothesis space for which the supports can be computed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural stress test not run in the paper: on a synthetic binary task with known label-flip noise in one region, the gap between $u_e$ sampling and $u_a$ sampling should widen as the flip rate rises, because the aleatoric rule will systematically drain the budget in the noisy region.
The logistic-regression finding suggests a general dependency: the amount of epistemic uncertainty an instance carries depends on the expressiveness of the hypothesis space, so kernel-based or deep models, which keep model uncertainty alive longer, would plausibly show larger gains from epistemic sampling than linear models do.
The same $u_e$ measure could serve as a stopping rule: once the pool-wide epistemic uncertainty is low, the expected benefit of further queries is small, which would let an active learner stop early and save labeling cost.
If the conjecture is correct, the success of entropy-based uncertainty sampling in earlier work can be reinterpreted as approximating epistemic uncertainty; explicit separation should matter most when label noise is heterogeneous across the input space.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The short version: this is a competent extension of a known distinction to active learning, not a new conceptual breakthrough. The authors take the epistemic/aleatoric decomposition from their own earlier framework [19], define an epistemic uncertainty measure u_e, and propose querying the pool element with maximal u_e. For Parzen windows and decision trees the computations are closed-form and the experiments are clean enough to show a consistent advantage over entropy-based uncertainty sampling. That is a genuine, if modest, contribution.

What I like: the instantiations are worked out carefully, the aleatoric control is a nice touch, and the authors do not oversell the results in the main body. They include the logistic regression case where the advantage disappears, and in Section 5.2 they attribute it to the linearity bias of the model. That is honest reporting.

The soft spots are real but not fatal. The biggest issue is the conceptual gap between 'epistemic uncertainty' as disagreement within the chosen hypothesis space H and 'reducible uncertainty' with respect to the true data-generating process. Under misspecification, u_e can be near zero exactly where a label would reduce error the most. The paper concedes this implicitly in the logistic regression section; I would have liked an explicit test of whether u_e tracks error reduction on the datasets, rather than just an accuracy comparison. A second issue is the missing comparison to BALD, which is the established method for querying by epistemic uncertainty in Bayesian active learning. That omission weakens the novelty claim. Third, the logistic regression support calculation is a discretized search over 50 alpha values on a non-concave objective; there is no certificate that the global optimum is found. That is a minor implementation concern but should be stated as an approximation.

Given the paper's scope, none of these should block publication. The central claim is supported for the flexible model classes where it is most plausible, and the experiments, while lacking error bars, are consistent across eight datasets. I would send it to a serious referee, asking for error bars, a BALD comparison, and a short discussion of what u_e measures under misspecification.

For a reader in active learning or uncertainty quantification, this is worth a look. I don't see myself citing it unless I work on uncertainty sampling specifically, but it is a reasonable paper.

Referee Report

3 major / 4 minor

Summary. The paper proposes epistemic uncertainty sampling, a query strategy for active learning in binary classification. Building on the normalized-likelihood framework of Senge et al. (2014), it defines epistemic uncertainty u_e(x) (Eq. 9) and aleatoric uncertainty u_a(x) (Eq. 10) as functions of support degrees of the two classes, and suggests querying the instance maximizing u_e (Eq. 11). Closed-form support degrees are derived for Parzen window classifiers and decision trees (Section 4.1), while logistic regression is handled by discretizing the parameter constraint alpha (Section 4.2). Experiments on eight UCI data sets compare epistemic, aleatoric, standard uncertainty, and random sampling; epistemic sampling performs best for the two local learners and comparably to standard uncertainty for logistic regression.

Significance. If the central conjecture is valid, the paper makes a useful conceptual contribution by separating reducible from irreducible uncertainty in active learning and providing a concrete, algorithm-agnostic measure. The derivations for Parzen windows and decision trees are closed-form and clear. However, the claim that u_e tracks reducible uncertainty is asserted rather than proven, and the empirical evidence lacks error bars and significance tests. The paper's own logistic-regression results show the measure can collapse under model misspecification, so the generality of the conclusion is not established. The work is nevertheless a reasonable starting point for further study.

major comments (3)

[Section 3, Eqs. (9)-(10)] The central claim that u_e measures reducible uncertainty is not established. Equations (9)-(10) define u_e as the minimum of support degrees obtained from normalized likelihoods over a fixed hypothesis space H. This measures internal disagreement among plausible models, not reducibility with respect to the true data-generating process. Under model misspecification, plausible models can agree at a point even though labeling it would substantially reduce error, and u_e can vanish exactly where error reduction is most needed. The authors concede this behavior for logistic regression in Section 5.2, where u_e 'disappears quite quickly' under the linearity assumption. Since the conclusion in Section 6 states that 'the epistemic part is the better criterion' without qualification, this is a load-bearing gap. The authors should either state precise conditions on H under which arg max u_e corresponds to maximal expected error reduction, or add an experiment that directly measures whether u_e tracks error reduction under misspecification.
[Section 4.2, Eq. (23)] The global optimum of the non-concave problem (19) is approximated by scanning 50 discretized alpha values and taking the best restricted maximum, without any certificate of global optimality. Because u_e is used as the query score, a poor approximation could change instance rankings and thereby affect the experimental conclusions. The authors should provide evidence that the discretization is adequate, for example by comparing against finer grids or multiple initializations, or by using a certified global optimization technique on a subset of instances.
[Section 5, Figures 2-4] The experimental section reports only averaged accuracies over 10-fold cross-validation, without standard deviations, confidence intervals, or significance tests. The claimed advantage of epistemic over standard uncertainty sampling is described as 'not huge' but 'visible and consistent' (Section 5.1). Since the pool is also randomly subsampled at 10% in each iteration, the results may have substantial variability. Given that the paper's main evidence is empirical, the absence of any statistical assessment makes it difficult to evaluate whether the observed differences are reliable. Paired tests (e.g., Wilcoxon signed-rank across datasets) and error bars or per-fold spreads should be reported.

minor comments (4)

[Section 4.1, after Eq. (14)] The notation is confusing: 'p' denotes both the number of positive instances and the probability ph(1|x). Renaming the count variable (e.g., to 'k' or 'n_+') would improve readability.
[Algorithm 3, line 10] There appears to be a typo: the update for pi(0|x) uses '1 - 2*alpha_p' but should likely use '1 - 2*alpha_n' to match the negative-class support in Eq. (24).
[Section 5.1, Algorithm 2] Algorithm 2 sets epsilon to the average K-th nearest neighbor distance over training instances, but the text states this makes the average number of neighbors 'just K'. This is only approximately true, and the approximation is not quantified; a brief clarification would help.
[Figure 1] The color map has no colorbar or scale, so the reader cannot quantitatively interpret the uncertainty values shown. Adding a colorbar would improve the figure's informativeness.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the uncertainty measures are transparently restated from prior work and the central claim is an empirically tested conjecture, not a quantity forced by construction.

full rationale

The paper's derivation chain is not circular. The epistemic and aleatoric uncertainty measures in Eqs. (4)-(10) are explicitly recalled from the authors' earlier framework [19], but the paper says this is done "for the sake of completeness and self-containedness" and reproduces the full definitions, including the normalized likelihood, support degrees, and the resulting uncertainty degrees. Thus the reliance on [19] is transparent rather than a black-box self-citation. Moreover, the cited framework itself rests on external foundations (relative likelihoods of Birnbaum [2] and the likelihood-based possibility framework of Walley and Moral [23]), so it is not an unverified premise invented for this paper. The central active-learning claim, that epistemic uncertainty is a better query criterion than aleatoric uncertainty, is explicitly introduced as a conjecture ("We conjecture that, in uncertainty sampling, the usefulness of an instance is better reflected by its epistemic than by its aleatoric uncertainty") and then tested against UCI benchmark data sets in Section 5. The selection rule x* = arg max u_e(x), Eq. (11), follows from the interpretation of u_e as reducible uncertainty, but the paper does not present this as a formal derivation of empirical success; the experimental comparison supplies the evidence. No fitted parameter is renamed as a prediction: hyperparameters such as K and the regularization gamma are fixed in advance, and the method is not tuned to the active-learning outcome. The Section 5.2 observation that logistic regression's linearity bias makes epistemic uncertainty "disappear quite quickly" is an honest limitation of the approach under model misspecification, not a circular step. There is therefore no reduction of the paper's conclusions to its inputs, and no load-bearing self-citation chain.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method does not introduce new physical or model entities. It inherits the uncertainty decomposition from [19] and adds algorithmic parameters (K, Q, gamma). The main burden is the assumption that normalized likelihood plausibility correctly captures epistemic uncertainty for the chosen hypothesis spaces.

free parameters (3)

K (neighborhood size) = sqrt(N), where N is the size of the initial training plus pool set
Determines the Parzen window width and the scale of local uncertainty estimates (Section 5.1, Algorithm 2). Set by a rule of thumb from [10]; no sensitivity analysis is provided.
Q (alpha discretization count) = 50
Number of alpha values used in the logistic regression search (Section 4.2). The approximation quality depends on this arbitrary choice.
gamma (L2 regularization) = 1
Regularization coefficient in the logistic regression log-likelihood (Eq. 18). Fixed to 1 without tuning; it affects the normalized likelihood and thus u_e and u_a.

assumptions (5)

domain assumption Normalized likelihood pi_H(h) is a valid plausibility measure for hypotheses
Used in Eqs. (4)-(6) from [19]; the entire uncertainty decomposition rests on this.
domain assumption For every x, there are hypotheses h and h' with h(x) >= 0.5 and h'(x) <= 0.5
Footnote 1 in Section 3; required for pi(1|x) and pi(0|x) to be non-negative.
domain assumption Epistemic uncertainty u_e (Eq. 9) corresponds to reducible model uncertainty, while u_a (Eq. 10) is irreducible
The interpretive link that motivates selection by u_e; if the decomposition does not align with reducibility of test error, the method loses its rationale.
standard math Solving the concave log-likelihood over the hyperplane constraint (Eq. 22) gives the true restricted maximum
Valid because the L2-regularized log-likelihood is strictly concave (Eq. 18, citing [17]).
ad hoc to paper The global solution of the non-concave problem (19) is obtained by scanning alpha in [0.5, 1) and taking the best restricted maximum
Section 4.2 discretizes alpha into 50 values; no proof that this finds the global optimum of (7).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Epistemic Uncertainty Sampling." pith.science (2026). https://pith.science/paper/JS7GDTOE

@misc{pith2026190900218,
  author       = {Pith},
  title        = {Pith review of: Epistemic Uncertainty Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JS7GDTOE}},
  note         = {Machine review of arXiv:1909.00218}
}

read the original abstract

Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are almost exclusively of a probabilistic nature. In this paper, we advocate a distinction between two different types of uncertainty, referred to as epistemic and aleatoric, in the context of active learning. Roughly speaking, these notions capture the reducible and the irreducible part of the total uncertainty in a prediction, respectively. We conjecture that, in uncertainty sampling, the usefulness of an instance is better reflected by its epistemic than by its aleatoric uncertainty. This leads us to suggest the principle of "epistemic uncertainty sampling", which we instantiate by means of a concrete approach for measuring epistemic and aleatoric uncertainty. In experimental studies, epistemic uncertainty sampling does indeed show promising performance.

Figures

Figures reproduced from arXiv: 1909.00218 by the authors.

**Figure 2.** Average accuracies (y-axis) for the Parzen window classifier as a function of the [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

**Figure 3.** Average accuracies (y-axis) for the decision tree classifier as a function of the [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Average accuracies (y-axis) for logistic regression as a function of the number [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 21 canonical work pages

[19]

Reliable classiﬁcation: Learning classiﬁers that distinguish aleatoric and epistemic uncertainty

Robin Senge, Stefan B¨ osner, Krzysztof Dembczy´ nski, J¨ org Haasenritter, Oliver Hirsch, Norbert Donner-Banzhoﬀ, and Eyke H¨ ullermeier. Reliable classiﬁcation: Learning classiﬁers that distinguish aleatoric and epistemic uncertainty. Informa- tion Sciences, 255:16–29, 2014

work page 2014
[1]

Active learning by the naive credal classiﬁer

Alessandro Antonucci, Giorgio Corani, and Sandra Gabaglio. Active learning by the naive credal classiﬁer. In Proceedings of the Sixth European Workshop on Probabilistic Graphical Models (PGM), pages 3–10, 2012

work page 2012
[2]

On the foundations of statistical inference

Allan Birnbaum. On the foundations of statistical inference. Journal of the American Statistical Association, 57(298):269–306, 1962

work page 1962
[3]

Local learning algorithms

L´ eon Bottou and Vladimir Vapnik. Local learning algorithms. Neural Computation, 4(6):888–900, 1992

work page 1992
[4]

Active learning for parzen window classiﬁer

Olivier Chapelle. Active learning for parzen window classiﬁer. In Proceedings of the Tenth International Workshop on Artiﬁcial Intelligence and Statistics (AISTATS) , volume 5, pages 49–56, 2005

work page 2005
[5]

Nearest neighbor pattern classiﬁcation

Thomas Cover and Peter Hart. Nearest neighbor pattern classiﬁcation. IEEE Trans- actions on Information Theory , 13(1):21–27, 1967

work page 1967
[6]

A survey on instance selection for active learning

Yifan Fu, Xingquan Zhu, and Bin Li. A survey on instance selection for active learning. Knowledge and Information Systems , pages 1–35, 2013

work page 2013
[7]

The ele- ments of statistical learning: Data mining, inference and prediction

Trevor Hastie, Robert Tibshirani, Jerome Friedman, and James Franklin. The ele- ments of statistical learning: Data mining, inference and prediction. The Mathemat- ical Intelligencer, 27(2):83–85, 2005

work page 2005

Show all 23 references

[8]

Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management

Stephen C Hora. Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management. Reliability Engineering & System Safety, 54(2-3):217–223, 1996

1996
[9]

Kendall and Y

A. Kendall and Y. Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS), 2017

2017
[10]

A nearest neighbor bootstrap for resampling hy- drologic time series

Upmanu Lall and Ashish Sharma. A nearest neighbor bootstrap for resampling hy- drologic time series. Water Resources Research, 32(3):679–693, 1996

1996
[11]

A sequential algorithm for training text classi- ﬁers

David D Lewis and William A Gale. A sequential algorithm for training text classi- ﬁers. In Proceedings of the 17th annual International SIGIR Conference on Research and Development in Information Retrieval , pages 3–12. Springer, 1994

1994
[12]

Applied Logistic Regression Analysis, volume 106

Scott Menard. Applied Logistic Regression Analysis, volume 106. Sage, 2002

2002
[13]

Reliable multi-class classiﬁcation based on pairwise epistemic and aleatoric uncer- tainty

Vu-Linh Nguyen, S´ ebastien Destercke, Marie-H´ el` ene Masson, and Eyke H¨ ullermeier. Reliable multi-class classiﬁcation based on pairwise epistemic and aleatoric uncer- tainty. In Proceedings of the 27th International Joint Conference on Artiﬁcial Intel- ligence (IJCAI), pa...

2018
[14]

Numerical Optimization

Jorge Nocedal and S Wright. Numerical Optimization. Springer New York, 2006

2006
[15]

Sequential quadratic programming methods

E Philip and WONG Elizabeth. Sequential quadratic programming methods. UCSD Department of Mathematics Technical Report NA-10-03 , 2010

2010
[16]

Ross Quinlan

J. Ross Quinlan. Induction of decision trees. Machine learning, 1(1):81–106, 1986

1986
[17]

Regularized logistic regression is strictly convex

Jason DM Rennie. Regularized logistic regression is strictly convex. Technical report, MIT, 2005

2005
[18]

A survey of decision tree classiﬁer method- ology

S Rasoul Safavian and David Landgrebe. A survey of decision tree classiﬁer method- ology. IEEE transactions on systems, man, and cybernetics , 21(3):660–674, 1991

1991
[20]

Active learning literature survey

Burr Settles. Active learning literature survey. Technical Report, University of Wis- consin, Madison, 52(55-66):11, 2010

2010
[21]

Evidence-based uncertainty sampling for active learning

Manali Sharma and Mustafa Bilgic. Evidence-based uncertainty sampling for active learning. Data Mining and Knowledge Discovery , 31(1):164–202, 2017

2017
[22]

An overview of statistical learning theory

Vladimir N Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks , 10(5):988–999, 1999

1999
[23]

Upper probabilities based only on the likelihood function

Peter Walley and Seraﬁn Moral. Upper probabilities based only on the likelihood function. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 61(4):831–847, 1999. 16

1999

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [19]

Reliable classiﬁcation: Learning classiﬁers that distinguish aleatoric and epistemic uncertainty

Robin Senge, Stefan B¨ osner, Krzysztof Dembczy´ nski, J¨ org Haasenritter, Oliver Hirsch, Norbert Donner-Banzhoﬀ, and Eyke H¨ ullermeier. Reliable classiﬁcation: Learning classiﬁers that distinguish aleatoric and epistemic uncertainty. Informa- tion Sciences, 255:16–29, 2014

work page 2014

[2] [1]

Active learning by the naive credal classiﬁer

Alessandro Antonucci, Giorgio Corani, and Sandra Gabaglio. Active learning by the naive credal classiﬁer. In Proceedings of the Sixth European Workshop on Probabilistic Graphical Models (PGM), pages 3–10, 2012

work page 2012

[3] [2]

On the foundations of statistical inference

Allan Birnbaum. On the foundations of statistical inference. Journal of the American Statistical Association, 57(298):269–306, 1962

work page 1962

[4] [3]

Local learning algorithms

L´ eon Bottou and Vladimir Vapnik. Local learning algorithms. Neural Computation, 4(6):888–900, 1992

work page 1992

[5] [4]

Active learning for parzen window classiﬁer

Olivier Chapelle. Active learning for parzen window classiﬁer. In Proceedings of the Tenth International Workshop on Artiﬁcial Intelligence and Statistics (AISTATS) , volume 5, pages 49–56, 2005

work page 2005

[6] [5]

Nearest neighbor pattern classiﬁcation

Thomas Cover and Peter Hart. Nearest neighbor pattern classiﬁcation. IEEE Trans- actions on Information Theory , 13(1):21–27, 1967

work page 1967

[7] [6]

A survey on instance selection for active learning

Yifan Fu, Xingquan Zhu, and Bin Li. A survey on instance selection for active learning. Knowledge and Information Systems , pages 1–35, 2013

work page 2013

[8] [7]

The ele- ments of statistical learning: Data mining, inference and prediction

Trevor Hastie, Robert Tibshirani, Jerome Friedman, and James Franklin. The ele- ments of statistical learning: Data mining, inference and prediction. The Mathemat- ical Intelligencer, 27(2):83–85, 2005

work page 2005