{"id":"5fd46c3f-e2e1-4bd1-b78a-89523c60da3c","arxiv_id":"2412.08225","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"New active learning acquisition functions derived from a possibilistic representation of epistemic uncertainty match or beat standard GP-based baselines on several classification datasets.","lead":"The paper proposes new active learning strategies that separate epistemic from aleatoric uncertainty by mixing probability and possibility theory, and introduces a possibilistic analogue of Gaussian processes. The strategies match or beat standard GP classification baselines on several datasets, but the supporting proofs and code availability need scrutiny.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proof of Proposition 1(A4) relies on a definition of 'strictly less informative' that is too weak; under the stated definition the claimed monotonicity of U^L is false, so the advertised desirable properties are unsupported.","rationale":"I read the paper as making two intertwined claims: the proposed acquisition functions have a list of desirable properties, and they perform well empirically. The empirical part is plausible: Tables 1 and 2 show consistent median improvements, and the PGP/Laplace machinery is standard. However, the theoretical claim is load-bearing for the paper's novelty and is not supported. The proof of Proposition 1(A4) for U^Y simply asserts an inequality that does not follow from footnote 1; the counterexample shows the inequality is false, so the property itself fails under the stated definition. The same definitional problem affects Proposition 4's strict A4 for N_bin. The A3 change-of-variables error for U^\\Theta is a separate and unambiguous mathematical mistake: on a (d−1)-dimensional simplex the Jacobian is \\alpha^{d-1}, not \\alpha. These errors mean the 'desirable properties' contribution, as written, collapses. The empirical results and the PGP construction are not themselves disproven, but the central claim that the new strategies are principled in the advertised sense does not hold. The missing code URL and lack of significance testing further weaken the empirical half, though they are secondary. Because the theoretical claim is central and false as stated, I do not see a basis to soften the reader's REJECT; a revision that corrects the definition, fixes the Jacobian, and releases code could change that.","tokens_in":21962,"tokens_out":14537,"duration_ms":153713,"concrete_test":"Construct the explicit counterexample: \\Theta=[0,1], Y={0,1}, p_Y(1|\\theta)=\\theta, f(0)=f(1)=1, f(\\theta)=0.5 on [0.4,0.6] and 0 elsewhere, and f'(\\theta)=1 on [0.4,0.6] and 0 elsewhere. Compute U^Y_f = \\sup_\\theta f(\\theta)\\theta + \\sup_\\theta f(\\theta)(1-\\theta) - 1 and U^Y_{f'} similarly. If U^Y_{f'} < U^Y_f, then Property A4 (weak) is false under the paper's own definition. As a second check, re-derive the A3 proof with a 3-class simplex and T(\\theta)=\\alpha\\theta+(1-\\alpha)c; the integral over the scaled simplex confirms the Jacobian is \\alpha^2, not \\alpha.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central theoretical contribution is that the new acquisition functions U^L and 1−N satisfy the desirable properties A0–A4 (Propositions 1 and 4). The load-bearing step is the proof of Property A4 for U^Y in Appendix D. Footnote 1 defines f' to be strictly less informative than f when f'(\\theta)>f(\\theta) on some positive-volume region A, with no constraint outside A. The proof then asserts \\int \\sup_\\theta f'(\\theta)p(y|\\theta)\\,dy \\ge \\int \\sup_\\theta f(\\theta)p(y|\\theta)\\,dy. This inequality is false. Take \\Theta=[0,1], Y={0,1}, p_Y(1|\\theta)=\\theta, f=1 at {0,1} and 0.5 on [0.4,0.6], and f'=1 on [0.4,0.6] and 0 elsewhere. Both are valid possibility functions and f'>f on A, so f' is 'strictly less informative' under the paper's definition. Yet U^Y_f=1 while U^Y_{f'}=0.2, violating even the weak version of A4 that Proposition 1 claims. The same defective definition breaks the strict-A4 proof for 1−N_bin: a Gaussian shifted to a far-away mode is 'less informative' under footnote 1 yet decreases 1−N_bin. Separately, the A3 proof for U^\\Theta performs a change of variables on the simplex; the Jacobian is \\alpha^{\\dim\\Theta}, so U^\\Theta_{f'} = \\alpha^{d-1}U^\\Theta_f, not \\alpha U^\\Theta_f, for multiclass problems. These are not cosmetic gaps: they falsify the stated properties, which are listed as Contribution 1. The empirical claim of strong performance is separable and may survive, but the theoretical claim as written does not.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops an active learning framework in which epistemic uncertainty is represented by possibility functions rather than probability distributions. It proposes two acquisition functions: U^L, an integral of a marginal outer probability measure over label space, and 1−N, the complement of the necessity of correct classification. A possibilistic Gaussian process (PGP) is introduced and shown to reproduce standard GP predictive equations. The authors prove axiomatic properties for the acquisition functions and report experiments on 11 real and synthetic datasets where the proposed functions often rank best or second best among local, update-free strategies.","tokens_in":22444,"tokens_out":13200,"duration_ms":140916,"significance":"If the theoretical results were valid, the paper would be a useful contribution: it gives explicit, computationally cheap acquisition functions in a coherent possibility-theoretic framework, states a clear list of axioms A0–A4, derives closed-form expressions for the necessity of correct classification, and provides a broad experimental comparison with rank summaries. The PGP construction is transparent, and the empirical protocol is reproducible in principle. The main advertised contribution, however, is the set of desirable properties in Propositions 1 and 4, and these are not established as stated; the current proofs rely on a definition of 'strictly less informative' that is too weak and on an incorrect Jacobian. The empirical comparison is plausible, but the headline claim of outperformance would benefit from significance testing.","major_comments":[{"comment":"The proof of Property A4 for U^Y is invalid under the definition of 'strictly less informative' in footnote 1, which only requires f'_θ > f_θ on a positive-volume set A and imposes no constraint on Θ\\A. The inequality ∫ sup_θ f'_θ(θ)p(y|θ) dy ≥ ∫ sup_θ f_θ(θ)p(y|θ) dy does not follow, and it is false. For instance, take Θ=[0,1], Y={0,1}, p_Y(1|θ)=θ, f=1 at {0,1} and 1/2 on [0.4,0.6], and f'=1 on [0.4,0.6] and 0 elsewhere. Then f' is strictly less informative than f in the sense of footnote 1, but U^Y_f = 1 and U^Y_{f'} = 0.2. The proof of strict A4 for U^Theta has the same gap: it assumes f'_θ ≥ f_θ on Θ\\A, which is not part of the definition. A pointwise domination condition would repair the argument, but it must be stated explicitly.","section":"§4.1 / Appendix D, Prop. 1, footnote 1"},{"comment":"In the multinomial case, T(θ)=αθ+(1−α)c is an affine map on the (m−1)-dimensional simplex. Its Jacobian with respect to Lebesgue measure on the simplex is α^{m−1}, not α. Therefore the change of variables in the proof gives U^Θ_{f_T(θ)} = α^{m−1} U^Θ_{f_θ}, which agrees with the claimed Property A3 only when m=2. Proposition 1's statement that U^Theta satisfies A3 as written is false for multiclass problems.","section":"Appendix D, Prop. 1, Property A3"},{"comment":"The proof of strict A4 for 1−Nbin states that the only way a Gaussian possibility function can be strictly less informative than another is by keeping the mean and increasing the variance. This is not true under footnote 1: shifting the mean from μ to μ′>μ makes the shifted Gaussian larger than the original on a neighborhood of μ′, which has positive volume, so the shifted function is 'strictly less informative' by the paper's definition. But, for μ>0, 1−Nbin = N(0;μ,σ²) becomes N(0;μ′,σ²), which is smaller, contradicting strict monotonicity. The proof of A1 in the same proposition also assumes μ_x≠0, so it does not cover a point mass at μ_x=0, where the most likely label is not uniquely defined. Proposition 4 needs a stronger definition of less-informativeness or a restricted statement.","section":"Appendix D, Prop. 4"},{"comment":"The claim that the new acquisition functions 'outperform acquisition functions with the same properties' is not supported by a statistical test. On several datasets the best proposed method is within the interquartile range of the Standard baseline (e.g., Breast Cancer: Nbin 0.960 [0.957,0.963] vs Standard 0.957 [0.950,0.960]; Wine: 0.988 [0.986,0.994] vs 0.988 [0.986,0.994]). The average-rank results in Table 2 are more encouraging, but the paper should either provide paired significance tests across runs or temper the wording.","section":"§6, Tables 1–2"}],"minor_comments":[{"comment":"The paper states that code is available at '[URL]' and Appendix E says it will be released upon acceptance; a stable repository link should be provided.","section":"§6 / Appendix E"},{"comment":"For binary classification the GP hyperparameters are fixed after being computed in advance, while for multiclass they are optimized during training; please state explicitly whether this asymmetry affects both proposed and baseline methods equally at each query.","section":"Appendix E.3"},{"comment":"The shaded regions are Q1–Q3 intervals, but the figure caption does not state the number of runs per dataset; the number of runs varies between 50 and 300 in Table 3, so the caption should be made consistent.","section":"Figure 1"},{"comment":"The binary least-confidence strategy is called 'Standard' in the experiments; this should be stated at first use in Section 6 so that readers do not confuse it with a generic standard GP baseline.","section":"Appendix E.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a candidate for resubmission rather than acceptance in its present form: the empirical part may survive, but the theoretical pillar needs a careful reworking of the definition of less-informativeness, the A3 scaling, and the proof of Proposition 4. I would not reject the underlying research line, but the current manuscript overstates what is proved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read of this paper. The headline: the two acquisition functions (U^L and 1−N) are genuinely new, closed-form, update-free rules, and the empirical comparison on 11 datasets is honest and fairly impressive—N_bin and U^L are usually best or second best against BALD, entropy, and the rest. The PGP construction is a nice formal exercise: it re-derives standard GP predictive equations from possibility theory, which is less novel than it first appears but still a useful conceptual shift.\n\nWhere it falls down is the theoretical core. The paper's first listed contribution is that these acquisition functions satisfy the desirable properties A0–A4. That claim is not supported as written. In Appendix D, the proof of A4 for both U^Theta and U^Y assumes that if f' is larger than f on some positive-volume region A, then the integral or supremum over the whole space must be larger. That's false under the paper's own definition of 'strictly less informative' (footnote 1), which says nothing about behavior outside A. A concrete counterexample is easy to build—the stress-test note gives one where U^Y decreases despite f' being 'less informative'. The A3 proof on the simplex also botches the Jacobian (alpha^(d-1) instead of alpha). These aren't cosmetic slips; they directly undermine the claimed 'desirable properties' that justify the methods.\n\nAlso worth noting: the proof of Prop 4's A1 fails at mu=0, where 1−N_bin is 1 instead of 0, so the 'EU vanishes when the parameter is known' property breaks at decision boundaries. And the code is not actually available—the main text says 'code is available at [URL]' but the appendix says it will be released upon acceptance.\n\nThe empirical results are separable and may survive. The methods are cheap, local, and competitive on the tested datasets. But the paper overclaims by tying them to a set of properties that are not proven. The authors could fix this by either strengthening the definition of 'less informative' (e.g., requiring f' <= f outside A) or trimming the theoretical claims to something the proofs actually support.\n\nWho is this for? Someone working in active learning with GPs, especially if they're open to possibility-theoretic formulations. It deserves a serious referee because the empirical idea and the PGP framework are worth engaging with, but as it stands the central theoretical claim doesn't hold. I'd push for major revision or resubmission with corrected proofs, not desk reject.","headline":"New acquisition functions with a solid empirical story, but the central proof of their advertised 'desirable properties' does not hold; the paper deserves a referee's time only if the authors can fix or scope down the theory.","tokens_in":22883,"tokens_out":3157,"would_cite":false,"duration_ms":31057,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62L05","62F15","62H30"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that representing epistemic uncertainty with possibility functions yields two new active-learning acquisition functions that satisfy a desired list of properties and beat standard local baselines on GP classification…","keywords":["active learning","epistemic uncertainty","aleatoric uncertainty","possibility theory","outer probability measures","Gaussian processes","acquisition functions","classification"],"falsifier":"Take two possibility functions that agree everywhere except on one interval, where the first is higher, and make the second higher on a larger interval outside it so its total integral is smaller; direct integration then contradicts the strict version of Property A4 that the proof relies on. In dimension $d\\ge 2$, apply $T(\\theta)=\\alpha\\theta+(1-\\alpha)c$ to a Gaussian possibility function and compute $U^\\Theta$; the change of variables contributes a Jacobian factor $\\alpha^d$, so the stated equality $U^\\Theta_{f_T}=\\alpha U^\\Theta_f$ fails.","tokens_in":21753,"feed_emoji":"🎯","tokens_out":7544,"duration_ms":75363,"temperature":0.7,"pith_summary":"The paper aims to show that epistemic uncertainty can be cleanly separated from aleatoric uncertainty by modelling the former with possibility functions instead of probability distributions. It proposes two acquisition functions: an integral measure of epistemic uncertainty in label space, and the necessity of correct classification, which measures uncertainty at the decision boundary. To deploy these in nontrivial settings, it introduces a possibilistic Gaussian process that reproduces the standard GP predictive equations. The authors report that at least one of the proposed functions outperforms the local, update-free baselines on each of 11 datasets.","feed_headline":"Possibility-based acquisition rules outperform active-learning baselines","feed_subtitle":"Both are local, update-free rules that land best or second-best in accuracy on most of 11 datasets.","key_machinery":"The central object is the outer probability measure (OPM), built from a possibility function $f_\\theta$ with supremum 1, which assigns credibility $\\bar P(A)=\\sup_{\\theta\\in A} f_\\theta(\\theta)$ and combines with a likelihood through $\\bar P(A\\times B)=\\sup_{\\theta\\in A}f_\\theta(\\theta)\\int_B p_\\theta(y)\\,dy$. This yields the uncertainty measures $U^\\Theta=\\int f_\\theta(\\theta)\\,d\\theta$, $U^Y=\\int\\sup_\\theta f_\\theta(\\theta)p_Y(y|\\theta)\\,dy-1$, and the necessity $N(E)=1-\\bar P(E^c)$. For classification, the possibilistic GP with Gaussian possibility functions and a Laplace approximation produces the same predictive mean and covariance as a standard GP, so the acquisition functions can be evaluated in closed form or with simple numerical optimization.","core_discovery":"The central claim is that combining probability and possibility theories via outer probability measures gives acquisition functions with desirable axiomatic properties and strong empirical performance in GP-based classification. Specifically, the integral measure $U^L$ and the necessity of correct classification $N$ are local and update-free, satisfy adapted versions of the properties A0--A4 proposed for epistemic-uncertainty measures, and achieve best or second-best accuracy on most of the tested datasets. The possibilistic Gaussian process is the enabling object: it provides closed-form Gaussian posterior possibility functions whose predictive equations coincide with those of standard GPs, allowing the acquisition functions to be implemented for binary and multiclass classification.","pith_inferences":["The proof issue in Property A3 suggests the rescaling property should involve $\\alpha^d$, where $d$ is the dimension of the parameter space, rather than $\\alpha$; the acquisition functions could still work empirically, but the axiomatic comparison to the Wimmer et al. properties would need reformulation.","The same probability--possibility decomposition could be carried beyond GP classification, for example to regression or bandit problems, since the paper already derives possibilistic GP regression equations that match the standard ones.","A natural test is to compare $U^L$ and $N$ against global acquisition functions such as MOCU, BEMPS, and EPIG on the same datasets; the paper only benchmarks against local, update-free methods, so its claim is about that class.","The closed-form expression for $N_{\\text{multi}}$ suggests a very cheap active learning rule for problems with many classes, which would be worth stress-testing on high-dimensional label spaces."],"forward_implications":["If the central claim is correct, active learning can target epistemic uncertainty directly without conflating it with irreducible aleatoric uncertainty, using local and update-free rules that are easy to deploy.","The possibilistic GP gives a principled way to run GP classification while interpreting the posterior as information about a fixed true function rather than as a distribution over random functions.","The necessity-based acquisition function is computationally light, reported as about 12% slower than standard acquisition functions and 9% faster than BALD, making it attractive when query selection cost matters.","The proposed measures come with a checklist of axiomatic properties, so future epistemic-uncertainty measures can be compared against the same criteria rather than evaluated only by empirical accuracy."],"supporting_citations":[{"why":"Supplies the list of desirable properties A0--A4 that the proposed epistemic-uncertainty measures are designed to satisfy.","marker":"Wimmer et al. (2023)"},{"why":"Provides the outer-probability-measure formalism used to combine probability and possibility and to define posterior possibility functions.","marker":"Houssineau (2018)"},{"why":"Supplies the Gaussian possibility function, the possibilistic notion of expectation and variance, and the asymptotic justification for these objects.","marker":"Houssineau et al. (2019)"},{"why":"Provides the GP classification framework and Laplace approximation that the possibilistic GP mirrors.","marker":"Williams and Rasmussen (2006)"},{"why":"Defines the BALD baseline used in the experiments and supplies the two synthetic 'Block' datasets.","marker":"Houlsby et al. (2011)"},{"why":"Provides the numerical possibility theory and integration rules underlying the integrals $U^\\Theta$ and $U^Y$.","marker":"De Cooman (2001)"},{"why":"Supplies the Kalman-filter identity used to solve the supremum in the possibilistic GP posterior.","marker":"Houssineau and Bishop (2018)"},{"why":"Supplies the OpenML repository from which the real-world datasets used in the evaluation are taken.","marker":"Vanschoren et al. (2014)"},{"why":"Provides the GPyTorch implementation used to build the GP models in the experiments.","marker":"Gardner et al. (2018)"}],"fun_headline_variants":["Possibility-based active learning rules outperform baselines","Local update-free acquisition rules win on most datasets","New active learning strategies improve GP classification accuracy","Bayesian-possibilistic approach yields superior acquisition rules"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof of the desirable properties assumes that making a function higher over part of its range always increases its total area, and that a certain rescaling multiplies the uncertainty by exactly $\\alpha$; under the paper's own definitions the area claim does not follow from the stated definition and the rescaling claim holds only in one dimension.","fun_headline_variants_meta":{"raw":{"variants":["Possibility-based active learning rules outperform baselines","Local update-free acquisition rules win on most datasets","New active learning strategies improve GP classification accuracy","Bayesian-possibilistic approach yields superior acquisition rules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000188,"raw_usage":{"total_tokens":1261,"prompt_tokens":804,"completion_tokens":457,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":420,"completion_tokens_details":{"reasoning_tokens":397}},"tokens_in":420,"tokens_out":457,"duration_ms":5037,"temperature":1.0,"reasoning_tokens":397,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:04:34.443308+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two possibility functions that agree everywhere except on one interval, where the first is higher, and make the second higher on a larger interval outside it so its total integral is smaller; direct integration then contradicts the strict version of Property A4 that the proof relies on. In dimension $d\\ge 2$, apply $T(\\theta)=\\alpha\\theta+(1-\\alpha)c$ to a Gaussian possibility function and compute $U^\\Theta$; the change of variables contributes a Jacobian factor $\\alpha^d$, so the stated equality $U^\\Theta_{f_T}=\\alpha U^\\Theta_f$ fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the list of desirable properties A0--A4 that the proposed epistemic-uncertainty measures are designed to satisfy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the numerical possibility theory and integration rules underlying the integrals $U^\\Theta$ and $U^Y$."},{"cited_title":"N., Bischl, B., and Torgo, L","cited_arxiv_id":null,"evidence_quote":"Supplies the OpenML repository from which the real-world datasets used in the evaluation are taken."},{"cited_title":"Q., Bindel, D., and Wilson, A","cited_arxiv_id":null,"evidence_quote":"Provides the GPyTorch implementation used to build the GP models in the experiments."}],"review_version":1}