pith. sign in

arxiv: 1907.02015 · v1 · pith:3UOCBTVFnew · submitted 2019-07-03 · 💻 cs.LG · stat.ML

libconform v0.1.0: a Python library for conformal prediction

Jonas Fassbender This is my paper

Pith reviewed 2026-05-25 10:20 UTC · model grok-4.3

classification 💻 cs.LG stat.ML
keywords conformal predictionpython librarymachine learningprediction intervalssoftware packageuncertainty quantification
0
0 comments X

The pith

libconform v0.1.0 is a Python library that implements the main algorithms of the conformal prediction framework and documents its API.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces libconform v0.1.0, a Python library for conformal prediction released under the MIT license. It describes the main algorithms the library contains and provides documentation for its application programming interface. It also covers some implementation details along with notes on changes planned for future versions. A reader might care because the library offers packaged access to conformal prediction methods in a widely used programming language, even though the release is marked as not yet stable.

Core claim

This paper introduces libconform v0.1.0, a Python library for the conformal prediction framework, licensed under the MIT-license. libconform is not yet stable. This paper describes the main algorithms implemented and documents the API of libconform. Also some details about the implementation and changes in future versions are described.

What carries the argument

The libconform library itself, which packages conformal prediction algorithms into an importable Python module with a documented public API.

Load-bearing premise

The library correctly and completely implements the conformal prediction algorithms it claims to contain.

What would settle it

Execution of the library's functions on standard conformal prediction test cases that produces outputs inconsistent with the known mathematical behavior of those algorithms.

Figures

Figures reproduced from arXiv: 1907.02015 by Jonas Fassbender.

Figure 1
Figure 1. Figure 1: Confusion matrix for binary abstention classifier [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
read the original abstract

This paper introduces libconform v0.1.0, a Python library for the conformal prediction framework, licensed under the MIT-license. libconform is not yet stable. This paper describes the main algorithms implemented and documents the API of libconform. Also some details about the implementation and changes in future versions are described.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces libconform v0.1.0, a Python library for the conformal prediction framework licensed under the MIT license. It describes the main algorithms implemented, documents the public API, provides some implementation details, and notes planned changes for future versions. The library is explicitly stated to be not yet stable.

Significance. If the library correctly implements the claimed conformal prediction methods, it could serve as a practical, open-source resource for applying and experimenting with conformal prediction in Python-based machine learning workflows. As a v0.1.0 release without stability guarantees, benchmarks, or verification artifacts, however, its significance remains limited to facilitating community inspection rather than providing a production-ready tool.

major comments (1)
  1. [Abstract] Abstract, paragraph 1: the central claim that libconform implements the conformal prediction framework rests on the unverified assumption that the library 'correctly and completely' realizes the algorithms; the manuscript contains no test cases, empirical checks, or reproducibility artifacts to support this.
minor comments (2)
  1. The high-level algorithm descriptions would be clearer if they explicitly named the specific conformal prediction variants (e.g., inductive conformal prediction) and their key parameters.
  2. Inclusion of minimal usage examples or API call snippets would improve accessibility for readers intending to adopt the library.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and the opportunity to respond. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract, paragraph 1: the central claim that libconform implements the conformal prediction framework rests on the unverified assumption that the library 'correctly and completely' realizes the algorithms; the manuscript contains no test cases, empirical checks, or reproducibility artifacts to support this.

    Authors: The manuscript is a descriptive paper for a v0.1.0 library release that is explicitly stated to be unstable. Its abstract and body limit themselves to describing the algorithms that were implemented and documenting the public API; they do not assert empirical correctness, completeness, or production readiness. Because the source is released under an open MIT license, any reader can inspect, execute, or extend the code. Adding test suites or reproducibility artifacts would shift the manuscript from documentation to verification, which lies outside its stated scope. We therefore see no need to revise the text on this point. revision: no

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The manuscript is a library release announcement and API documentation for libconform v0.1.0. It describes implemented algorithms at a high level and documents the public interface without advancing a novel theoretical claim, derivation, or empirical result. No mathematical identities, scaling arguments, or statistical guarantees are asserted that would require external validation beyond code inspection. The central assertion reduces to faithful description of existing conformal prediction methods plus software engineering details; the paper is self-contained against external benchmarks and contains no load-bearing derivations or fitted quantities.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities are introduced because the paper contains no derivations or new theoretical claims.

pith-pipeline@v0.9.0 · 5567 in / 960 out tokens · 30193 ms · 2026-05-25T10:20:07.076610+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages · 1 internal anchor

  1. [1]

    doi: 10.1145/2358 15.235821

    ISSN 0098-3500. doi: 10.1145/2358 15.235821. URL http://doi.acm.org/10.1145/235815.235821. Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedr egosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramf ort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Ga¨ elVaroquaux. API design for mac...

    work page doi:10.1145/2358

  2. [2]

    doi: 10.1016/j.patrec .2005.10.010

    ISSN 0167-8655. doi: 10.1016/j.patrec .2005.10.010. URL http://dx.doi.org/10.1016/j.patrec.2005.10.010. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learn- ing. Springer, second edition,

    work page doi:10.1016/j.patrec 2005

  3. [3]

    D., 2007, @doi [Computing in Science and Engineering] 10.1109/MCSE.2007.55 , 9, 90

    doi: 10.1109/MCSE.2007.55. Eric Jones, Travis Oliphant, Pearu Peterson, et al. SciPy: O pen source scientific tools for Python, 2001–2019. URL http://www.scipy.org/. [Online; accessed 14.06.2019]. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W . Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code re cognition. Neural...

    work page doi:10.1109/mcse.2007.55 2007

  4. [4]

    Backpropagation applied to handwritten zip code recognition,

    ISSN 0899-7667. doi: 10.1162 /neco.1989.1.4.541. URL http://dx.doi.org/10.1162/neco.1989.1.4.541. Ilia Nouretdinov, Tom Melluish, and Vladimir Vovk. Ridge re gression confidence machine. In In Proceedings of the Eighteenth International Conference on M achine Learning, pages 385–392. Morgan Kaufmann,

    work page doi:10.1162/neco.1989.1.4.541 1989

  5. [5]

    Inductive confidence machines for regression

    Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Al ex Gammerman. Inductive confidence machines for regression. In Tapio Elomaa, Heikki Mannila, and Hannu Toivo- nen, editors, Machine Learning: ECML 2002 , pages 345–356, Berlin, Heidelberg,

    work page 2002

  6. [6]

    doi: 10.1109/ICTAI.2007.47

    ISBN 978- 0-7695-3015-4. doi: 10.1109/ICTAI.2007.47. Craig Saunders, Alex Gammerman, and Vladimir Vovk. Transdu ction with confidence and credibility

    work page doi:10.1109/ictai.2007.47 2007

  7. [7]

    Stijn Vanderlooy, Ida G

    doi: 10.3233/IDA-2009-0400. Stijn Vanderlooy, Ida G. Sprinkhuizen-Kuyper, Evgueni N. S mirnov, and H. Jaap van den Herik. The roc isometrics approach to construct reli able classi- fiers. Intell. Data Anal. , 13(1):3–37, January

    work page doi:10.3233/ida-2009-0400 2009

  8. [8]

    URL http://dl.acm.org/citation.cfm?id=1551758.1551760

    ISSN 1088-467X. URL http://dl.acm.org/citation.cfm?id=1551758.1551760. Vladimir Vovk. Cross-conformal predictors. Annals of Mathematics and Artificial Intelli- gence, 74, 08

    work page arXiv

  9. [9]

    Vladimir Vovk and Ivan Petej

    doi: 10.1007/s10472-013-9368-4. Vladimir Vovk and Ivan Petej. Venn-abers predictors. In Proceedings of the Thirti- eth Conference on Uncertainty in Artificial Intelligence , UAI’14, pages 829–838, Ar- lington, Virginia, United States,

    work page doi:10.1007/s10472-013-9368-4

  10. [10]

    ISBN 97 8-0-9749039-1-0

    AUAI Press. ISBN 97 8-0-9749039-1-0. URL http://dl.acm.org/citation.cfm?id=3020751.3020837. Vladimir Vovk, Alex Gammerman, and Craig Saunders. Machine -learning applications of algorithmic randomness

    work page arXiv

  11. [12]

    URL http://arxiv.org/abs/1603.04416. 37

    work page internal anchor Pith review Pith/arXiv arXiv