Pith. sign in

REVIEW 3 major objections 5 minor 22 references

Efficient Precision Control in Object Detection Models for Enhanced and Reliable Ovarian Follicle Counting

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A model-agnostic threshold-selection procedure certifies precision in object detection and uses a second criterion to recover recall, demonstrated on ovarian follicle counts.

desk verdict Valuable dataset and a plausible method, but the precision guarantee is invalid as written because the p-value formula is inverted relative to the stated test statistic. read the letter →

arxiv 2501.14036 v1 pith:B3365OPH submitted 2025-01-23 cs.LG

Vincent Blot , Alexandra Lorenzo de Brionne , Ines Sellami , Olivier Trassard , Isabelle Beau , Charlotte Sonigo , Nicolas J-B. Brunel This is my paper · ORCID
classification cs.LG
keywords Distribution-freeriskcontrolMultipletestingLearnThenTestObjectdetectionPrecisionguaranteeOvarianfolliclecountingWhole-slideimagingModel-agnosticpost-processing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the uncertainty in choosing a decision threshold for object detection can be controlled so that a user-chosen precision level is guaranteed with high probability on future images. It applies the Learn Then Test (LTT) multiple-testing procedure to select an objectness threshold, then replaces the single threshold with a two-parameter rule that also uses either biological depth information or a learned false-detection classifier. On ovarian follicle counts from whole-slide histology images, the two-parameter rule keeps the precision guarantee while recovering much of the recall lost by conservative single-threshold selection, improving F1. Because the procedure is model-agnostic, the same post-processing step could improve reproducibility and trust in any object detector without retraining it.

What carries the argument

The engine is the Learn Then Test (LTT) construction: for each candidate threshold (or pair), define the null hypothesis that expected precision is at most $P_0$, compute p-values from the per-image precision losses using the Hoeffding-Bentkus inequality, and keep only thresholds whose hypotheses survive a family-wise-error-rate-controlling procedure (fixed-sequence testing). The second ingredient is the two-parameter post-processing operator that adds a second criterion: ovary depth, computed by dilating the ovary contour, or the score of a small auxiliary convolutional classifier trained to distinguish true from false detections. This enlarges the search space of statistically compatible thresholds, which is what lets recall increase without losing the precision guarantee.

What would settle it

Re-run the LTT calibration on the released dataset with a fixed, explicit patch-aggregation and matching rule, then count across many resplits how often average precision on the test cuts falls below $P_0 = 0.4$; if the fraction below target exceeds $\delta = 10^{-3}$ for any reasonable aggregation choice, the claim is not robust to the paper's unspecified preprocessing.

Watch

Extended reading notes

Core claim

The central claim is that a post-processing decision operator $T_{\lambda,\mu}(b) = \{b_i : c_i \geq \lambda \text{ and } d_i \leq \mu\}$ (or with a classifier score $g(b_i) \geq \mu$ instead of depth) selected through LTT satisfies $P_D(E_{P_new}[\mathrm{Precision}(b_{\mathrm{new}}, T_{\lambda^*,\mu^*}(\hat{f}(b_{\mathrm{new}})))] \geq P_0) \geq 1-\delta$, exactly as the single-threshold operator does, while yielding higher recall and F1 than both naive thresholding and LTT on objectness alone. The guarantee follows from treating each candidate threshold pair as a null hypothesis and controlling the family-wise error rate with fixed-sequence testing; the recall gain comes from searching a larger two-dimensional grid of compatible thresholds. The paper supports this with experiments on 643 mouse ovary sections, where the two-parameter rules meet the precision target $P_0 = 0.4$ with probability at least $1-10^{-3}$ across 100 calibration/test splits, whereas the naive threshold falls below target roughly half the time.

Load-bearing premise

The guarantee stands on treating $\mathrm{Precision}(b_i, T_\lambda(\hat{f}(x_i)))$ as a well-defined, measurable function for each cut image, but the paper never specifies how predictions from overlapping patches are merged into cut-level boxes, what IoU matching rule links predicted to ground-truth follicles, or what precision is when no box survives thresholding; if any of these undefined choices changes the loss values, Eq. (2) cannot be verified or reproduced as stated.

Editorial extensions

If this is right

  • If the guarantee holds, a lab can fix $P_0$ in advance and trust that, over calibration randomness, the expected precision on a new slide stays at least $P_0$, making repeated follicle counts comparable across studies.
  • The two-parameter selection rule needs no retraining of the detector; any existing detector can be wrapped with this post-processing step and an auxiliary cue, so performance gains transfer across architectures such as EfficientDet and YOLO.
  • The same LTT machinery extends directly to more than two hyperparameters: any finite grid of post-processing choices can be tested similarly, so richer contextual rules (e.g., size, shape, stain intensity) could be added without changing the framework.
  • Using a learned false-detection classifier as the second criterion shows that purely data-driven cues can replace hand-crafted biological ones while preserving the precision guarantee, which makes the method portable to detection problems outside histology.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An immediate testable extension is to check sensitivity of the selected thresholds to the choice of patch aggregation (e.g., non-maximum suppression across overlapping $1000 \times 1000$ patches) and to the IoU matching rule, since these choices determine the exact precision values that enter the LTT p-values.
  • Because the auxiliary classifier is trained on the training set and then used to filter boxes at calibration and test time, its own generalization error is not covered by the LTT guarantee; measuring how much the guarantee degrades with classifier accuracy would be a direct extension.
  • The depth-based rule relies on ovary contour quality; a contour-free context cue (such as a distance transform from tissue segmentation) might provide the same recall gain while being less sensitive to contour errors, which the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes a post-processing threshold selection procedure for object detection in whole-slide ovarian follicle images. The authors apply the Learn Then Test (LTT) framework to select an objectness threshold that, with probability at least 1−δ over the calibration set, guarantees an expected precision of at least P0 on new images, and they extend this to two-parameter decision rules that also use follicle depth or an auxiliary classifier score to improve recall and F1. Experiments on a newly released dataset compare the methods with EfficientDet and YOLOv8 across 100 calibration/test splits.

Significance. If correct, the procedure would be a useful model-agnostic way to turn an uncalibrated detector into one with a distribution-free precision guarantee without retraining, and the release of code and data is a valuable contribution. However, the central statistical derivation in §2.2 has a direction error: the reported Hoeffding-Bentkus p-value is applied to a precision (larger is better) instead of a risk (smaller is better), so the p-values and the guarantee in Eq. (2) are not valid as written. The experimental results therefore do not currently support the main claim. The paper is an application of existing LTT theory rather than a new theoretical contribution, and its significance depends on correcting the p-value formulation and clarifying the evaluation protocol.

major comments (3)
  1. [§2.2, Eq. (1)] The text states that the test statistic is (1/n)Σ_i Precision(b_i, T_λ^{(1)}(f-hat(b_i))) but never defines R̂_j or α. If, as the notation suggests, R̂_j is the sample mean precision and α = P0, then for any λ_j with observed precision above P0, R̂_j∧α = P0, so h1(R̂_j∧α, α) = 0 and P(Bin(n, α) ≤ ⌈n R̂_j⌉) ≈ 1, yielding p_j ≈ 1. Hence the p-value is largest precisely when the data most strongly support rejecting H_{1,λ_j}^0: P(1)(λ_j) ≤ P0, and the LTT selection would return the wrong compatible set. The Hoeffding-Bentkus inequality is valid for a risk that is small in the direction of rejection; the risk should be 1 − Precision with α = 1 − P0. The manuscript must specify R̂_j and α, state the risk formulation, and re-derive the guarantee in Eq. (2). As written, Eq. (2) does not follow from the stated procedure.
  2. [§3.1, §2.1] The paper does not define how per-patch predictions are aggregated into the cut-level precision used in the p-values. The calibration set is described as 35 ovary cuts, and the OD model is run on 1000×1000 patches with stride 500, but the mapping from overlapping patch detections to the set of boxes for a cut is never specified, nor is the matching rule between predicted and ground-truth follicles (e.g., IoU threshold), nor the convention when no boxes survive T_λ. Without a well-defined measurable loss Precision(b_i, T_λ(f-hat(b_i))) on the calibration data, the computed p-values and the guarantee in Eq. (2) are not reproducible.
  3. [§2.3, Table 1] The grids ΛO, Λ′, the number of thresholds m, and m′ are not fully specified. The paper reports selected thresholds such as (0.642, 0.515) for depth and (0.534, 0.230) for classification, but the reader cannot determine the multiple-testing correction applied or the search space. This matters because FST's conservativeness depends on the ordered grid and the number of starting points m′, and the guarantee is only over the tested set Λ(2). Please provide the grid definitions and the FST details in the main text or a complete supplementary.
minor comments (5)
  1. [Throughout] The text contains several instances where 'LLT' is written instead of 'LTT' (e.g., §2.2, 'selected by the LLT procedure'); these typos should be corrected.
  2. [§2.3, §3.2] The manuscript has unresolved '??' placeholders for the depth computation, detailed performance tables, and training parameters in the supplementary material; these must be filled in before the claims can be assessed.
  3. [§2.2] The text first says the approach controls the False Discovery Rate, but then defines and uses family-wise error rate control via Fixed Sequence Testing; the terminology should be harmonized to avoid confusion.
  4. [§2.1] The notation 'Precision(b, ˆb)' is used before the arguments are defined; the matching rule (e.g., IoU threshold) between predicted and ground-truth boxes should be specified in the main text.
  5. [Abstract] The phrase 'overperforming way' is awkward and should be rephrased for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the LTT guarantee is applied from external prior work to held-out calibration thresholds, with no fitted quantity relabeled as a prediction.

full rationale

The paper's central claim is that applying the Learn-Then-Test (LTT) procedure of Angelopoulos et al. [4] and the Hoeffding-Bentkus p-values of Bates et al. [6] to select decision thresholds on a calibration set yields a probabilistic precision guarantee on new images. This is a direct application of external, published theorems; the authors do not re-derive these results, and neither [4] nor [6] shares authors with the present paper. The target precision P0 and confidence delta are user-defined, not fitted from data, and the selected thresholds are evaluated on independent test splits (100 random calibration/test splits at the mouse level), so no fitted parameter is renamed as a prediction. The contextual operators T_{lambda,mu} and the auxiliary classifier g are defined and trained before threshold selection and do not encode test-set outcomes. The only self-citations (refs. [19,20]) support biological background and a prior counting method, not the load-bearing statistical argument. Underspecified details (patch-to-cut prediction aggregation, the box-matching rule, and whether Eq. (1)'s p-value direction matches the stated null) are genuine reproducibility and correctness concerns, but they are not instances of circularity: the derivation does not reduce to its own inputs by construction, and the statistical guarantee rests on externally established results rather than on a self-citation chain or a fitted-input prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces engineered features (depth, auxiliary classifier) but no new physical or mathematical entities; the main load-bearing inputs are user-chosen parameters and assumptions about data exchangeability and loss definition.

free parameters (3)
  • Target precision P0 = 0.4
    User-set target precision that defines the null hypotheses in Section 2.2; all guarantees and threshold selections are reported for this value.
  • Confidence level δ = 10^-3
    User-set error level for the probabilistic guarantee in Eq. (2).
  • Threshold grid step = 0.1 (grids ΛO = {0.1,...,0.9})
    Discretization of the threshold space, chosen by hand; the guarantee only covers the testable grid points.
assumptions (4)
  • standard math Learn Then Test theorem (Angelopoulos et al. 2021) and Hoeffding-Bentkus inequality (Bates et al. 2021)
    The central guarantee Eq. (2) is imported from these cited works rather than proved; the paper relies on their validity and on the p-values being correctly computed.
  • domain assumption Calibration and test images are exchangeable from the same distribution
    The LTT guarantee requires this; the authors split by mouse to mitigate similarity, but the assumption is not explicitly tested.
  • domain assumption Cut-level precision is a well-defined, bounded function of the image, the detector, and the threshold
    The per-image loss used in the p-value computation requires a precise definition of box merging across patches, matching to ground truth, and handling of zero-detection images; the manuscript does not provide these definitions.
  • domain assumption An ovary contour is available for every image when using the depth filter
    The depth criterion in Section 2.3 requires the ovary contour; the paper defers the computation to missing supplementary material and does not state whether it is manual or automatic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Precision Control in Object Detection Models for Enhanced and Reliable Ovarian Follicle Counting." pith.science (2026). https://pith.science/paper/B3365OPH

@misc{pith2026250114036,
  author       = {Pith},
  title        = {Pith review of: Efficient Precision Control in Object Detection Models for Enhanced and Reliable Ovarian Follicle Counting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B3365OPH}},
  note         = {Machine review of arXiv:2501.14036}
}
read the original abstract

Image analysis is a key tool for describing the detailed mechanisms of folliculogenesis, such as evaluating the quantity of mouse Primordial ovarian Follicles (PMF) in the ovarian reserve. The development of high-resolution virtual slide scanners offers the possibility of quantifying, robustifying and accelerating the histopathological procedure. A major challenge for machine learning is to control the precision of predictions while enabling a high recall, in order to provide reproducibility. We use a multiple testing procedure that gives an overperforming way to solve the standard Precision-Recall trade-off that gives probabilistic guarantees on the precision. In addition, we significantly improve the overall performance of the models (increase of F1-score) by selecting the decision threshold using contextual biological information or using an auxiliary model. As it is model-agnostic, this contextual selection procedure paves the way to the development of a strategy that can improve the performance of any model without the need of retraining it.

Figures

Figures reproduced from arXiv: 2501.14036 by the authors.

Figure 1
Figure 1. Example of an ovary cut with a zoom on an annotated area with PMF [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Computation of the depth of a box. The box is predicted by the OD model [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Precision, Recall and F1-score for target precision [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 19 canonical work pages

  1. [1]

    Cancers 14(21) (2022)

    Ahmed, A.A., Abouzid, M., Kaczmarek, E.: Deep learning approaches in histopathology. Cancers 14(21) (2022)

    work page 2022

  2. [2]

    Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling

    And´ eol, L., Fel, T., De Grancey, F., Mossina, L.: Confident object detection via conformal prediction and conformal risk control: an application to railway signal- ing. arXiv:2304.06052 (2023)

    work page Pith review arXiv 2023

  3. [3]

    Foun- dations and Trends® in Machine Learning 16(4), 494–591 (2023)

    Angelopoulos, A.N., Bates, S.: Conformal prediction: A gentle introduction. Foun- dations and Trends® in Machine Learning 16(4), 494–591 (2023)

    work page 2023

  4. [4]

    arXiv:2110.01052 (2021)

    Angelopoulos, A.N., Bates, S., Cand` es, E.J., Jordan, M.I., Lei, L.: Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv:2110.01052 (2021)

    arXiv 2021

  5. [5]

    arXiv:2208.02814 (2023)

    Angelopoulos, A.N., Bates, S., Fisch, A., Lei, L., Schuster, T.: Conformal risk control. arXiv:2208.02814 (2023)

    arXiv 2023

  6. [6]

    Journal of ACM 68(6) (2021)

    Bates, S., Angelopoulos, A., Lei, L., Malik, J., Jordan, M.: Distribution-free, risk- controlling prediction sets. Journal of ACM 68(6) (2021)

    work page 2021

  7. [7]

    In: IEEE International Con- ference on Cyberworlds

    Cai, H., Wu, Q., Ando, H., Watanabe, R.: Automatic counting of follicles in neona- tal mice ovarian section images by using deeplabv3+. In: IEEE International Con- ference on Cyberworlds. pp. 145–148 (2020)

    work page 2020

  8. [8]

    In: IEEE Conference on Computer Vision and Pattern Recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (2009)

    2009

Show all 22 references
  1. [9]

    Springer, 1st edn

    Dudoit, S., Van Der Laan, M.J., van der Laan, M.J.: Multiple testing procedures with applications to genomics. Springer, 1st edn. (2008)

    2008

  2. [10]

    Molecular Human Reproduction 27(1), gaaa084 (2021)

    Huang, C.C., Chou, C.H., Yang, Y.S., Ho, H.N., Shun, C.T., Wen, W.F., Chen, S.U., Chen, M.J.: Metformin: a novel promising option for fertility preservation during cyclophosphamide-based chemotherapy. Molecular Human Reproduction 27(1), gaaa084 (2021)

    2021

  3. [11]

    Computers in biology and medicine 112, 103350 (2019)

    ˙Inik, ¨O., Ceyhan, A., Balcıo˘ glu, E.,¨Ulker, E.: A new method for automatic counting of ovarian follicles on whole slide histological images based on convolutional neural network. Computers in biology and medicine 112, 103350 (2019)

    2019

  4. [12]

    com/ultralytics/ultralytics

    Jocher, G., Chaurasia, A., Qiu, J.: Ultralytics YOLO (2023), https://github. com/ultralytics/ultralytics

    2023

  5. [13]

    In: 3rd IAPR Asian Conference on Pattern Recognition

    Liu, S., Deng, W.: Very deep convolutional neural network based image classifica- tion using small training sample size. In: 3rd IAPR Asian Conference on Pattern Recognition. pp. 730–734 (2015)

    2015

  6. [14]

    Journal of Pathology Informatics 12(1), 42 (2021)

    Mehrvar, S., Himmel, L.E., Babburi, P., Goldberg, A.L., Guffroy, M., Janardhan, K., Krempley, A.L., Bawa, B.: Deep learning approaches and applications in toxico- logic histopathology: Current status and future perspectives. Journal of Pathology Informatics 12(1), 42 (2021)

    2021

  7. [15]

    Reproduction 17(3), 555–557 (1968)

    Pedersen, T., Peters, H.: Proposal for a classification of oocytes and follicles in the mouse ovary. Reproduction 17(3), 555–557 (1968)

    1968

  8. [16]

    Trends in Endocrinology and Metabolism21(2), 96–103 (2010)

    Reddy, P., Zheng, W., Liu, K.: Mechanisms maintaining the dormancy and survival of mammalian primordial follicles. Trends in Endocrinology and Metabolism21(2), 96–103 (2010)

    2010

  9. [17]

    Journal of Ovarian Research 13, 121 (2020)

    Sarma, U., Winship, A., Hutt, K.: Comparison of methods for quantifying primor- dial follicles in the mouse ovary. Journal of Ovarian Research 13, 121 (2020)

    2020

  10. [18]

    The MIT Press (2012)

    Schapire, R.E., Freund, Y.: Boosting: Foundations and Algorithms. The MIT Press (2012)

    2012

  11. [19]

    International Journal of Molecular Sciences 20, 5342 (2019) Efficient Precision control 11

    Sonigo, C., Beau, I., Binart, N., Grynberg, M.: The impact of chemotherapy on the ovaries: Molecular aspects and the prevention of ovarian damage. International Journal of Molecular Sciences 20, 5342 (2019) Efficient Precision control 11

    2019

  12. [20]

    Scientific reports 8(1), 13499 (2018)

    Sonigo, C., Jankowski, S., Yoo, O., Trassard, O., Bousquet, N., Grynberg, M., Beau, I., Binart, N.: High-throughput ovarian follicle counting by an innovative deep learning approach. Scientific reports 8(1), 13499 (2018)

    2018

  13. [21]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Tan, M., Pang, R., Le, Q.V.: Efficientdet: Scalable and efficient object detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    2020

  14. [22]

    Reproductive biology and endocrinology : RB&E 1, 11 (2003)

    Tilly, J.: Ovarian follicle counts-not as simple as 1, 2, 3. Reproductive biology and endocrinology : RB&E 1, 11 (2003)

    2003

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.