Pith. sign in

REVIEW 4 major objections 5 minor 41 references

GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SliceLens reformulates error slice discovery as grounded hypothesis generation and VLM verification, and claims it lifts detection Precision@10 from 0.31 to 0.73 on the new FeSD benchmark.

desk verdict A promising generate-and-verify framework for instance-level slice discovery, but the FeSD headline number is internally inconsistent and rests on an unspecified 'best slice' selection, so the SOTA claim needs scrutiny before being trusted. read the letter →

arxiv 2512.24592 v3 pith:26YO3WNT submitted 2025-12-31 cs.CV

Wei Zhang , Chaoqun Wang , Zixuan Guan , Ping Sheng Kao , Pengfei Zhao , Peng Wu , Sifeng He This is my paper
classification cs.CV
keywords errorslicediscoveryinstance-levelvisionobjectdetectioninstancesegmentationvision-languagemodelhypothesisgenerationgroundedverificationrepair
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to find the subsets of examples on which a vision model systematically fails — error slices — for instance-level tasks like detection and segmentation, not just image classification. It claims these failures are often relational and spatially local (e.g., 'bicycles partially occluded by a person'), so the right unit is the instance, not the image. To capture them, SliceLens generates hypotheses from an LLM (world knowledge plus data-driven attribute inference), retrieves candidate error regions by asking a VLM yes/no questions grounded by boxes or points, and verifies a slice by checking that error rate rises with slice confidence. On the paper's new FeSD benchmark, it reports Precision@10 of 0.73 for detection (vs. 0.31 for the adapted HiBug baseline) and 0.805 for segmentation. If correct, this turns slice discovery from clustering into a testable hypothesis-verification loop and makes discovered slices directly usable for model repair.

What carries the argument

The load-bearing object is the grounded VLM verification head: the score P(yes|q,r) = softmax over the 'yes' and 'no' logits of a prompt that names the hypothesis q and localizes the error region r with a bounding box (detection) or point (segmentation). Around it sits the hypothesis generator (knowledge-driven LLM prompts plus a data-driven path: grounded attribute-aware captions → LLM attribute inference → LLM refinement into natural-language query phrases) and the slope-trend verifier, which replaces thresholding error rates with a sliding-window local linear fit between slice confidence and error rate, keeping slices whose maximum positive slope is large. Together they let SliceLens sear

What would settle it

Take the FeSD detection slices and for each error region have human annotators label whether the region satisfies the corresponding relational hypothesis; then ask Qwen2.5-VL-7B the same box-prompted question. If human-VLM agreement is at or near chance on relational hypotheses (e.g., occlusion, part-whole), while agreement on simple attributes is high, then the slice-confidence ranking in Eq. (1) is not evidence about the hypothesis and the reported P@10 improvements would not transfer. Alternatively, run SliceLens with a VLM that always answers 'yes': if Precision@10 stays high, the groundin

Watch

Extended reading notes

Core claim

SliceLens claims that fine-grained instance-level error slices can be discovered by a generate-and-verify loop: an LLM proposes natural-language failure hypotheses; a VLM scores each error region's match to the hypothesis using box or point grounding and the softmax of 'yes'/'no' logits (Eq. 1); and a slope-trend analysis — not a fixed error-rate threshold — decides whether the retrieved slice is a genuine systematic error by requiring error likelihood to rise as slice confidence rises. On FeSD, a benchmark of 42 expert-annotated slices over COCO, KITTI, and public face imagery, the method reports average Precision@10 of 0.729 across 21 detection slices (21/21 matched) and 0.805 across 21 se

Load-bearing premise

The whole pipeline rests on VLM reliability for grounded relational questions: a 7B VLM, prompted with only a box or point, must correctly answer 'yes' or 'no' to complex queries such as 'bicycle obscured by a person'; the paper does not independently validate this grounding accuracy.

Editorial extensions

If this is right

  • Instance-level grounding is what unlocks detection/segmentation slices: image-level analysis dilutes failures (P@10 0.60 vs 0.80; slope 0.84 vs 1.32 on the partial-face slice).
  • Language-based hypotheses are more compact and semantically aligned than tag combinations: 102 hypotheses reach 0.90 recall / 0.74 precision, versus 38,639 tag combinations at 0.57/0.54.
  • The slope-trend test yields higher F1 (3.7% average improvement) and robustness to threshold choice than error-rate thresholding.
  • Discovered slices support targeted repair: instance-level GroupDRO on the 'bicycle occluded by person' slice improves bicycle mAP from 27.2 to 33.59 and mAR from 39.33 to 47.09.
  • On classification benchmarks the same pipeline matches or beats prior slice-discovery methods, reaching 1.0 Precision@10 on Waterbirds and CelebA and 0.64/0.66/0.69 on NICO++ across correlation strengths.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If VLM grounding is trustworthy, the same hypothesis-generate-verify loop could transfer to other fine-grained instance tasks (pose, tracking, video) and to open-vocabulary failure-mode search, since the query language is unrestricted natural language.
  • The slope-trend verifier is a domain-general criterion for 'slice-ness' that could sit on top of any retriever that produces a ranking — a clean way to test whether a ranking corresponds to a real error mode, independent of VLM scores.
  • The FeSD benchmark's 42 slices are only a sample of failure modes; high Precision@10 mostly reflects retrieval of known, annotated slices, and a full evaluation of discovery completeness would need a held-out 'unknown slice' protocol, which the paper leaves to future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript (whose full text uses the name SliceLens, while the preceding abstract uses GH-ESD/GESD) proposes a hypothesis-driven error-slice discovery method for instance-level vision tasks. SliceLens generates failure hypotheses from an LLM in two ways — knowledge-driven, from task context, and data-driven, from VLM-generated grounded captions — and then verifies them by using a VLM to compute P(yes | q, r) for each error region, followed by a slope-based trend analysis that links slice confidence to model error. The paper also introduces FeSD, a new expert-annotated benchmark of fine-grained detection/segmentation error slices, and reports substantial Precision@10 gains over adapted FACTS and HiBug baselines, as well as model-repair experiments in both classification and object detection.

Significance. If the reported results are reliable, the paper would make a meaningful contribution: instance-level, relational error slices such as “bicycle obscured by a person” are not well captured by image-level tag or cluster methods, and the proposed generate-and-verify pipeline is well motivated. The paper is also commendable for attempting a new benchmark targeting detection and segmentation, for reporting detailed per-slice tables in the appendix, and for including repair experiments that test actionability. However, the central FeSD comparison is currently not reproducible because the benchmark is not released and, more importantly, the protocol for selecting the “Best Predicted Slice” in Table 7 is unspecified. Since the headline 0.42 gain rests on this protocol, the significance cannot be assessed until the evaluation is fully specified.

major comments (4)
  1. [§5.2, Table 7, Eq. (2)] The FeSD Precision@10 comparison is not well-defined. Table 7 reports, for each ground-truth slice, a “Best Predicted Slice” and its P@10, and Table 2 averages these values to 0.73. The text never states how the “best” slice is chosen. If it is the hypothesis with maximum P@10 among SliceLens’s 102 hypotheses, then 0.73 is an oracle upper bound, not the precision of a deployed pipeline, and methods with more hypotheses are favored by selection alone. If it is chosen by the Gemini semantic-relevance evaluator of §5.3.1, then the reported metric is not the P@10 of Eq. (2) but a semantic-matching variant. The comparison to FACTS* and HiBug* is valid only if the same selection rule is applied to those baselines; this is not documented. Since FeSD is not released, a reader cannot verify the protocol. This issue is load-bearing for the central claim and must be fixed by specifying and applying
  2. [Abstract and §5.2 / Table 2] The headline result is internally inconsistent. The abstract states “improving Precision@10 by 0.10 (0.73 vs. 0.63) on the GESD benchmark,” while the full text, §5.2 and Table 2, report 0.73 vs. 0.31/0.28, an improvement of 0.42 above HiBug*. These are incompatible versions of the same experiment. A journal submission must have one unambiguous headline number; the discrepancy also makes it unclear whether the authors are comparing against a 0.63 baseline (perhaps an earlier version of their own benchmark) or the open-source baselines. This needs correction in the next version.
  3. [§5.3.1, Appendix A.3] The semantic-relevance evaluation appears circular and is insufficiently validated. The evaluator that judges whether a discovered slice matches a ground-truth slice is Gemini-2.5-Pro, the same LLM family used to generate the hypotheses. The prompt permits loose matches (e.g., “bicycle partially occluded by a person” is listed as the best match for the GT “Bicycles seen from the front/back that being ridden/pushed by people”), and no human agreement is reported. If this evaluator is also used to select the “Best Predicted Slice” in Table 7, the reported P@10 is not model-error precision but the output of a semantic judge. Please report inter-annotator agreement with human experts and make the matching criterion explicit and identical for all compared methods.
  4. [§3.2.1, Appendix B.1] The pipeline relies on the assumption that a VLM can answer grounded yes/no questions about complex relational hypotheses from an error-region bounding box or point. This is not independently validated on the FeSD setting. The VLM ablation in Appendix B.1 is on NICO++ image classification, which uses global image-level decisions and simpler queries; it does not measure whether Qwen2.5-VL-7B can reliably judge descriptions like “bicycle obscured by a person” from a box. Since P(yes|q,r) is the ranking signal that feeds Eq. (1) and all downstream slice identification, the paper should report a small human-labeled validation set for FeSD-like grounded queries, or otherwise quantify VLM grounding accuracy.
minor comments (5)
  1. [Throughout] The manuscript contains inconsistent naming: the top abstract uses GH-ESD and GESD, while the full text uses SliceLens and FeSD. The title also differs between the two versions. The authors should unify terminology and ensure the abstract matches the body.
  2. [§4.1, Step 4] Overwriting all non-slice model predictions with ground truth makes the benchmark only contain the predefined error modes, potentially making slice discovery artificially clean. Please discuss why this cleaning does not inflate the reported precision relative to a real deployment setting with many unrelated errors.
  3. [Eq. (1), §3.2.2] The paper uses “slice confidence” for P(yes|q,r) but later slopes are computed between confidence and error rate. The exact ranking and thresholding procedure for retrieving the top k instances in a candidate slice should be stated in a single place; currently it is spread across §3.2.1, §3.2.2, and Eq. (2).
  4. [Table 4] Typographical inconsistency: “GroupDRO + Slicelens” capitalizes different parts of the method name. Also, the caption could state more clearly what “Baseline model repair” means and how it differs from “No model repair.”
  5. [Appendix B.2] The HiBug* baseline implementation details only list the attribute corpus; the actual tag-generation, tag-combination, and slice-selection procedure used to produce the 0.31 P@10 number is not described. Please provide the full protocol, ideally with code, so the comparison is reproducible.

Circularity Check

2 steps flagged · score 4.0 of 10

No derivation-level self-definition, but the FeSD headline precision is a per-GT 'Best Predicted Slice' statistic and the semantic-evaluation ablation uses the same LLM as generator and judge; these are partial self-references, not full circularity.

  1. fitted input called prediction [Section 5.2, Table 2, Appendix B.4.1 Table 7, Eq. (2)]
    "Table 7: ... 'Best Predicted Slice' ... 'Average Precision@10 0.729'; Section 5.2: 'SliceLens consistently outperforms both baseline methods across all error categories, achieving an average precision@10 of 0.73 compared to HiBug* (0.31) and FACTS* (0.28).'"

    Eq. (2) defines Precision@k for one predicted slice S_pred. The FeSD result instead averages the per-GT 'Best Predicted Slice' P@10 values shown in Table 7. No selection rule for 'best' is given. If the best slice is chosen by matching to the GT slice (or by the highest P@10 against it), the 0.73 is an oracle/max-over-hypotheses statistic: the 'prediction' is selected after seeing the GT, so the reported precision is partly constructed from the label rather than produced by the pipeline. If the best is chosen by the Gemini-2.5-Pro semantic evaluator, the selection and scoring are made by the same model that generated the hypotheses (Sec. 5.3.1), which is again self-referential. In either reading the headline is not the plain Precision@k of Eq. (2).

  2. self definitional [Section 5.1.2 and Section 5.3.1 / Appendix A.3]
    "Section 5.1.2: 'Our SliceLens implementation employs Gemini-2.5-Pro as LLM and Qwen-2.5-VL-7B as VLM...' Section 5.3.1: 'an LLM-based (Gemini-2.5-pro) evaluator determines whether a discovered slice is semantically related to any ground-truth slice.'"

    The semantic-relevance measurement used to support the hypothesis-generation ablation is performed by the same LLM family that generated the hypotheses. The 'correctness' of a hypothesis is therefore defined as agreement with the recommender's own prior, so the reported recall/precision (0.90/0.74) is partly a self-match. This does not invalidate the main FeSD GT-based precision, but it makes the ablation's superiority claim over tag-based hypotheses partially circular.

full rationale

The core SliceLens pipeline — LLM hypothesis generation, VLM grounded confidence in Eq. (1), and slope trend verification — is not defined in terms of the FeSD ground-truth slices; it is a generate-and-verify procedure that can be applied to externally computed detection/segmentation errors. The FeSD benchmark itself is constructed from model errors and human annotation rather than from SliceLens's output, so evaluating on it is not circular by itself. The main concern is the protocol of Table 7: the reported 0.73 is an average over per-GT 'Best Predicted Slice' values with no stated selection mechanism, so the central SOTA claim may be a fitted/oracle selection rather than the precision of a single discovered slice; this is a partial construction of the headline prediction. A second self-reference appears in the Section 5.3.1 ablation, where Gemini-2.5-Pro both generates hypotheses and judges their semantic match to GT. These issues are load-bearing for the headline and the ablation respectively, but the underlying method has independent content (external classification benchmarks, model-repair experiments, GT-based error regions), so this is not a fully circular derivation. The abstract's inconsistent baseline (0.63 vs. 0.31/0.28) is a consistency/correctness problem, not a circularity problem, and is noted separately.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on the reliability of VLM grounding, the validity of the expert-annotated FeSD benchmark, and the statistical trend criterion, none of which are independently verified or accompanied by released artifacts. The free parameters are mostly thresholds and window sizes that are not fully specified.

free parameters (4)
  • slope_threshold
    Used in the automated error-slice identification (Section 3.2.2, Figure 7) to decide whether a positive confidence-error slope indicates a systematic slice; no concrete value is given.
  • sliding-window size/step
    The dynamic slope trend analysis in Section 3.2.2 depends on unspecified sliding-window parameters.
  • GroupDRO confidence threshold = 0.9
    In the detection repair experiment (Section 5.4.2), an image-level binary attribute is defined by thresholding the maximum instance-level slice confidence at 0.9; this is a hand-chosen threshold.
  • sampled-subset size for caption extraction
    The data-driven hypothesis generation (Section 3.1) samples a subset of images for VLM captioning, but the subset size is not specified.
assumptions (5)
  • domain assumption VLM yes/no logits with box/point grounding provide reliable slice-confidence estimates for complex relational queries.
    Invoked in Section 3.2.1, Eq. (1); if the VLM cannot judge relational hypotheses accurately, the entire retrieval pipeline fails.
  • domain assumption Monotonic increase of error rate with slice confidence identifies systematic error slices.
    Section 3.2.2 asserts that a positive slope in confidence-error trend indicates a systematic failure; no formal justification is given.
  • domain assumption FeSD expert annotations are valid ground truth, and overwriting non-slice predictions with ground truth removes noise without biasing the benchmark.
    Section 4.1, step 4 makes the dataset artificially clean; this could make evaluation easier and less representative of real deployment.
  • domain assumption LLM world knowledge is sufficient to propose relevant failure hypotheses for a given task.
    Section 3.1 relies on LLM priors to generate knowledge-driven hypotheses; completeness is acknowledged as challenging in Section 6.
  • ad hoc to paper Semantic relevance judged by the same LLM family (Gemini-2.5-Pro) used for hypothesis generation is a valid evaluation of slice matching.
    Section 5.3.1 and Appendix A.3 use Gemini-2.5-Pro as evaluator; using the same model family for generation and evaluation introduces circularity in the reported recall/precision.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks." pith.science (2026). https://pith.science/paper/26YO3WNT

@misc{pith2026251224592,
  author       = {Pith},
  title        = {Pith review of: GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/26YO3WNT}},
  note         = {Machine review of arXiv:2512.24592}
}
read the original abstract

Systematic failures of vision models on semantically coherent subsets, known as error slices, reveal limitations in robustness and evaluation. Existing slice discovery approaches largely model slices as clusters in representation space or combinations of predefined attributes. While effective for image-level classification, such formulations are insufficient for instance-level tasks such as object detection and segmentation, where failures often arise from contextual relational and spatially grounded visual patterns. We propose GH-ESD (Grounded Hypothesis-Driven Error Slice Discovery), a generate and verify framework that reformulates slice discovery as grounded hypothesis generation and statistical verification. GH-ESD constructs relational failure hypotheses using LLM priors and grounded visual evidence, discovers hypothesis slices at the instance level via Vision Language Models, and verifies them through statistical trend analysis over instance-level errors. We also introduce GESD (Grounded Error Slice Dataset), a new benchmark for instance-level error slice discovery, providing expert-defined and spatially grounded slices derived from detection and segmentation failures. Extensive experiments demonstrate that GH-ESD consistently outperforms baselines, improving Precision@10 by 0.10 (0.73 vs. 0.63) on the GESD benchmark for detection tasks, while also supporting segmentation scenarios. GH-ESD identifies interpretable slices that facilitate actionable model improvements.

Figures

Figures reproduced from arXiv: 2512.24592 by the authors.

Figure 1
Figure 1. Error slice discovery illustrated on multi-instance tasks [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of error slice benchmark datasets. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of SliceLens. Illustrated with the “bicycle detection” example. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Performance comparison across different error cate [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Why grounded (instance-level) retrieval reveals fail￾ure slices while image-level retrieval dilutes them. Instance￾level retrieval produces stronger alignment between VLM similar￾ity and model errors [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Comparison between tags and our hypotheses on seman [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Comparison between error rate thresholding and slope [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The detailed system prompt for generating a comprehensive set of knowledge-driven hypotheses. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: The prompt used to instruct the Vision-Language Model (VLM) for grounded attribute-aware caption extraction. The [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: The prompts used for attribute inference via text clustering. Step 1 extracts relevant values from attributes and captions, and [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: The prompt used for refining attributes and values into grounded query phrases. The [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: The specialized user prompt used for object detection hypothesis generation on the FeSD dataset. The prompt focuses on the [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: The specialized user prompt used for instance segmentation hypothesis generation on the FeSD dataset. This prompt addresses [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: SliceLens main interface with interactive chat functionality. The left panel displays the data gallery for dataset visualization, while the right panel provides a conversational interface for users to interact with the LLM for hypothesis generation and error slice dis…
Figure 15
Figure 15. Figure 15: Hypothesis selection and submission interface. Users can review automatically generated failure pattern hypotheses, assess their plausibility, and submit selected candidates for validation [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Error slice verification task management interface. The interface displays all submitted validation tasks with their current status, allowing users to track progress and access completed results including discovered slice data and trend analysis [PITH_FULL_IMAGE:figu…
Figure 17
Figure 17. Figure 17: Trend analysis results interface. Example showing the statistical validation of a hypothesis through slope trend analysis between slice confidence scores and model performance, demonstrating robust evidence for discovered error slices. Note that the Y-axis in this exa…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 4 linked inside Pith

  1. [1]

    Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings

    Mohsan Alvi, Andrew Zisserman, and Christoffer Nell ˚aker. Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings. InECCV Workshop, pages 0–0, 2018. 3

    2018

  2. [2]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Jun- yang Lin. Qwen2.5-vl technical repor...

    arXiv 2025

  3. [3]

    The fishyscapes benchmark: Measuring blind spots in semantic segmentation.IJCV, 129 (12):3119–3135, 2021

    Hermann Blum, Paul-Edouard Sarlin, Juan Nieto, Roland Siegwart, and Cesar Cadena. The fishyscapes benchmark: Measuring blind spots in semantic segmentation.IJCV, 129 (12):3119–3135, 2021. 1, 3

    2021

  4. [4]

    End- to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End- to-end object detection with transformers. InECCV, pages 213–229, 2020. 5

    2020

  5. [5]

    Hibug: On human- interpretable model debug

    Muxi Chen, Yu Li, and Qiang Xu. Hibug: On human- interpretable model debug. InNeurIPS, pages 4753–4766,

  6. [6]

    Hibug2: Effi- cient and interpretable error slice discovery for comprehen- sive model debugging

    Muxi Chen, Chenchen Zhao, and Qiang Xu. Hibug2: Effi- cient and interpretable error slice discovery for comprehen- sive model debugging. InICLR, 2025. 1, 2, 3, 4, 7

    2025

  7. [7]

    Wright, and Kevin Leyton-Brown

    Greg d’Eon, Jason d’Eon, James R. Wright, and Kevin Leyton-Brown. The spotlight: A general method for dis- covering systematic errors in deep learning models. InPro- ceedings of the 2022 ACM Conference on Fairness, Account- ability, and Transparency (FAccT), pages 1962–1981, 2022. 2

    2022

  8. [8]

    Face detection dataset

    Fares Elmenshawii. Face detection dataset. Kaggle dataset,

Show all 41 references
  1. [9]

    Domino: Discovering systematic errors with cross-modal embeddings.ICLR, 2022

    Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher R ´e. Domino: Discovering systematic errors with cross-modal embeddings.ICLR, 2022. 1, 2, 5, 6

    2022

  2. [10]

    Detecting system- atic weaknesses in vision models along predefined human- understandable dimensions.Transactions on Machine Learning Research (TMLR), 2025

    Sujan Sai Gannamaneni, Rohil Prakash Rao, Michael Mock, Maram Akila, and Stefan Wrobel. Detecting system- atic weaknesses in vision models along predefined human- understandable dimensions.Transactions on Machine Learning Research (TMLR), 2025. 1

    2025

  3. [11]

    Are we ready for autonomous driving? the KITTI vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the KITTI vision benchmark suite. InCVPR, pages 3354–3361, 2012. 5

    2012

  4. [12]

    Poynton, Shyam Visweswaran, and Kayhan Batmanghelich

    Shantanu Ghosh, Rayan Syed, Chenyu Wang, Vaib- hav Choudhary, Binxu Li, Clare B. Poynton, Shyam Visweswaran, and Kayhan Batmanghelich. LADDER: Language-driven slice discovery and error rectification in vi- sion classifiers. InFindings of the Association for Computa- tional Lin...

    2025

  5. [13]

    Classifier-to-bias: Toward unsupervised au- tomatic bias detection for visual classifiers

    Quentin Guimard, Moreno D’Inc `a, Massimiliano Mancini, and Elisa Ricci. Classifier-to-bias: Toward unsupervised au- tomatic bias detection for visual classifiers. InCVPR, pages 15151–15161, 2025. 1, 2

    2025

  6. [14]

    Mask R-CNN

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask R-CNN. InICCV, pages 2961–2969, 2017. arXiv:1703.06870. 5

    2017 arXiv

  7. [15]

    Towards non-iid image classification: A dataset and baselines

    Yiyou He, Yaojie Shen, Hongxin Zhang, Pingyu Chen, Lei Ren, and Pin-Yu Liu. Towards non-iid image classification: A dataset and baselines. InNeurIPS Datasets and Bench- marks Track, 2021. 3

    2021

  8. [16]

    Distilling model failures as directions in latent space

    Saachi Jain, Hannah Lawrence, Ankur Moitra, and Alek- sander Madry. Distilling model failures as directions in latent space. InArXiv preprint arXiv:2206.14754, 2022. 2, 5

    2022 arXiv

  9. [17]

    Discovering and mitigat- ing visual biases through keyword explanation (bias-to-text)

    Younghyun Kim, Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jaeho Lee, and Jinwoo Shin. Discovering and mitigat- ing visual biases through keyword explanation (bias-to-text). InCVPR, 2024. 2, 5

    2024

  10. [18]

    Last layer re-training is sufficient for robustness to spu- rious correlations

    Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wil- son. Last layer re-training is sufficient for robustness to spu- rious correlations. InICLR, 2023. 8

    2023

  11. [19]

    Medsegbench: A compre- hensive benchmark for medical image segmentation in di- verse data modalities.Scientific Data, 11(1):1283, 2024

    Zeynep Kus ¸ and Mustafa Aydin. Medsegbench: A compre- hensive benchmark for medical image segmentation in di- verse data modalities.Scientific Data, 11(1):1283, 2024. 3

    2024

  12. [20]

    Ryu, and Kangwook Lee

    Sehyun Kwon, Jaeseung Park, Minkyu Kim, Jaewoong Cho, Ernest K. Ryu, and Kangwook Lee. Image clustering condi- tioned on text criteria. InICLR, 2024. 4

    2024

  13. [21]

    Lawrence Zitnick, and Piotr Doll ´ar

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Doll ´ar. Microsoft coco: Common objects in context. InECCV, pages 740–755,

  14. [22]

    Focal loss for dense object detection.IEEE TPAMI, 42(2):318–327, 2020

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll ´ar. Focal loss for dense object detection.IEEE TPAMI, 42(2):318–327, 2020. Originally in ICCV 2017 ver- sion. 5

    2020

  15. [23]

    Just train twice: Improving group robustness without training group information

    Evan Zheran Liu, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. Just train twice: Improving group robustness without training group information. InProceedings of the In- ternational Conference on Machine Learning (ICML), pages 6781–6792, 2021. 3

    2021

  16. [24]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. InICCV, 2015. 3, 5

    2015

  17. [25]

    Llm as dataset ana- lyst: Subpopulation structure discovery with large language model

    Yulin Luo, Ruichuan An, Bocheng Zou, Yiming Tang, Ji- aming Liu, and Shanghang Zhang. Llm as dataset ana- lyst: Subpopulation structure discovery with large language model. InECCV, pages 235–252. Springer, 2024. 2

    2024

  18. [26]

    Coco-o: A benchmark for object detectors under natural dis- tribution shifts

    Xiaofeng Mao, Yuefeng Chen, Yao Zhu, Da Chen, et al. Coco-o: A benchmark for object detectors under natural dis- tribution shifts. InICCV, 2023. 3

    2023

  19. [27]

    Vig-bias: Visually grounded bias discovery and mitigation

    Badr-Eddine Marani, Mohamed Hanini, Nihitha Mala- yarukil, Stergios Christodoulidis, Maria Vakalopoulou, and Enzo Ferrante. Vig-bias: Visually grounded bias discovery and mitigation. InECCV, pages 414–429. Springer, 2024. 3, 5

    2024

  20. [28]

    Hidden stratification causes clinically meaningful failures in machine learning for medical imag- ing

    Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher R ´e. Hidden stratification causes clinically meaningful failures in machine learning for medical imag- ing. InProceedings of the ACM Conference on Health, In- ference, and Learning (CHIL), pages 151–159, 2020. 1

    2020

  21. [29]

    Faster R-CNN: Towards real-time object detection with re- gion proposal networks.IEEE TPAMI, 39(6):1137–1149,

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with re- gion proposal networks.IEEE TPAMI, 39(6):1137–1149,

  22. [30]

    Hashimoto, and Percy Liang

    Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In ICLR, 2020. 2, 3, 5, 8

    2020

  23. [31]

    Sensitiveloss: Improving accuracy and fair- ness of face representations with discrimination-aware deep learning

    Ignacio Serna, Aythami Morales, Julian Fierrez, and Javier Ortega-Garcia. Sensitiveloss: Improving accuracy and fair- ness of face representations with discrimination-aware deep learning. InCVPRW, pages 0–0, 2020. 3

    2020

  24. [32]

    Laleh Seyyed-Kalantari, Haoran Zhang, Matthew B. A. Mc- Dermott, Irene Y . Chen, and Marzyeh Ghassemi. Under- diagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations.Na- ture Medicine, 27(12):2176–2182, 2021. 1

    2021

  25. [33]

    YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

    Chien-Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. InCVPR,

  26. [34]

    Error discovery by cluster- ing influence embeddings

    Fulton Wang, Julius Adebayo, Sarah Tan, Diego Garcia- Olano, and Narine Kokhlikyan. Error discovery by cluster- ing influence embeddings. InNeurIPS, pages 41765–41777,

  27. [35]

    Towards fairness in visual recognition: Effective strategies for bias mitigation

    Michelle Wang and Jia Deng. Towards fairness in visual recognition: Effective strategies for bias mitigation. In CVPR, pages 8919–8928, 2020. 3

    2020

  28. [36]

    Vislix: An XAI framework for val- idating vision models with slice discovery and analysis

    Xinyuan Yan, Xiwei Xuan, Jorge Piazentin Ono, Jiajing Guo, Vikram Mohanty, Shekar Arvind Kumar, Liang Gou, Bei Wang, and Liu Ren. Vislix: An XAI framework for val- idating vision models with slice discovery and analysis. In Computer Graphics Forum (CGF), page e70125, 2025. 1, 3

    2025

  29. [37]

    Yenamandra et al

    S. Yenamandra et al. First amplify correlations and then slice to discover bias (facts). InICCV, 2023. 1, 2, 5, 6

    2023

  30. [38]

    Nico++: Towards better benchmarking for domain generalization

    Xingxuan Zhang, Yue He, Renzhe Xu, Han Yu, Zheyan Shen, and Peng Cui. Nico++: Towards better benchmarking for domain generalization. InCVPR, pages 16036–16047,

  31. [39]

    cluster” or “search

    Enshen Zhou, Qi Su, Cheng Chi, Zhizheng Zhang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, and He Wang. Code-as-monitor: Constraint-aware visual programming for reactive and proactive robotic failure detection. InCVPR, pages 6919–6929, 2025. 1 SliceLens: Fine-Grained and Grounded ...

    2025

  32. [2017]

    Originally in arXiv:1506.01497 / NIPS 2015 version. 5

    2015 arXiv

  33. [2025]

    22, 2025

    Accessed: Aug. 22, 2025. 5

    2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.