Pith. sign in

REVIEW 3 major objections 5 minor 68 references

VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VISTA claims that combining automatic multi-perspective metrics with a human-in-the-loop visual interface improves foundation-model-generated segment-label data, and that the improved data raises retrieval-based open-vocabulary image…

desk verdict A genuine visual analytics system for a real data-quality problem, but Table I's quantitative claim is undercut by the circularity between VISTA's own detection metric and the retrieval evaluator; needs a threshold-filter baseline and an independent quality metric before the performance gain is credible. read the letter →

arxiv 2507.09008 v1 pith:2DDA6AIK submitted 2025-07-11 cs.CV

Xiwei Xuan , Xiaoqi Wang , Wenbin He , Jorge Piazentin Ono , Liang Gou , Kwan-Liu Ma , Liu Ren This is my paper
classification cs.CV
keywords visualanalyticsdataqualityfoundationmodelsopen-vocabularyimagesegmentationhuman-in-the-loopauto-labelingmulti-modallabelvalidation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the quality of foundation-model-generated image labels can be improved at scale by combining automatic multi-perspective metrics with a human-in-the-loop visual interface, and that the improved data materially helps a downstream open-vocabulary image segmentation task. The authors build VISTA, a visual analytics framework that flags suspect segment-label pairs from several angles, groups the suspects into coherent patterns, and lets validators inspect and fix the problems. On the two benchmarks they test, the VISTA-processed data raises retrieval-based segmentation mIoU from 32.39 to 41.18 and from 35.55 to 49.90, with pixel accuracy improving by similar margins. Ten external machine-learning experts rated the interface's usability and usefulness positively. If this holds, it suggests that careful human review of auto-labeled data, focused by algorithmic summarization, can be a practical alternative to simply generating ever more data.

What carries the argument

The load-bearing mechanism is a set of three quality metrics computed from cosine similarities in a shared vision-language embedding space. M1 measures segment-label similarity; M2 counts how many distinct segment groups share the same label root; M3 measures multi-granularity alignment through bounding-box-label similarity, image-label similarity, and segment size. These metrics feed two summarization routes: label groups are organized by root noun and segments by a two-step 2D projection plus density clustering, with semantics discovered from dense cluster cores; alignment issues are summarized by discretizing the M3 sub-metrics and mining subgroups that share consistent error patterns. The interface then lets validators brush histograms, inspect embedding and sample views, and annotate corrections, giving the human the final say.

What would settle it

Take VISTA's original FM-generated data, apply a simple automatic rule (drop segment-label pairs whose M1 similarity is below the global median and whose label frequency is high) with no human review, and run the same retrieval-based OVIS evaluation. If this rule matches VISTA's mIoU gains on the two benchmarks, then the human-in-the-loop layer is not what drives the reported improvement. Alternatively, have human annotators judge label correctness on random samples of the original and VISTA-enhanced sets; if label accuracy per validated sample is unchanged, the gains are an artifact of the retrieval scoring.

Watch

Extended reading notes

Core claim

VISTA improves FM-generated open-vocabulary segmentation data by structuring labels and segments, quantifying quality along three families of metrics, summarizing issues into interpretable groups, and guiding human validators through explore-verify-annotate stages. Its central empirical claim is that data corrected through this loop substantially outperforms the original auto-labeled data when used as a reference set in a training-free retrieval-based OVIS method. In the first use case, annotating 27 labels and 20 alignment groups fixed around 1.1M segment-label pairs and raised mIoU from 32.39 to 41.18; in the second, fixing 15k samples raised mIoU from 35.55 to 49.90. The paper also claims that all ten external ML experts found the system usable and useful, and that experts could identify hidden errors such as non-noun labels assigned to many segments, labels like "looking at the camera" applied to segments without cameras, and road-label segments that capture only road edges.

Load-bearing premise

The measured improvement comes from a retrieval-based evaluator that scores segment-label pairs with the same type of embedding similarity that VISTA uses to flag low-quality pairs, so removing low-similarity pairs may help that evaluator even if the labels are not actually more correct for other models or for the real world.

Editorial extensions

If this is right

  • If VISTA's gains replicate, data-quality curation can be invoked as a first-line fix for FM-generated training sets before retraining or expanding data volume.
  • The same pipeline could be applied to other segment-label auto-labeling chains, since the three metric families do not depend on a specific grounding or segmentation model.
  • Human review effort becomes scalable because the system bundles millions of segment-label pairs into a small number of label groups and alignment patterns; fixing 47 issues addressed 1.1M pairs in the first use case.
  • Because the correction options are model-free (remove label, remove segment, re-assign matching), the enhanced data remains usable by any downstream OVIS or detection method.
  • For practitioners, VISTA offers a template for closed-loop data validation: automatic flagging, pattern summarization, human verification, and measured downstream improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported evaluation measures a retrieval method whose scoring uses the same embeddings and cosine similarity as VISTA's own flagging metric, so some of the gain may be mechanical pruning of low-similarity pairs; a separate label-correctness audit would test whether the curated data is genuinely better for unrelated models.
  • A natural stress test is to apply a simple automatic threshold filter that drops low M1 or high M2 pairs without human review; if it matches VISTA's gains, the value of the visualization layer lies in transparency and verification rather than in the selection itself.
  • The same summarization logic could be extended to caption-quality validation or to other modality pairs, since label-root grouping and pattern mining translate to any setting where one side of a pair is textual.
  • Running a training-based open-vocabulary segmentation model on the curated reference set would show whether the gains persist beyond the training-free retrieval setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces VISTA, a visual analytics framework for validating and correcting segment-label pairs generated by foundation models for open-vocabulary image segmentation. The method defines three quality metrics—M1 (segment–label cosine similarity, Eq. 1), M2 (label frequency, Eq. 2), and M3 (multi-granularity alignment, Eq. 3)—and uses clustering, semantic discovery, and DivExplorer-based pattern summarization to organize and explain data issues. A coordinated multi-view interface lets human validators explore, verify, and annotate issues, with remediation options such as removing labels, removing segments, or rematching labels. The paper reports two use cases on COCO Stuff and Cityscapes in which a training-free retrieval-based OVIS evaluator (Eqs. 4–6) improves mIoU from 32.39 to 41.18 and from 35.55 to 49.90 when using the VISTA-enhanced reference data, plus a qualitative expert study with ten ML experts.

Significance. If the reported gains are real, VISTA addresses a genuine gap: scalable, multi-faceted validation of auto-labeled multi-modal data with human expertise. The framework is thoughtfully designed and the algorithmic pipeline is described in sufficient detail to be implemented. Strengths include the use of multiple complementary quality metrics, the application of DivExplorer to summarize alignment error patterns, and the expert study confirming usability. However, the central quantitative claim is not yet backed by an independent evaluation: the OVIS evaluator and the primary issue-detection metric share the same CLIP cosine similarity, so the gains in Table I may partly reflect mechanical pruning of low-similarity pairs. The lack of controls, variance estimates, and an independent label-correctness measure means the paper currently demonstrates feasibility rather than a robust, generalizable performance gain.

major comments (3)
  1. [Section IV-D, Eqs. (4)-(6)] The online OVIS evaluator retrieves labels via cosine similarity in the same CLIP embedding space used by VISTA's primary issue metric M1 (Eq. 1). Since the workflow in Secs. IV-C3 and V-C flags and removes/corrects exactly the low-M1 pairs, the mIoU gains in Table I are expected for this evaluator even if the human corrections do not improve semantic correctness. Add control baselines—e.g., automatically pruning the same number of lowest-M1 pairs, and random removal matched for size—and report the resulting mIoU; without these, the improvements of 8.79 and 14.35 mIoU points do not isolate the contribution of human-in-the-loop validation.
  2. [Table I and Secs. VI-A/VI-B] The quantitative results come from a single user session per dataset (Alice, Kevin), with no multiple annotators, no repeated runs, and no variance or significance reporting. Because the entire quantitative effectiveness claim rests on these two numbers, the paper should either add a small multi-user/repeated-run study or explicitly state and justify the single-user design as a limitation. This matters because different validators may annotate different subsets and produce different gains.
  3. [Secs. IV-C3 and V-C] The paper does not compare VISTA to a simple automatic baseline such as threshold-based filtering on M1 alone, despite the fact that DivExplorer already uses the global median of M1 to flag misalignment and the remediation options include removing low-quality labels/segments. Without such a comparison, it is unclear whether the human-in-the-loop workflow adds value beyond automatic pruning; this is load-bearing for the paper's central claim. In addition, no independent label-correctness metric (e.g., agreement with ground-truth labels or human judgment on a held-out sample) is reported, so there is no direct evidence that the VISTA-corrected labels are semantically more correct.
minor comments (5)
  1. [Section IV, first paragraph] The text contains a typo: 'VISTA;s' should be 'VISTA's'.
  2. [Equation (3)] The sentence says 'w∗ and f∗ represent the width and height', but the symbol for height should be 'h∗' to match the preceding formula; please fix this and define all symbols explicitly.
  3. [Section VI-C] The expert list is defined as E0–E9, but the text later refers to 'E10' in 'although both E5 and E10 haven't used any V A system'; this should be corrected to an existing expert ID or the demographics should be updated.
  4. [Section V-B] The phrase 'Interpret & Verifystage' is missing a space; it should read 'Interpret & Verify stage'.
  5. [Algorithm 1 and Section IV-D] The KDE bandwidth h, the top-p% value, and the softmax temperatures β1 and β2 are not specified in the main text, and the ablation study for β1/β2 is only mentioned as 'in the supplementary'; please include these details or a pointer to the actual supplementary material.

Circularity Check

1 steps flagged · score 5.0 of 10

The Table I OVIS gains are partly circular: VISTA flags pairs by CLIP segment-label cosine similarity (Eq. 1) and quantifies gains with a retrieval evaluator over the same CLIP embeddings and cosine similarities (Eqs. 4-6), with no threshold-only or random-removal control.

  1. fitted input called prediction [Sec. IV-B (Eq. 1), Sec. IV-C3, Sec. II-C, Sec. IV-D (Eqs. 4-6), Sec. VI-A (Table I)]
    "we leverage segment-label similarity (M1) as the foundation to identify problematic data... we set its global median as the default value of the threshold—If the value of M1 falls below it, the pair is flagged for misalignment... we adopt a training-free OVIS method [65] that performs semantic segmentation by retrieving a reference set with segment-label pairs... retrieval-based methods provide a valuable evaluation for our framework, whose performance is mainly determined by the quality of reference sets."

    The flagging input and the evaluated output are the same construct. Eq. 1 defines pair quality as CLIP cosine similarity of segment and label embeddings; the data-derived global-median threshold then removes or re-matches low-M1 pairs. Eqs. 4-6 measure OVIS quality by cosine retrieval over the same CLIP segment and label embeddings, and the paper states retrieval performance 'is mainly determined by the quality of reference sets.' A low-M1 pair is by definition one whose label mismatches its visual content, so it contributes noise to cosine retrieval for visually similar test segments; pruning such pairs raises mIoU mechanically, regardless of whether the human edits were semantically correct.

full rationale

The quantitative effectiveness claim rests on Table I, where mIoU and pixel accuracy of a training-free retrieval OVIS method (Eqs. 4-6) improve after VISTA enhancement. The partial circularity is that M1 (Eq. 1), the paper's 'primary data issue indicator,' is the cosine similarity between segment and label embeddings in CLIP space, and DivExplorer's flagging uses M1's global median as the threshold to select pairs for removal or relabeling; the evaluator then retrieves labels via cosine similarity over the very same CLIP embeddings, with the paper itself stating that retrieval performance is mainly determined by the reference set quality that M1 directly measures. Removing low-M1 pairs therefore helps this evaluator substantially by construction. Independent content keeps the score moderate: the test labels are external ground truth (COCO Stuff, Cityscapes); the concrete human corrections (e.g., 'camera' describing context rather than content, 'front' being a non-noun) are semantically meaningful and not derived from any embedding metric; the paper explicitly says it 'consciously refrain[s] from utilizing these models at the error-addressing stage' (Sec. V-C); and the expert study with ten external ML experts is independent qualitative evidence. The self-citation to USE ([3]) is contextual and not load-bearing. However, the absence of a threshold-only baseline, a matched random-removal control, and an independent label-correctness metric means the central quantitative claim does not yet separate human-in-the-loop semantic correction from mechanical M1-based pruning, which is a genuine partial circularity in the demonstration rather than in the framework's construction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on unvalidated domain assumptions: that CLIP cosine similarity reflects true label-segment correctness, that the same embedding-based retrieval is an independent quality benchmark, that UMAP/HDBSCAN clusters are semantically meaningful, and that single human validators are reliable. Several thresholds and hyperparameters are fitted to the data. No formal verification or released code is provided.

free parameters (5)
  • M1 misalignment threshold = Global median of M1 in the dataset
    In Sec. IV-C3, pairs with M1 below the dataset's global median are flagged as misaligned; this is a data-dependent threshold.
  • M3 discretization thresholds = 33rd and 66th percentiles
    Sec. IV-C3 converts each M3 sub-metric into three categories at the 33% and 66% percentiles of the data.
  • Top-p% for semantics discovery = Not specified in the paper
    Algorithm 1 uses top p% of ranked segments to derive cluster semantics; p is not given in the main text.
  • Softmax temperatures β1, β2 = Not specified in main text; ablation deferred to supplementary
    Eqs. 4-5 in the online evaluation use temperatures β1 and β2 to control softmax sharpness; values are not reported.
  • KDE bandwidth h = Not specified
    Algorithm 1 line 3 uses KDE with bandwidth h for density estimation; no value or selection procedure is given.
assumptions (6)
  • domain assumption CLIP image and text embedding cosine similarity reflects semantic correctness of segment-label pairs.
    Both the quality metrics (Eq. 1-3) and the OVIS retrieval evaluation (Eq. 4-6) assume this alignment without ground-truth validation.
  • domain assumption Removing pairs flagged by the metrics yields genuinely higher-quality data that benefits downstream models.
    No independent label-quality benchmark or downstream training evaluation is provided; only a retrieval method sharing the same embeddings.
  • domain assumption UMAP followed by HDBSCAN produces semantically meaningful segment clusters.
    Sec. IV-A states silhouette scores led to this choice but provides no quantitative comparison in the main text.
  • domain assumption The two use-case human validators are reliable and their corrections are correct.
    The personas are likely the authors or close collaborators; there is no inter-annotator agreement or verification.
  • domain assumption The root noun of a label is the invariant that organizes label semantics.
    Sec. IV-A claims the invariant is the noun, which may fail for multi-word or compositional phrases.
  • domain assumption DivExplorer's pattern divergence identifies coherent and practically useful misalignment patterns.
    Sec. IV-C3 applies DivExplorer to find subgroups with consistent M3 attributes but does not validate the patterns against ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels." pith.science (2026). https://pith.science/paper/2DDA6AIK

@misc{pith2026250709008,
  author       = {Pith},
  title        = {Pith review of: VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DDA6AIK}},
  note         = {Machine review of arXiv:2507.09008}
}
read the original abstract

The advances in multi-modal foundation models (FMs) (e.g., CLIP and LLaVA) have facilitated the auto-labeling of large-scale datasets, enhancing model performance in challenging downstream tasks such as open-vocabulary object detection and segmentation. However, the quality of FM-generated labels is less studied as existing approaches focus more on data quantity over quality. This is because validating large volumes of data without ground truth presents a considerable challenge in practice. Existing methods typically rely on limited metrics to identify problematic data, lacking a comprehensive perspective, or apply human validation to only a small data fraction, failing to address the full spectrum of potential issues. To overcome these challenges, we introduce VISTA, a visual analytics framework that improves data quality to enhance the performance of multi-modal models. Targeting the complex and demanding domain of open-vocabulary image segmentation, VISTA integrates multi-phased data validation strategies with human expertise, enabling humans to identify, understand, and correct hidden issues within FM-generated labels. Through detailed use cases on two benchmark datasets and expert reviews, we demonstrate VISTA's effectiveness from both quantitative and qualitative perspectives.

Figures

Figures reproduced from arXiv: 2507.09008 by the authors.

Figure 1
Figure 1. OVIS data generation pipeline, involving FMs to construct segment [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. VISTA data quality enhancement framework, comprising the method phase for issue detection and summarization and the visual analytic interface for issue validation and mitigation. With VISTA, we enhance foundation model-generated data quality with human assistance. B. Challenges for Quality Enhancement Building upon the identified data quality considerations, we delineate the core challenges that must be addressed. C… view at source ↗
Figure 3
Figure 3. Method in VISTA for data issue detection and summarization, with three interlocked phases: label and segment data structuring, quality quantification for different issues, and data issue summarization for forming subgroups with coherent patterns or semantics. dog”. Regardless of various descriptions, the root “dog” is unchanged. Therefore, by extracting the root from noun phrases, we categorize various labels accord… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The system interface of VISTA supports the quality enhancement of foundation-model generated labels, integrating: ●A Label View presents groups of labels in a grid-based layout, allowing for label issue validation; ●B Issue Pattern View organizes segment-label pairs in…
Figure 5
Figure 5. Figure 5: Analytical workflow for human validators to conduct issue validation and mitigation with [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Data pipeline for getting segment-label pairs in our use cases. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The use case of traffic scene understanding, where the user identified: (1) Labels “people”, “pedestrian” mistakenly assigned to vague image segments [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Expert perception of the system according to Likert-type questions. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 52 canonical work pages

  1. [1]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2023

  2. [2]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  3. [3]

    USE: Universal segment embeddings for open-vocabulary image segmentation,

    X. Wang, W. He, X. Xuan, C. Sebastian, J. P. Ono, X. Li, S. Behpour, T. Doan, L. Gou, H. W. Shen, and L. Ren, “USE: Universal segment embeddings for open-vocabulary image segmentation,” in Conference on Computer Vision and Pattern Recognition 2024 , 2024

    work page 2024

  4. [4]

    Kosmos-2: Grounding multimodal large language models to the world,

    Z. Peng, W. Wang, L. Dong, Y . Hao, S. Huang, S. Ma, and F. Wei, “Kosmos-2: Grounding multimodal large language models to the world,” arXiv preprint arXiv:2306.14824 , 2023

    arXiv 2023

  5. [5]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems , vol. 36, 2024

    2024

  6. [6]

    Emu: Enhancing image generation models using photogenic needles in a haystack,

    X. Dai, J. Hou, C.-Y . Ma, S. Tsai, J. Wang, R. Wang, P. Zhang, S. Vandenhende, X. Wang, A. Dubey et al. , “Emu: Enhancing image generation models using photogenic needles in a haystack,” arXiv preprint arXiv:2309.15807, 2023

    arXiv 2023

  7. [7]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    arXiv 2023

  8. [8]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,

    C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki, “Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,”arXiv preprint arXiv:2111.02114, 2021

    arXiv 2021

Show all 68 references
  1. [9]

    Demystifying clip data,

    H. Xu, S. Xie, X. E. Tan, P.-Y . Huang, R. Howes, V . Sharma, S.-W. Li, G. Ghosh, L. Zettlemoyer, and C. Feichtenhofer, “Demystifying clip data,” arXiv preprint arXiv:2309.16671 , 2023

    2023 arXiv

  2. [10]

    Foundation models meet visualizations: Challenges and opportunities,

    W. Yang, M. Liu, Z. Wang, and S. Liu, “Foundation models meet visualizations: Challenges and opportunities,” Computational Visual Media, pp. 1–26, 2024

    2024

  3. [11]

    Comparing visual-interactive labeling with active learning: An ex- perimental study,

    J. Bernard, M. Hutter, M. Zeppelzauer, D. Fellner, and M. Sedlmair, “Comparing visual-interactive labeling with active learning: An ex- perimental study,” IEEE transactions on visualization and computer graphics, vol. 24, no. 1, pp. 298–308, 2017. JOURNAL OF LATEX CLASS FILES,...

    2017

  4. [12]

    Interactive labelling of a multivariate dataset for supervised machine learning using linked visualisations, clustering, and active learning,

    M. Chegini, J. Bernard, P. Berger, A. Sourin, K. Andrews, and T. Schreck, “Interactive labelling of a multivariate dataset for supervised machine learning using linked visualisations, clustering, and active learning,” Visual Informatics, vol. 3, no. 1, pp. 9–17, 2019

    2019

  5. [13]

    Vassl: A visual analytics toolkit for social spambot labeling,

    M. Khayat, M. Karimzadeh, J. Zhao, and D. S. Ebert, “Vassl: A visual analytics toolkit for social spambot labeling,” IEEE transactions on visualization and computer graphics , vol. 26, no. 1, pp. 874–883, 2019

    2019

  6. [14]

    V AC-CNN: A visual analytics system for comparative studies of deep convolutional neural networks,

    X. Xuan, X. Zhang, O.-H. Kwon, and K.-L. Ma, “V AC-CNN: A visual analytics system for comparative studies of deep convolutional neural networks,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 6, pp. 2326–2337, 2022

    2022

  7. [15]

    On the cost of interactions in interactive visual machine learning,

    Y . Zhang, B. Coecke, and M. Chen, “On the cost of interactions in interactive visual machine learning,” in 2019 IEEE workshop on EValuation of Interactive VisuAl Machine Learning systems (Eviva-ML). IEEE, 2019, pp. 1–5

    2019

  8. [16]

    A simulation-based approach for quantifying the impact of interactive label correction for machine learning,

    Y . Wang, J. Zhao, J. Hong, R. G. Askin, and R. Maciejewski, “A simulation-based approach for quantifying the impact of interactive label correction for machine learning,” IEEE Transactions on Visualization and Computer Graphics , pp. 1–17, 2024

    2024

  9. [17]

    Aila: Attentive interactive labeling assistant for document classification through attention-based deep neural networks,

    M. Choi, C. Park, S. Yang, Y . Kim, J. Choo, and S. R. Hong, “Aila: Attentive interactive labeling assistant for document classification through attention-based deep neural networks,” in Proceedings of the 2019 CHI conference on human factors in computing systems , 2019, pp. 1–12

    2019

  10. [18]

    Onela- beler: A flexible system for building data labeling tools,

    Y . Zhang, Y . Wang, H. Zhang, B. Zhu, S. Chen, and D. Zhang, “Onela- beler: A flexible system for building data labeling tools,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems , 2022, pp. 1–22

    2022

  11. [19]

    A unified process for visual-interactive labeling,

    J. Bernard, M. Zeppelzauer, M. Sedlmair, and W. Aigner, “A unified process for visual-interactive labeling,” in Proceedings of the EuroVis Workshop on Visual Analytics , 2017, pp. 73–77

    2017

  12. [20]

    Improving the usability of hierarchical representations for interactively labeling large image data sets,

    J. Moehrmann, S. Bernstein, T. Schlegel, G. Werner, and G. Heidemann, “Improving the usability of hierarchical representations for interactively labeling large image data sets,” in Human-Computer Interaction. Design and Development Approaches: 14th International Conference, HC...

    2011

  13. [21]

    Increasing the speed and accuracy of data labeling through an ai assisted interface,

    M. Desmond, M. Muller, Z. Ashktorab, C. Dugan, E. Duesterwald, K. Brimijoin, C. Finegan-Dollak, M. Brachman, A. Sharma, N. N. Joshi et al., “Increasing the speed and accuracy of data labeling through an ai assisted interface,” in 26th International Conference on Intelligent Us...

    2021

  14. [22]

    Interactive visual labelling versus active learning: an experimental comparison,

    M. Chegini, J. Bernard, J. Cui, F. Chegini, A. Sourin, K. Andrews, and T. Schreck, “Interactive visual labelling versus active learning: an experimental comparison,” Frontiers of Information Technology & Electronic Engineering, vol. 21, pp. 524–535, 2020

    2020

  15. [23]

    Inter-active learning of ad-hoc classifiers for video visual analytics,

    B. H ¨oferlin, R. Netzel, M. H ¨oferlin, D. Weiskopf, and G. Heidemann, “Inter-active learning of ad-hoc classifiers for video visual analytics,” in 2012 IEEE conference on visual analytics science and technology (VAST). IEEE, 2012, pp. 23–32

    2012

  16. [24]

    Oodanalyzer: Interactive analysis of out-of-distribution samples,

    C. Chen, J. Yuan, Y . Lu, Y . Liu, H. Su, S. Yuan, and S. Liu, “Oodanalyzer: Interactive analysis of out-of-distribution samples,” IEEE transactions on visualization and computer graphics , vol. 27, no. 7, pp. 3335–3349, 2020

    2020

  17. [25]

    Towards better caption supervision for object detection,

    C. Chen, J. Wu, X. Wang, S. Xiang, S.-H. Zhang, Q. Tang, and S. Liu, “Towards better caption supervision for object detection,” IEEE Transactions on Visualization and Computer Graphics , vol. 28, no. 4, pp. 1941–1954, 2021

    1941

  18. [26]

    Diagnosing ensemble few-shot classifiers,

    W. Yang, X. Ye, X. Zhang, L. Xiao, J. Xia, Z. Wang, J. Zhu, H. Pfister, and S. Liu, “Diagnosing ensemble few-shot classifiers,” IEEE Trans- actions on Visualization and Computer Graphics , vol. 28, no. 9, pp. 3292–3306, 2022

    2022

  19. [27]

    Trafficvis: visualizing organized activity and spatio-temporal patterns for detecting and labeling human trafficking,

    C. Vajiac, D. H. Chau, A. Olligschlaeger, R. Mackenzie, P. Nair, M.- C. Lee, Y . Li, N. Park, R. Rabbany, and C. Faloutsos, “Trafficvis: visualizing organized activity and spatio-temporal patterns for detecting and labeling human trafficking,” IEEE Transactions on Visualizatio...

    2022

  20. [28]

    In- teractive graph construction for graph-based semi-supervised learning,

    C. Chen, Z. Wang, J. Wu, X. Wang, L.-Z. Guo, Y .-F. Li, and S. Liu, “In- teractive graph construction for graph-based semi-supervised learning,” IEEE Transactions on Visualization and Computer Graphics , vol. 27, no. 9, pp. 3701–3716, 2021

    2021

  21. [29]

    Interactive reweighting for mitigating label quality issues,

    W. Yang, Y . Guo, J. Wu, Z. Wang, L.-Z. Guo, Y .-F. Li, and S. Liu, “Interactive reweighting for mitigating label quality issues,”IEEE Trans- actions on Visualization and Computer Graphics , 2023

    2023

  22. [30]

    Enhancing single-frame supervision for better temporal action localization,

    C. Chen, J. Chen, W. Yang, H. Wang, J. Knittel, X. Zhao, S. Koch, T. Ertl, and S. Liu, “Enhancing single-frame supervision for better temporal action localization,” IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  23. [31]

    Perspective: leveraging human understanding for identifying and characterizing image atypicality,

    S. Sharifi Noorian, S. Qiu, B. Sayin, A. Balayn, U. Gadiraju, J. Yang, and A. Bozzon, “Perspective: leveraging human understanding for identifying and characterizing image atypicality,” in Proceedings of the 28th International Conference on Intelligent User Interfaces , 2023, ...

    2023

  24. [32]

    Classifier-guided visual correction of noisy labels for image classification tasks,

    A. B ¨auerle, H. Neumann, and T. Ropinski, “Classifier-guided visual correction of noisy labels for image classification tasks,” in Computer graphics forum, vol. 39, no. 3. Wiley Online Library, 2020, pp. 195– 205

    2020

  25. [33]

    AttributionScanner: A visual analytics system for model validation with metadata-free slice finding,

    X. Xuan, J. P. Ono, L. Gou, K.-L. Ma, and L. Ren, “AttributionScanner: A visual analytics system for model validation with metadata-free slice finding,” arXiv preprint arXiv:2401.06462 , 2024

    2024 arXiv

  26. [34]

    SUNY: A visual interpretation framework for convolutional neural networks from a necessary and sufficient perspective,

    X. Xuan, Z. Deng, H.-T. Lin, Z. Kong, and K.-L. Ma, “SUNY: A visual interpretation framework for convolutional neural networks from a necessary and sufficient perspective,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 8371–8376

    2024

  27. [35]

    An interactive method to improve crowdsourced annotations,

    S. Liu, C. Chen, Y . Lu, F. Ouyang, and B. Wang, “An interactive method to improve crowdsourced annotations,” IEEE transactions on visualization and computer graphics , vol. 25, no. 1, pp. 235–245, 2018

    2018

  28. [36]

    Interactive correction of mislabeled training data,

    S. Xiang, X. Ye, J. Xia, J. Wu, Y . Chen, and S. Liu, “Interactive correction of mislabeled training data,” in 2019 IEEE Conference on Visual Analytics Science and Technology (VAST) . IEEE, 2019, pp. 57–68

    2019

  29. [37]

    Labelvizier: Interactive validation and relabeling for technical text annotations,

    X. Zhang, X. Xuan, A. Dima, T. Sexton, and K.-L. Ma, “Labelvizier: Interactive validation and relabeling for technical text annotations,” in 2023 IEEE 16th Pacific Visualization Symposium (PacificVis) . IEEE, 2023, pp. 167–176

    2023

  30. [38]

    SLIM: Spuriousness mitigation with minimal human annotations,

    X. Xuan, Z. Deng, H.-T. Lin, and K.-L. Ma, “SLIM: Spuriousness mitigation with minimal human annotations,” in Proceedings of the European Conference on Computer Vision (ECCV) , September 2024

    2024

  31. [39]

    Visual concept programming: A visual analytics approach to injecting human intelligence at scale,

    M. N. Hoque, W. He, A. K. Shekar, L. Gou, and L. Ren, “Visual concept programming: A visual analytics approach to injecting human intelligence at scale,” IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 1, pp. 74–83, 2022

    2022

  32. [40]

    Open-vocabulary instance segmentation via robust cross-modal pseudo-labeling,

    D. Huynh, J. Kuen, Z. Lin, J. Gu, and E. Elhamifar, “Open-vocabulary instance segmentation via robust cross-modal pseudo-labeling,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 7020–7031

    2022

  33. [41]

    A simple baseline for open-vocabulary semantic segmentation with pre- trained vision-language model,

    M. Xu, Z. Zhang, F. Wei, Y . Lin, Y . Cao, H. Hu, and X. Bai, “A simple baseline for open-vocabulary semantic segmentation with pre- trained vision-language model,” in European Conference on Computer Vision. Springer, 2022, pp. 736–753

    2022

  34. [42]

    Groupvit: Semantic segmentation emerges from text supervision,

    J. Xu, S. De Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, and X. Wang, “Groupvit: Semantic segmentation emerges from text supervision,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 134–18 144

    2022

  35. [43]

    Open- vocabulary panoptic segmentation with text-to-image diffusion models,

    J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, and S. De Mello, “Open- vocabulary panoptic segmentation with text-to-image diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2955–2966

    2023

  36. [44]

    Side adapter network for open-vocabulary semantic segmentation,

    M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open-vocabulary semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2945–2954

    2023

  37. [45]

    Generalized decoding for pixel, image, and language,

    X. Zou, Z.-Y . Dou, J. Yang, Z. Gan, L. Li, C. Li, X. Dai, H. Behl, J. Wang, L. Yuan et al. , “Generalized decoding for pixel, image, and language,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 15 116–15 127

    2023

  38. [46]

    Extract free dense labels from clip,

    C. Zhou, C. C. Loy, and B. Dai, “Extract free dense labels from clip,” in European Conference on Computer Vision. Springer, 2022, pp. 696– 712

    2022

  39. [47]

    Proxyclip: Proxy attention improves clip for open-vocabulary segmentation,

    M. Lan, C. Chen, Y . Ke, X. Wang, L. Feng, and W. Zhang, “Proxyclip: Proxy attention improves clip for open-vocabulary segmentation,” in ECCV, 2024

    2024

  40. [48]

    Explore the potential of clip for training-free open vocabulary semantic segmentation,

    T. Shao, Z. Tian, H. Zhao, and J. Su, “Explore the potential of clip for training-free open vocabulary semantic segmentation,” arXiv preprint arXiv:2407.08268, 2024

    2024 arXiv

  41. [49]

    Reco: Retrieve and co-segment for zero-shot transfer,

    G. Shin, W. Xie, and S. Albanie, “Reco: Retrieve and co-segment for zero-shot transfer,” Advances in Neural Information Processing Systems, vol. 35, pp. 33 754–33 767, 2022

    2022

  42. [50]

    Diffusion models for open-vocabulary segmentation,

    L. Karazija, I. Laina, A. Vedaldi, and C. Rupprecht, “Diffusion models for open-vocabulary segmentation,” in European Conference on Com- puter Vision. Springer, 2025, pp. 299–317. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2025

  43. [51]

    Fossil: Free open-vocabulary semantic segmentation through synthetic references retrieval,

    L. Barsellotti, R. Amoroso, L. Baraldi, and R. Cucchiara, “Fossil: Free open-vocabulary semantic segmentation through synthetic references retrieval,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 1464–1473

    2024

  44. [52]

    kNN-CLIP: Retrieval enables training-free segmentation on continually expanding large vocabularies,

    Z. Gui, S. Sun, R. Li, J. Yuan, Z. An, K. Roth, A. Prabhu, and P. Torr, “kNN-CLIP: Retrieval enables training-free segmentation on continually expanding large vocabularies,” Transactions on Machine Learning Research, 2024

    2024

  45. [53]

    Image-to-image matching via foundation models: A new perspective for open-vocabulary semantic segmentation,

    Y . Wang, R. Sun, N. Luo, Y . Pan, and T. Zhang, “Image-to-image matching via foundation models: A new perspective for open-vocabulary semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 3952–3963

    2024

  46. [54]

    Phrasecut: Language- based image segmentation in the wild,

    C. Wu, Z. Lin, S. Cohen, T. Bui, and S. Maji, “Phrasecut: Language- based image segmentation in the wild,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 10 216–10 225

    2020

  47. [55]

    spacy: Industrial-strength natural language processing in python,

    Explosion AI, “spacy: Industrial-strength natural language processing in python,” 2020, software available from https://spacy.io/

    2020

  48. [56]

    Umap: Uniform manifold approximation and projection,

    L. McInnes, J. Healy, N. Saul, and L. Großberger, “Umap: Uniform manifold approximation and projection,” Journal of Open Source Soft- ware, vol. 3, no. 29, 2018

    2018

  49. [57]

    hdbscan: Hierarchical density based clustering

    L. McInnes, J. Healy, S. Astels et al. , “hdbscan: Hierarchical density based clustering.” J. Open Source Softw. , vol. 2, no. 11, pp. 205–206, 2017

    2017

  50. [58]

    Visualizing data using t-sne

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research , vol. 9, no. 11, 2008

    2008

  51. [59]

    A density-based algorithm for discovering clusters in large spatial databases with noise,

    M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al. , “A density-based algorithm for discovering clusters in large spatial databases with noise,” in kdd, vol. 96, 1996, pp. 226–231

    1996

  52. [60]

    Clip surgery for better ex- plainability with enhancement in open-vocabulary tasks,

    Y . Li, H. Wang, Y . Duan, and X. Li, “Clip surgery for better ex- plainability with enhancement in open-vocabulary tasks,” arXiv preprint arXiv:2304.05653, 2023

    2023 arXiv

  53. [61]

    De- diffusion makes text a strong cross-modal interface,

    C. Wei, C. Liu, S. Qiao, Z. Zhang, A. Yuille, and J. Yu, “De- diffusion makes text a strong cross-modal interface,” arXiv preprint arXiv:2311.00618, 2023

    2023 arXiv

  54. [62]

    Looking for trouble: Analyzing classifier behavior via pattern divergence,

    E. Pastor, L. De Alfaro, and E. Baralis, “Looking for trouble: Analyzing classifier behavior via pattern divergence,” in Proceedings of the 2021 International Conference on Management of Data, 2021, pp. 1400–1412

    2021

  55. [63]

    Sliceteller: A data slice-driven approach for machine learning model validation,

    X. Zhang, J. P. Ono, H. Song, L. Gou, K.-L. Ma, and L. Ren, “Sliceteller: A data slice-driven approach for machine learning model validation,” IEEE Transactions on Visualization and Computer Graphics , vol. 29, no. 1, pp. 842–852, 2022

    2022

  56. [64]

    Fairness-aware range queries for selecting unbiased data,

    S. Shetiya, I. P. Swift, A. Asudeh, and G. Das, “Fairness-aware range queries for selecting unbiased data,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 2022, pp. 1423–1436

    2022

  57. [65]

    Tip-Adapter: Training-free adaption of clip for few-shot classification

    R. Zhang, W. Zhang, R. Fang, P. Gao, K. Li, J. Dai, Y . Qiao, and H. Li, “Tip-Adapter: Training-free adaption of clip for few-shot classification.” Berlin, Heidelberg: Springer-Verlag, 2022, p. 493–510. [Online]. Available: https://doi.org/10.1007/978-3-031-19833-5 29

    2022 doi

  58. [66]

    UpSet: visualization of intersecting sets,

    A. Lex, N. Gehlenborg, H. Strobelt, R. Vuillemot, and H. Pfister, “UpSet: visualization of intersecting sets,” IEEE transactions on visualization and computer graphics , vol. 20, no. 12, pp. 1983–1992, 2014

    1983

  59. [67]

    Coco-stuff: Thing and stuff classes in context,

    H. Caesar, J. Uijlings, and V . Ferrari, “Coco-stuff: Thing and stuff classes in context,” in Computer vision and pattern recognition (CVPR), 2018 IEEE conference on . IEEE, 2018

    2018

  60. [68]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3213– 3223. ...

    2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.