Pith. sign in

REVIEW 4 major objections 5 minor 59 references

KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Keyword-based interaction can steer text embeddings toward user intent without requiring users to label or even read individual documents, at a lower workload cost than document-level interaction.

desk verdict Genuinely novel interaction paradigm for tuning embeddings, but the denoising pipeline's load-bearing assumption is validated in the wrong regime. read the letter →

arxiv 2607.20556 v2 pith:ENBBPMCO submitted 2026-07-17 cs.AI

Yan Zhu , Y. Chen , Rebecca Faust This is my paper
classification cs.AI
keywords textembeddingssemanticinteractionhumanfeedbackkeyword-basedconceptspecificationpseudo-supervisionembeddingtuninginteractivevisualizationfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KeySI is an interaction framework that lets users adapt a pretrained text-embedding model by grouping representative keywords into concepts, rather than by manipulating or labeling documents. The paper argues that keyword-level feedback is a lower-cost yet sufficiently strong supervision signal for early-stage concept specification, and that translating keyword groups into clean document-level pseudo-supervision can meaningfully restructure the embedding space. If correct, non-expert users can tailor embedding models to domain-specific semantics through lightweight, inspectable interactions, without labeled data or training-pipeline expertise.

What carries the argument

The central mechanism is a translation pipeline from keyword groups to document-level pseudo-supervision, anchored by a gap-based semantic denoising step. The pretrained embedding space is used as a weak semantic prior: for each keyword, a centroid is computed over its lexically retrieved documents, keyword centroids are averaged to form a concept-group prototype, and each candidate document receives a gap score equal to the difference between its cosine similarity to the best-matching prototype and the second-best prototype. Candidates with a small gap are discarded as ambiguous or off-topic. This denoised pseudo-supervision feeds a two-stage tuning procedure that first repels confusing neg

What would settle it

A concrete observation that would falsify the central claim: on a corpus where a keyword's lexical matches are semantically scattered (e.g., a polysemic term like 'baseball' in crime news versus sports news) and the pretrained embeddings do not cluster those matches by topical intent, the gap-based denoising should fail to improve pseudo-supervision purity compared to lexical retrieval alone, and the tuned embeddings should show no gain in concept-set purity or silhouette.

Watch

Extended reading notes

Core claim

KeySI's central claim is that keyword-level concept specification—users organizing extracted keywords into concept groups, plus an optional Exclude group—provides a supervision signal strong enough to adapt text embeddings, and that this signal generalizes beyond the documents directly retrieved during interaction. The framework operationalizes this by retrieving candidate documents via stemmed lexical matching, denoising them with a gap-based semantic filter, and tuning the encoder with a two-stage loss (semi-hard triplet loss followed by prototype center-pull loss). The paper shows that the resulting embedding space better separates user-specified concepts and improves neighborhood purity,

Load-bearing premise

The gap-based semantic denoising assumes that the pretrained embedding space preserves coarse topical structure well enough that documents matching a user's concept lie closer to that concept's prototype than off-topic lexical matches do; if this weak-prior assumption fails, the denoising will admit noise or discard true positives, and tuning may reinforce the wrong structure.

Editorial extensions

If this is right

  • If the central claim holds, text-embedding adaptation becomes feasible for non-experts: users can steer models by grouping keywords in a visual interface, with no need for labeled datasets or manual document labeling.
  • The improvement in concept-set purity and silhouette scores indicates that the tuning generalizes beyond interaction documents, meaning a small keyword-based interaction can reorganize the entire corpus around the user's concepts.
  • The staged denoising strategy (strict seed, then relaxed recovery after an intermediate update) offers a practical trade-off between supervision precision and coverage, which could be reused in other pseudo-supervision pipelines.
  • The framework's refinement stage, where users reassign boundary documents after tuning, suggests that human-in-the-loop embedding steering benefits from an inspectable, correctable loop rather than a one-shot interaction.
  • The user-study results, if replicated, imply that keyword-level interaction reduces workload and is preferred by users for early-stage concept exploration, while document-level interaction retains value for understanding topics in depth.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the gap-based denoising assumes the pretrained space preserves coarse topical structure; on corpora where this assumption fails (e.g., highly domain-specific texts with deceptive lexical matches), tuning might reinforce the wrong structure. An experiment measuring purity after denoising across corpora with varying topical coherence would directly probe this boundary.
  • The framework's reliance on keyword extraction and curation implies that the quality of the keyword pool limits the ceiling; adaptive keyword ranking or user-driven keyword suggestion could reduce the visual search burden, especially at scale, and is a plausible next step the paper leaves implicit.
  • The distinction between keyword-level and document-level interaction suggests a hybrid interaction model—where users start with keywords and then refine via document-level reassignment—that could combine the lower entry cost of keywords with the richer context of document inspection. The paper touches on this but does not evaluate it as a standalone condition.
  • While the paper evaluates downstream concept purity, the effect on downstream tasks like retrieval, classification, or clustering is not measured; if the tuning genuinely captures user intent, those tasks should also improve for concept-relevant queries, which would be a natural follow-up experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents KeySI, a semantic-interaction framework that lets users steer text embeddings by grouping extracted keywords into concept groups (and an optional Exclude group). KeySI translates these keyword groups into document-level pseudo-supervision via stemmed lexical retrieval, gap-based semantic denoising in the pretrained embedding space, and a two-stage strict/relaxed denoising procedure; it then fine-tunes the encoder with a triplet margin loss and a prototype center-pull loss. The authors argue that keyword-level interaction is a lower-cost and sufficiently strong supervision signal for early-stage concept specification compared with document-level interaction. Evaluation consists of two usage scenarios, an N=8 user study comparing KeySI with a document-manipulation baseline (DeepSI), and quantitative experiments on 20 Newsgroups, a COVID-19 abstract corpus, and AG News, reporting purity and silhouette improvements.

Significance. If the main claims held, KeySI would be a useful addition to visual analytics for embedding adaptation: it lowers the entry barrier for non-expert users and reduces the need to inspect many documents. The paper has clear strengths: the evaluation uses external ground-truth labels that are not used for tuning; repeated runs with different random seeds are reported; a code repository is provided; and the user study includes qualitative/think-aloud analysis. The low-structure COVID scenario is a good stress test. However, gaps in the evaluation of the core denoising mechanism in the target low-separability regime, and in the fairness/statistical reporting of the user study and quantitative experiments, mean that the current evidence is not yet sufficient to support the broad claims as stated.

major comments (4)
  1. [§3.2.3, Eqs. (1)–(5); Tables 1 and 13] The load-bearing assumption of the translation pipeline is that the pretrained embedding space is a weak semantic prior. The only denoising validation is on 20 Newsgroups stress tests with broad topic pairs, where BERT already separates the topics. The motivating COVID-19 scenario, however, is the low-structure regime: Table 2 reports a pre-tuning silhouette of 0.06 for Cancer vs Smoking, and Table 10 reports concept purity at k=10 of 0.392. No purity/NRR/TPL numbers are reported for the denoised COVID pseudo-labels, and no ablation compares tuning with versus without denoising. As a result, it is unclear whether denoising is doing the work in the exact regime KeySI was designed for, or whether lexical matching alone would suffice. Please report denoising metrics on the COVID corpus (and ideally other low-separability domains), and an ablation of the full pipeline with and without gap-ba
  2. [§5.1–5.5, §7] The user study's headline result is lower workload for keyword interaction (KeySI M=1.75 vs DeepSI M=3.63, W=0, p=.004; 8/8 preference). But the DeepSI baseline was the original interface 'without retrieval assistance' (as acknowledged in §7). Participants had to locate representative documents by visual search in the projection; even a simple search box would likely make document-level interaction less costly. Since the paper's central claim is that keyword-level interaction reduces cost relative to document-level interaction, the current comparison conflates interaction modality with the absence of a basic feature. This is acknowledged as a limitation, but the abstract and conclusion still state the stronger claim. The workload comparison should be rerun with a more competitive document-level baseline, or the claim should be explicitly narrowed to 'the implemented prototypes'.
  3. [§6, Fig. 7, Tables 1–2, Supp. Tables 9–11] Quantitative claims of embedding improvement rest on means across five runs, but no variance or significance is reported anywhere in the main text. Fig. 7 has no error bars; Tables 1 and 2 report only means; the supplement reports means only. The statement 'we repeated each experiment five times with different random seeds' is not enough: without seed-level SDs/CIs and paired tests, the reader cannot tell whether the pre/post differences (e.g., silhouette 0.06→0.97 on COVID) are robust or reflect a favorable seed. Please add per-seed results, error bars, and appropriate statistical tests.
  4. [§6.2, Table 10/12] The generalization claim ('beyond just the retrieved documents involved in the interaction') is not fully supported because the overlap between the pseudo-supervision set and the evaluation set is never quantified. For the COVID corpus (62 documents total; Table 12 says 28 documents retrieved), the training set may contain a large fraction of the ground-truth concepts, so concept-set purity could partly reflect memorization rather than generalization. AG News, with 120 training documents in a 7,600-document corpus, is more convincing, but the paper needs to report overlap/Jaccard between the pseudo-supervised documents and the concept ground-truth sets, or evaluate on a held-out split, before making the global-generalization claim.
minor comments (5)
  1. [§3.2.3, Eq. (7)] The center-pull loss uses mu_k, but the prototype in Eq. (2) is p_k. Clarify whether these are the same object or recomputed per batch; the current notation is ambiguous.
  2. [§5.5.1] The Wilcoxon p-values appear consistent with one-sided tests. Please state explicitly whether tests were one-sided or two-sided and justify the choice; this affects the interpretation of p=.031 and p=.004.
  3. [Table 12] For the COVID row, the table lists '3' concept groups, but the usage scenario defined two target groups and an Exclude group. Clarify whether the Exclude group is counted as a concept group here.
  4. [Fig. 1 caption] The caption describes the initial embeddings (left), keyword groups (right), and resulting embeddings (center), but the order is confusing. Consider labeling the panels explicitly as (a), (b), (c).
  5. [Supp. Table 13] Stage 1+2 is not always better than Stage 1 alone (e.g., P5 purity drops from 0.950 to 0.925). The text says the benefit 'varied', but it should explicitly note this non-monotonic behavior to avoid overstating the value of Stage 2.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's key claims rest on user-provided keyword groups and external ground-truth evaluation, not on self-referential fits.

full rationale

The central derivation chain is not circular. User-specified keyword groups are the input; the system translates them into pseudo-labeled documents via lexical retrieval and embedding-based gap filtering (Eqs. 1-5). The quality of that translation is then evaluated against ground-truth labels that were not used to construct the pseudo-labels (Table 1 and Table 13), so the reported purity gains are externally falsifiable rather than forced by definition. The tuning is evaluated with concept-set purity and silhouette computed on full ground-truth concept sets, the majority of whose documents were not directly supervised (Sec. 6.2, Tables 2, 9-11), so the generalization claim is not a renamed fit on the training set. The Stage 2 relaxed recovery does reselect pseudo-supervision using the model's own updated embeddings, which is a self-training loop that could reinforce initial biases; however, this is a stated algorithmic design choice and not a hidden reduction of the evaluation to the model's own affinities, because the reported metrics use external labels. Likewise, the 'weak semantic prior' assumption in Sec. 3.2.3 is a load-bearing correctness assumption, but its failure would be a robustness limitation, not a circularity. There are no load-bearing self-citations: DeepSI, ImageSI, and NeuralSI are contextual related work, and the DeepSI baseline used in the user study is an external system whose implementation limitations are explicitly acknowledged in the Discussion. No fitted parameter is renamed as a prediction, and no uniqueness theorem or prior result is invoked to forbid alternatives. Overall, the paper's derivation is self-contained against external benchmarks.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirically motivated assumptions about embedding spaces and keyword curation; no new ontological entities are introduced. The main tuning hyperparameters are not reported, which affects reproducibility.

free parameters (3)
  • triplet margin m
    Equation (6); not specified in the paper. It controls how far negatives must be pushed and must be chosen by the implementer.
  • Exclude coherence threshold tau_ex
    Section 3.2.3; decides when the Exclude prototype participates in gap ranking. Value not reported, affecting reproducibility.
  • gap thresholds tau_k for Stage 1 and Stage 2
    Section 3.2.3 and Supplemental E describe adaptive, group-specific thresholds estimated from gap-score distributions, but the strictness levels for the two stages are not pinned down.
assumptions (4)
  • domain assumption The pretrained embedding space is a weak semantic prior: concept-consistent documents lie closer to their concept prototype than off-topic lexical matches.
    Section 3.2.3 explicitly states this and uses it to filter pseudo-supervision; if false, the denoising pipeline fails.
  • domain assumption KeyBERT keyword extraction on the pretrained model yields a representative pool of concept-relevant keywords after noun filtering and stemming.
    Section 3.2.1; if the curated keyword pool misses discriminative terms, users cannot express their intended concepts through keyword groups.
  • domain assumption Small pseudo-labeled document sets obtained from keyword groups generalize to the broader corpus and can restructure the global embedding space.
    This is tested empirically in Experiment 2, but it is assumed by the design in Section 3.3 and underpins the claim that low-cost interaction suffices.
  • domain assumption Fine-tuning a BERT encoder with triplet and center-pull losses preserves useful pretrained semantics while separating user concepts.
    Section 3.3; the paper does not analyze catastrophic forgetting or downstream task degradation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback." pith.science (2026). https://pith.science/paper/ENBBPMCO

@misc{pith2026260720556,
  author       = {Pith},
  title        = {Pith review of: KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ENBBPMCO}},
  note         = {Machine review of arXiv:2607.20556}
}
read the original abstract

In large-scale text analysis tasks, pre-trained language models are often used to embed text corpora for downstream analysis. However, such models may struggle to capture domain-specific semantics and adapting them typically requires large amounts of labeled data and technical expertise to implement training pipelines. Recent approaches have demonstrated how visual interactions in document projections can capture human feedback as training signals for model tuning. However, these methods operate on document-level feedback, which requires users to open and assess individual documents in order to provide effective feedback. In this paper, we propose KeySI, an interaction framework that enables feature-level feedback through keyword-based concept specification. Users specify feedback by organizing extracted keywords into groups representing concepts, which KeySI translates into document-level supervision for subsequent tuning. By operating on keywords as the primary interaction medium, KeySI reduces the need for manual document inspection and labeling and lowers the barrier to adapting embedding models. We present a prototype implementation that, given a corpus, curates representative keywords, visualizes keywords and document embeddings via dimensionality reduction, allows interactive specification of keyword groups, and supports iterative refinement through system feedback. We evaluate KeySI through a user study, usage scenarios, and quantitative experiments demonstrating its effectiveness in capturing user intent and improving embedding alignment.

Figures

Figures reproduced from arXiv: 2607.20556 by the authors.

Figure 1
Figure 1. KeySI on a dataset of COVID-19 article abstracts containing four risk factors: smoking, cancer, neurological, and kidney. The [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. KeySI UI. (A) Controls for setting the number of keyword groups and starting model tuning.(B) A keyword view for browsing and selecting extracted keywords.(C) A 2D document projection in which documents associated with the selected keyword or concept group are highlighted.(D) A group panel showing user-created keyword groups and the Exclude group.(E) A document list showing documents associated with the selected key… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: KeySI System Pipeline. KeySI first curates a set of keywords, then presents visualization of keywords and documents for concept specification. Next, it translates keyword-level feedback into document￾level feedback and uses this feedback to tune the model. Note that it…
Figure 5
Figure 5. Figure 5: 20 Newsgroups Dataset. The user specifies two concepts, space and computers, using the keyword view (left), resulting in the groups on the right. In the initial model (center) these concepts are not cleanly structured. After the interaction (right), they are better sep…
Figure 6
Figure 6. Figure 6: Subjective workload (N=8). Bars show mean ratings for each NASA TLX question (lower is better), with 95% confidence intervals. * indicates significance after Bonferroni correction. advantage: full documents provided richer context for understanding topics (P2, P5, P6-P…
Figure 7
Figure 7. Figure 7: Concept-set purity at k=10 (pre → tuned). Each line shows one dataset/task pair. Higher is better. Setup: To evaluate whether semantic denoising removes noisy matches, we use the same curated six-class subset of 20 Newsgroups as in the usage scenario. We report results…
Figure 8
Figure 8. Figure 8: AGNews Dataset. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Refinement interface. Users can inspect documents in the projection to identify concept-irrelevant documents. The Refinement panel, in the bottom right, allows users to move documents to the groups or exclude them. The panel shows a preview of documents as well as a hi…
Figure 10
Figure 10. Figure 10: Post Refinement Projection. The embeddings after refinement. The boundaries become cleaner after refinement. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 7 canonical work pages

  1. [1]

    A. I. F. AI, A. Goldbloom, B. Hamner, C. Schoenick, T. Bozsolik, P. Mooney et al. COVID-19 Open Research Dataset Challenge (CORD- 19), 2022. 5

    2022

  2. [2]

    Bekkerman, H

    R. Bekkerman, H. Raghavan, J. Allan, and K. Eguchi. Interactive cluster- ing of text collections according to a user-specified criterion. InIJCAI, vol. 7, pp. 684–689, 2007. 3

    2007

  3. [3]

    Y . Bian, R. Faust, and C. North. Neuralsi: Neural design of semantic interaction for interactive deep learning.arXiv preprint arXiv:2402.17178,

    arXiv

  4. [4]

    Bian and C

    Y . Bian and C. North. Deepsi: Interactive deep learning for semantic interaction. InProc. 26th International Conference on Intelligent User Interfaces, pp. 197–207, 2021. doi: 10.1145/3397481.3450670 1, 2, 3, 6

    arXiv 2021

  5. [5]

    Y . Bian, J. Wenskovitch, and C. North. Deepva: Bridging cognition and computation through semantic interaction and deep learning. In2019 IEEE Workshop on Machine Learning from User Interaction for Visualization and Analytics (MLUI), pp. 1–10. IEEE, 2019. doi: 10.1109/mlui52769. 2019.10075565 3

    arXiv 2019

  6. [6]

    S. Bird. Nltk: the natural language toolkit. InProc. COLING/ACL interactive presentation sessions, pp. 69–72, 2006. doi: 10.3115/1225403. 1225421 4

    work page doi:10.3115/1225403 2006

  7. [7]

    Bradel, N

    L. Bradel, N. Wycoff, L. House, and C. North. Big text visual analytics in sensemaking. In2015 Big Data Visual Analytics (BDVA), pp. 1–8. IEEE,

  8. [8]

    K. Caine. Local standards for sample size at CHI. InProc. SIGCHI Conference on Human Factors in Computing Systems, pp. 981–992. ACM, New York, NY , 2016. doi: 10.1145/2858036.2858498 6

    arXiv 2016

Show all 59 references
  1. [9]

    J. Choo, C. Lee, C. K. Reddy, and H. Park. UTOPIAN: User-driven topic modeling based on interactive nonnegative matrix factorization.IEEE Transactions on Visualization and Computer Graphics, 19(12):1992–2001,

    1992

  2. [10]

    Dowling, N

    M. Dowling, N. Wycoff, B. Mayer, J. Wenskovitch, L. House, N. Polys et al. Interactive visual analytics for sensemaking with big text.Big Data Research, 16:49–58, 2019. doi: 10.1016/j.bdr.2019.04.003 2

    2019 doi

  3. [11]

    El-Assady, R

    M. El-Assady, R. Kehlbeck, C. Collins, D. A. Keim, and O. Deussen. Semantic concept spaces: Guided topic model refinement using word- embedding projections.IEEE Transactions on Visualization and Computer Graphics, 26(1):1001–1011, 2020. doi: 10.1109/TVCG.2019.2934665 2

    2020

  4. [12]

    Endert, P

    A. Endert, P. Fiaux, and C. North. Semantic interaction for visual text analytics. InProc. SIGCHI conference on Human factors in computing systems, pp. 473–482, 2012. doi: 10.1145/2207676.2207741 3

    2012

  5. [13]

    Faruqui, J

    M. Faruqui, J. Dodge, S. K. Jauhar, C. Dyer, E. Hovy, and N. A. Smith. Retrofitting word vectors to semantic lexicons. InProceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1606– 16...

    2015 doi

  6. [14]

    C. Görg, Z. Liu, J. Kihm, J. Choo, H. Park, and J. Stasko. Combining computational analyses and interactive visualization for document explo- ration and sensemaking in jigsaw.IEEE transactions on Visualization and Computer Graphics, 19(10):1646–1663, 2012. doi: 10.1109/tvcg.20...

    2012 doi

  7. [15]

    Grootendorst

    M. Grootendorst. Keybert: Minimal keyword extraction with bert, 2020. doi: 10.5281/zenodo.4461265 4

    2020 doi

  8. [16]

    S. G. Hart and L. E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati, eds.,Human Mental Workload, pp. 139–183. North-Holland,

  9. [17]

    X. Ho, A. K. D. Nguyen, A. T. Dao, J. Jiang, Y . Chida, K. Sugimoto et al. A survey of pre-trained language models for processing scientific text,

  10. [18]

    Y . Hou, J. Li, Z. He, A. Yan, X. Chen, and J. McAuley. Bridging language and items for retrieval and recommendation.arXiv preprint arXiv.2403.03952, 2024. doi: 10.48550/arXiv.2403.03952 6

    2024 doi arXiv

  11. [19]

    Y . Hu, E. E. Milios, and J. Blustein. Interactive feature selection for document clustering. InProc. ACM symposium on applied computing, pp. 1143–1150, 2011. doi: 10.1145/1982185.1982436 3

    2011

  12. [20]

    Y . Hu, E. E. Milios, and J. Blustein. Interactive document clustering with feature supervision through reweighting.Intelligent Data Analysis, 18(4):561–581, 2014. doi: 10.3233/ida-140658 3

    2014 doi

  13. [21]

    Huang, D

    Z. Huang, D. Witschard, K. Kucher, and A. Kerren. V A + embeddings STAR: A state-of-the-art report on the use of embeddings in visual analyt- ics.Computer Graphics Forum, 42(3):539–571, 2023. doi: 10.1111/cgf. 14859 2

    2023 doi

  14. [22]

    doi: 10.48550/arXiv.2401.17824 1

    doi arXiv

  15. [23]

    Kiefer, M

    S. Kiefer, M. Hoffmann, and U. Schmid. Semantic interactive learning for text classification: a constructive approach for contextual interactions. Machine Learning and Knowledge Extraction, 4(4):994–1010, 2022. doi: 10.3390/make4040050 2

    2022 doi

  16. [24]

    M. S. Lam, J. Teoh, J. A. Landay, J. Heer, and M. S. Bernstein. Concept induction: Analyzing unstructured text with high-level concepts using lloom. InProc. CHI Conference on Human Factors in Computing Systems. ACM, 2024. doi: 10.1145/3613904.3642830 1

    2024

  17. [25]

    H. Lee, J. Kihm, J. Choo, J. Stasko, and H. Park. ivisclustering: An interactive visual document clustering via topic modeling. InComputer graphics forum, vol. 31, pp. 1155–1164. Wiley Online Library, 2012. doi: 10.1111/j.1467-8659.2012.03108.x 3

    2012

  18. [26]

    J. Lin, R. Faust, and C. North. Imagesi: Semantic interaction for deep learning image projections. InIEEE Visualization and Visual Analytics (VIS), pp. 91–95. IEEE, 2024. doi: 10.1109/vis55277.2024.00026 1, 2, 3

    2024

  19. [27]

    Kang and J

    Y .-a. Kang and J. Stasko. Examining the use of a visual analytics system for sensemaking tasks: Case studies with domain experts.IEEE Transactions on Visualization and Computer Graphics, 18(12):2869–2878, 2012. doi: 10.1109/tvcg.2012.224 2

    2012 doi

  20. [28]

    Meinecke, D

    C. Meinecke, D. J. Wrisley, and S. Jänicke. Explaining semi-supervised text alignment through visualization.IEEE Transactions on Visualization and Computer Graphics, 28(12):4797–4809, 2022. doi: 10.1109/TVCG. 2021.3105899 2

    2022

  21. [29]

    Mishra and J

    S. Mishra and J. M. Rzeszotarski. Designing interactive transfer learning tools for ml non-experts. InProc. CHI Conference on Human Factors in Computing Systems, pp. 1–15, 2021. doi: 10.1145/3411764.3445096 1, 2

    2021

  22. [30]

    Mitchell

    T. Mitchell. Twenty newsgroups, 1997. 6

    1997

  23. [31]

    Nourashrafeddin, E

    S. Nourashrafeddin, E. Milios, and D. Arnold. Interactive text document clustering using feature labeling. InProc. ACM symposium on Document Engineering, pp. 61–70, 2013. doi: 10.1145/2494266.2494279 3

    2013

  24. [32]

    S. Liu, M. X. Zhou, S. Pan, W. Qian, W. Cai, and X. Lian. Interactive, topic-based visual text summarization and analysis. InProc. 18th ACM conference on Information and knowledge management, pp. 543–552,

  25. [33]

    Rezaeipourfarsangi, N

    S. Rezaeipourfarsangi, N. Pei, E. Sherkat, and E. Milios. Interactive clustering and high-recall information retrieval using language models. In Proc. International Conference on Advanced Visual Interfaces, pp. 1–5,

  26. [34]

    P. J. Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.Journal of computational and applied mathematics, 20:53–65, 1987. doi: 10.1016/0377-0427(87)90125-7 8

    1987 doi

  27. [35]

    Ruppert, M

    T. Ruppert, M. Staab, A. Bannach, H. Lücke-Tieke, J. Bernard, A. Kuijper et al. Visual interactive creation and validation of text clustering workflows to explore document collections.Electronic Imaging, 29:46–57, 2017. doi: 10.2352/issn.2470-1173.2017.1.vda-388 3

    2017

  28. [36]

    Sakai and S

    H. Sakai and S. Masuyama. A multiple-document summarization system with user interaction. InProc. 20th International Conference on Computa- tional Linguistics, pp. 1001–1007, 2004. doi: 10.3115/1220355.1220499 2

    2004

  29. [37]

    P. Sen, Y . Li, E. Kandogan, Y . Yang, and W. Lasecki. Heidl: Learning linguistic expressions with deep learning and human-in-the-loop. InProc. 57th Annual Meeting of the Association for Computational Linguistics: 10 © 2026 IEEE. This is the author’s version of the article tha...

    2026 doi

  30. [38]

    D. Park, S. Kim, J. Lee, J. Choo, N. Diakopoulos, and N. Elmqvist. ConceptVector: Text visual analytics via interactive lexicon building using word embedding.IEEE Transactions on Visualization and Computer Graphics, 24(1):361–370, 2018. doi: 10.1109/TVCG.2017.2744478 2

    2018

  31. [39]

    Sherkat, S

    E. Sherkat, S. Nourashrafeddin, E. E. Milios, and R. Minghim. Interactive document clustering revisited: A visual analytics approach. InProc. 23rd International Conference on Intelligent User Interfaces, pp. 281–292,

  32. [40]

    Stasko, C

    J. Stasko, C. Gorg, Z. Liu, and K. Singhal. Jigsaw: supporting investigative analysis through interactive visualization. InIEEE Symposium on Visual Analytics Science and Technology, pp. 131–138. IEEE, 2007. doi: 10. 1109/vast.2007.4389006 2

    2007

  33. [41]

    van der Maaten and G

    L. van der Maaten and G. Hinton. Visualizing data using t-sne.Journal of Machine Learning Research, 9(86):2579–2605, 2008. 4

    2008

  34. [42]

    H. Wang, J. Li, H. Wu, E. Hovy, and Y . Sun. Pre-trained language models and their applications.Engineering, 25:51–65, 2023. doi: 10.1016/j.eng. 2022.04.024 1

    2023 doi

  35. [43]

    Y . Yang, E. Kandogan, Y . Li, P. Sen, and W. S. Lasecki. A study on interaction in human-in-the-loop machine learning for text analytics. In IUI Workshops, 2019. 2 11 Supplemental Material KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback Yan ...

    2019

  36. [45]

    Sherkat, E

    E. Sherkat, E. E. Milios, and R. Minghim. A visual analytics approach for interactive document clustering.ACM Transactions on Interactive Intelligent Systems (TiiS), 10(1):1–33, 2019. doi: 10.1145/3241380 3

    2019 doi

  37. [52]

    Avoid common words that appear across multiple topics

    High concept specificity.Prefer domain-specific terms, named entities, technical nouns, or abbreviations. Avoid common words that appear across multiple topics

  38. [53]

    Terms such as player, student, or motorcycle may correspond to multiple topics and lead to mixed candidates

    Low ambiguity.Prioritize keywords whose meaning is stable within the dataset context. Terms such as player, student, or motorcycle may correspond to multiple topics and lead to mixed candidates

  39. [54]

    Avoid adding many near-synonyms that provide little additional infor- mation

    Complementary coverage.Keywords within a group should cover different facets of the concept (e.g., subtopics, entities, events) while maintaining a consistent semantic direction. Avoid adding many near-synonyms that provide little additional infor- mation

  40. [55]

    Practical selection workflow.We recommend an iterative work- flow:

    High verifiability.Keywords should allow users to reliably judge relevance via quick inspection (e.g., top-keyword previews or short snippets), before resorting to full-text reading. Practical selection workflow.We recommend an iterative work- flow:

  41. [56]

    Start with the most specific anchor terms.Add 1–2 highly representative keywords first and inspect whether retrieved can- didates and projection highlights form a relatively concentrated region

  42. [57]

    If candi- dates clearly span multiple themes or the keyword meaning shifts across contexts, replace that keyword

    Use candidate previews for rapid quality checks.Inspect the candidate list using top-keyword previews or short snippets (and open full text when needed) to estimate topic mixing. If candi- dates clearly span multiple themes or the keyword meaning shifts across contexts, replac...

  43. [58]

    Add keywords only to address two cases:(i)insufficient cov- erage(too few candidates / sparse regions), where adding more specific entities or terms within the same concept can improve recall; and (ii)boundary mixing(candidates overlap with other concept regions), where users ...

  44. [59]

    hard to tune

    Iterate rather than aiming for a perfect initial definition.In open-ended exploration, keyword groups typically converge from coarse to fine; users should refine groups progressively based on system feedback. Common pitfalls.We observed several recurring issues: • Using generi...

    2026

  45. [1988]

    doi: 10.1016/S0166-4115(08)62386-9 7

    doi

  46. [2009]

    doi: 10.1145/1645953.1646023 2

  47. [2013]

    doi: 10.1109/TVCG.2013.212 2

    2013 doi

  48. [2015]

    doi: 10.1109/bdva.2015.7314287 2

    2015

  49. [2018]

    doi: 10.1145/3172944.3172964 3

  50. [2022]

    doi: 10.1145/3531073.3531174 3

  51. [2024]

    doi: 10.48550/arXiv.2402.17178 1

    doi arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.