Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections

Christopher Collins; Daniel Keim; Mennatallah El-Assady; Oliver Deussen; Rebecca Kehlbeck

REVIEW 3 major objections 4 minor 59 references

Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hand-editing word-embedding maps makes topic models more distinct.

desk verdict A well-engineered visual analytics system for injecting domain knowledge into topic models, but the empirical claim of quality improvement is not supported by the reported numbers. read the letter →

arxiv 1908.00475 v1 pith:WETWZNDR submitted 2019-08-01 cs.HC cs.CLcs.IR

Mennatallah El-Assady , Rebecca Kehlbeck , Christopher Collins , Daniel Keim , Oliver Deussen This is my paper · ORCID

classification cs.HCcs.CLcs.IR

keywords topicmodelrefinementwordembeddingsvisualanalyticshuman-in-the-loopconcepthierarchysemanticinteractionuserguidancemachineteaching

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a non-expert can improve an automatic topic model by directly editing a two-dimensional map of word meanings. The interface shows words as points arranged by a word-embedding projection, lets users regroup words into concepts, and then retrains the topic model using the revised word weights as constraints. The authors report that this human-in-the-loop process made topics more distinct and that the refined concept definitions carried over to other document collections.

What carries the argument

The machinery is a pair of parallel hierarchies over one shared word-embedding space: the user-driven concept hierarchy (base words, descriptors, concepts, super-concepts) and the data-driven topic hierarchy (keywords, documents, topics). The load-bearing link is a weighted word vector: every word carries scores for its relevance to concepts, topics, documents, and the corpus, and user edits alter those weights, which are then used to readjust keyword weighting in topic-model training. A concept-anchored t-SNE projection and topic glyphs with spikes to related concepts make the semantic relations visible and actionable.

What would settle it

Instrument a run of the system and log the word weights before and after a single concept edit; if the edit does not change the weights that feed topic-model retraining, the central mechanism is not operating. A second check: have independent users refine the same corpus from the same starting model and compare the eight quality metrics; if distinctiveness gains are not reproducible across users, the claim of robust improvement fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that user knowledge can be externalized as a hierarchy of concepts over a word-embedding space, and that changing this hierarchy changes the scoring of words, which in turn reweights the keywords used by topic modeling. The refinement acts as must-link and cannot-link constraints, so promoting, demoting, merging, or reassigning words teaches the model the user's semantics. Two user studies and an annotation study are offered as evidence that the resulting topics are more distinct and that guided recommendations achieve gains with less feedback.

Load-bearing premise

The load-bearing premise is that a user's edits to the concept hierarchy are translated reliably into changed word weights and constraints for the topic model; the paper describes this mapping in words but gives no equations or algorithm for it.

Editorial extensions

If this is right

Domain experts can refine topics without touching the underlying model, because the interaction happens in the concept space rather than in algorithm parameters.
Concepts refined on one corpus can seed the analysis of a related corpus, avoiding a cold start.
The guided recommendation queue targets high-impact words, so small numbers of accepted suggestions can produce visible quality gains.
The evaluation's quantitative result—distinctiveness rising sharply while coherence and separation fall slightly—implies that refinement buys interpretable separation at some cost to statistical coherence.
Must-link and cannot-link constraints can be expressed through spatial editing rather than through explicit rule specification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An unstated corollary is that the same interaction scheme could steer other embedding-based models, such as clustering or retrieval, by treating the edited concept hierarchy as a prior over the vector space.
The transferability claim suggests a practical test the paper does not run: refine concepts on one debate corpus, apply them to a second debate, and compare topic quality against a cold-start model.
Because the user's edits are meant to act as constraints, instrumenting the pipeline to log exact weight changes would let future work verify that the system's concept-to-model mapping matches user intent.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a serious systems paper with a genuinely useful interaction design, but the abstract overclaims. The headline 'topic model quality improvements' is not backed by the quantitative results in Section 6.2; the qualitative findings are the real contribution.

What's new: the dual-hierarchy design is a real combination. Users edit a concept hierarchy on top of a data-driven topic hierarchy, both in the same embedding space, and the refinements feed back into a model-agnostic topic model update. The guided relevance feedback ('magic wand') and cross-corpus concept transfer are practical additions beyond ConceptVector and UTOPIAN. The paper works through the visualization design carefully: t-SNE anchored by concept vectors, quadtree-based overlap reduction, Voronoi super-concept boundaries, and the topic glyph encoding are all sensible and described with enough detail to reproduce.

The qualitative study is the strongest part. Six participants from political science and linguistics found the tool useful, and the quoted feedback is concrete. The authors are honest about mixed reactions to the guidance component.

The soft spot is the evaluation of the central claim. The eight quality metrics in Section 6.2 show average relative changes: coherence -5.5%, separation -12.1%, distinctiveness +331%, PMI +4.3%, certainty +0.7%, branching factor -26.5%, compactness -11.8%, topic size +1.5%. Five of the eight move in the wrong direction. The paper then says topics became 'significantly more distinct' without any significance test. No p-values, no per-participant breakdown, no confidence intervals. A reader cannot take 'confirm improvements' from these numbers. At best, this supports a trade-off (more distinct but less coherent). The annotation study uses four annotators and ranks five models, but the manual and guided models come from different procedures, so it's not a controlled paired comparison. Also, the quality metrics and the guidance recommender both draw on the authors' earlier work [16], so the loop may be optimizing its own objectives. That's not disqualifying, but it doesn't give independent evidence.

The other gap is Section 5.2: the mapping from user edits in the concept hierarchy to changes in the topic model is described qualitatively ('must-link' and 'cannot-link' constraints) with no equations or algorithm. For a paper whose whole premise is that this mapping works, that's a real hole.

Who is this for? Researchers in visual analytics and interactive machine learning, especially anyone building human-in-the-loop topic modeling tools. If I were an editor, I'd send it out. The system is real, the design is thoughtful, and the qualitative results are worth publishing. But the authors need to either soften the improvement claim to 'trade-offs and perceived quality' or provide proper statistical analysis. My recommendation: engage with it, but tell the authors to fix the evaluation before it's citable as evidence for quality improvement.

Referee Report

3 major / 4 minor

Summary. The paper presents Semantic Concept Spaces, a visual analytics framework that lets users refine a topic model by manipulating a concept hierarchy in a word-embedding space. The system maintains two parallel hierarchies—a user-driven concept hierarchy and a data-driven topic hierarchy—over a shared vector space, and maps user interactions into word-weight adjustments that act as must-link/cannot-link constraints for topic model retraining. The interface supports direct manipulation, guided relevance feedback through recommended refinements, and topic glyphs that expose conceptual associations. The authors report qualitative feedback from six experts and quantitative quality-metric changes, plus a four-annotator ranking of five concept-space/topic-model outputs.

Significance. If the central claim were fully substantiated, this would be a useful contribution to human-in-the-loop topic modeling: the dual-hierarchy design, the transferability of refined concepts across corpora, and the explicit guidance component are novel and well-motivated. The qualitative study provides encouraging evidence that domain experts can externalize knowledge and see model responses. However, the quantitative evidence is currently too weak to support the abstract's 'confirm improvements' claim. The mixed metric changes, absence of significance testing, and the informal mapping between user actions and model updates mean the paper's main claim is not yet established. The strengths are the system's availability, the clear separation of concept and topic hierarchies, and the thoughtful discussion of interaction design.

major comments (3)

[Section 5.2] The central mechanism linking user edits to topic model changes is not specified. The text states that 'we use the learned weights and scores from the concept refinement to readjust the keyword weighting for the topic model training. These act as "must-link" and "cannot-link" constraints' but gives no equations, pseudocode, or formal description of how a hierarchy-level change (promotion, demotion, reassignment, merge) translates into a concrete weight change or constraint. This is load-bearing because the framework's claim of being a model-agnostic refinement method depends on this mapping. Please provide the precise update rule, including how the hierarchy level and descriptor-concept assignments affect the keyword weights and how the IHTM (or any topic model) consumes these constraints.
[Section 6.2] The quantitative results do not support the claim of topic-model quality improvements. Of the eight reported metrics, five move in the adverse direction (Coherence -5.49%, Separation -12.09%, Branching Factor -26.47%, Compactness -11.77%, Topic Size +1.45%), and only Distinctiveness shows a large positive change (+331.31%). The sentence 'topics became significantly more distinct' is unsupported because no significance test, confidence interval, or per-participant variance is reported. The average relative changes alone are insufficient to establish that the refinements improve topic quality; at best they indicate a trade-off. Please report the underlying per-participant values, the distribution of changes, and appropriate inferential statistics, or revise the central claim to describe a trade-off rather than an overall improvement.
[Table 1] The annotation study uses only four annotators and compares outputs that were produced under different procedures: the manual refinement model came from one participant in the first study, while the guided refinement model was generated through a different process. There is no paired design, no control for annotator differences, and no statistical comparison. The conclusion that 'manual refinement of the concept space yields the most well-perceived concept view, while the guided topic refinement leads to the highest ranking topic modeling result' is not supported by the reported rank means and standard deviations. Please either provide inferential statistics appropriate for the small sample or clearly present these results as descriptive observations that cannot be used to validate the central improvement claim.

minor comments (4)

[Section 3.2] The t-SNE parameters (perplexity=5, theta=0.5, 5000 learning iterations) are stated but no rationale or sensitivity analysis is given; since the projection underpins the entire visual workspace, a brief justification or reference would improve reproducibility.
[Section 6.2] It is unclear whether the reported average relative changes are computed across all six expert participants and whether each participant had multiple refinement cycles; please clarify the exact unit of analysis and the number of models compared.
[Section 6.1] The quote describing the interface as a 'neat combination of ecstatically pleasing components' appears to contain a typo; 'aesthetically pleasing' seems intended.
[Abstract] The abstract's phrase 'We confirm the improvements achieved through our approach' overstates the evidence given the mixed quantitative results and lack of significance tests; consider softening the language to match the actual findings.

Circularity Check

1 steps flagged · score 4.0 of 10

Quantitative "topic model quality improvements" are measured with the same metrics that drive the guided refinement, so the improvement claim is partly self-referential.

fitted input called prediction [Section 5.1 (Quality Monitoring / Refinement Recommendation) and Section 6.2 (Quantitative Results)]
""the quality monitoring component evaluates the internal quality of the topic modeling based on the criteria outlined in our previous work [16]" ... "Based on the results of the quality monitoring, the recommender keeps a constantly-updated queue of words and their suggested actions" ... "The average relative change, from the initial model to the refined model, based on the eight observed quality metrics [16] was as follows...""

The same metrics from [16] are used twice: first as the objective for the guided refinement recommender (Section 5.1), and second as the outcome measure for the claimed 'topic model quality improvements' (Section 6.2). A refinement queue that ranks words by these metrics and suggests actions to improve them will tend to move the same metrics in the reported direction even if the user's semantic knowledge contributes nothing. The abstract's confirmation claim therefore rests in part on an optimisation target being relabelled as an independent evaluation. The circularity is partial because participants could accept or reject suggestions and direct manipulation is not purely metric-driven, and because the annotation study offers separate qualitative evidence.

full rationale

The derivation chain is not formally specified: Section 5.2 says the learned weights 'act as must-link and cannot-link constraints' without equations, so no exact Eq X = Eq Y can be exhibited. The main circularity is the identity between the guidance objective and the evaluation metric: both come from the authors' prior work [16]. This makes the quantitative improvement claim partially self-referential. The negative average changes in five of eight metrics (coherence -5.49%, separation -12.09%, branching factor -26.47%, compactness -11.77%) also undercut the headline 'improvements'; the large distinctiveness gain is reported without significance testing. The annotation study (Table 1) provides some independent grounding, but it compares non-paired outputs with four annotators. Overall, the central claim has independent content in the visual interaction design and qualitative results, so score 4 rather than higher.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the assumption that user edits reliably change word weights and topic model constraints, and that the quality metrics capture meaningful improvement. The t-SNE and clustering thresholds are hand-set, and the update mechanism is described only at a high level.

free parameters (6)

t-SNE perplexity = 5
Chosen by projection inspection trials; affects spatialization and concept hierarchy building.
t-SNE theta = 0.5
Chosen by projection inspection trials; affects projection speed and accuracy.
t-SNE learning iterations = 5000
Chosen by projection inspection trials; affects convergence of the spatialization.
epsilon_similarity = 0.4
Default semantic cosine similarity threshold for hierarchical clustering; directly affects the concept hierarchy.
epsilon_neighborhood = 6
Default minimum neighborhood size for clustering; affects concept and super-concept abstraction levels.
top n,m keywords = 15
Default number of keywords per document and topic; affects the topic view and keyword weighting.

assumptions (5)

domain assumption User domain knowledge, externalized through concept edits, improves topic model semantic quality.
The entire approach assumes that users can teach the model useful semantics; this is the basis for the evaluation and the claimed improvements.
domain assumption Word embeddings from ConceptNet capture meaningful semantic similarity for the corpus words.
Concept vector expansion and semantic similarity computations rely on this; no validation of embedding quality is provided.
ad hoc to paper The mapping from hierarchy level changes to word weight changes and topic model constraints is well-defined.
Section 5.2 asserts that learned weights and scores readjust keyword weighting as must-link/cannot-link constraints, but no equations or algorithmic details are given.
standard math t-SNE projection preserves semantic neighborhoods sufficiently for clustering and concept hierarchy building.
This is a standard property of t-SNE, but the choice of anchors and parameters is heuristic.
domain assumption The eight quality metrics from [16] are appropriate measures of topic model quality.
The evaluation and the guidance recommender both depend on these metrics; their validity for capturing user-perceived quality is not independently established.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections." pith.science (2026). https://pith.science/paper/WETWZNDR

@misc{pith2026190800475,
  author       = {Pith},
  title        = {Pith review of: Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WETWZNDR}},
  note         = {Machine review of arXiv:1908.00475}
}

read the original abstract

We present a framework that allows users to incorporate the semantics of their domain knowledge for topic model refinement while remaining model-agnostic. Our approach enables users to (1) understand the semantic space of the model, (2) identify regions of potential conflicts and problems, and (3) readjust the semantic relation of concepts based on their understanding, directly influencing the topic modeling. These tasks are supported by an interactive visual analytics workspace that uses word-embedding projections to define concept regions which can then be refined. The user-refined concepts are independent of a particular document collection and can be transferred to related corpora. All user interactions within the concept space directly affect the semantic relations of the underlying vector space model, which, in turn, change the topic modeling. In addition to direct manipulation, our system guides the users' decision-making process through recommended interactions that point out potential improvements. This targeted refinement aims at minimizing the feedback required for an efficient human-in-the-loop process. We confirm the improvements achieved through our approach in two user studies that show topic model quality improvements through our visual knowledge externalization and learning process.

Figures

Figures reproduced from arXiv: 1908.00475 by the authors.

**Figure 2.** The human-in-the-loop workflow for Semantic Concept Spaces. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Semantic Abstraction Levels for Concepts. By default, the entry point for the visualization (0) shows all major concepts. Users can opt to [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Duality of Concept and Topic Views. Selected layers from each view ‘ [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Two options for Concept Refinement. The direct manipulation enables exploratory refinement, while the guided relevance feedback is [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 44 canonical work pages

[16]

El-Assady, F

M. El-Assady, F. Sperrle, O. Deussen, D. Keim, and C. Collins. Visual Analytics for Topic Model Optimization based on User-Steerable Specu- lative Execution. IEEE Trans. on Visualization and Computer Graphics, 25(1):374–384, 2019. doi: 10.1109/TVCG.2018.2864769

arXiv 2019
[1]

Alexander, C.-C

E. Alexander, C.-C. Chang, M. Shimabukuro, S. Franconeri, C. Collins, and M. Gleicher. Perceptual biases in font size as a data encoding. IEEE Trans. on Visualization and Computer Graphics, 24(8):2397–2410, 2017. doi: 10.1109/TVCG.2017.2723397

work page arXiv 2017
[2]

Alexander, J

E. Alexander, J. Kohlmann, R. Valenza, M. Witmore, and M. Gleicher. Serendip: Topic model-driven visual exploration of text corpora. In Proc. IEEE Symp. on Visual Analytics Science and Technology (VAST), pp. 173– 182, 2014

work page 2014
[3]

Andrzejewski, X

D. Andrzejewski, X. Zhu, and M. Craven. Incorporating domain knowl- edge into topic modeling via dirichlet forest priors. In Proc. Int. Conf. on Machine Learning, pp. 25–32, 2009

work page 2009
[4]

D. M. Blei. Probabilistic topic models. Communications of the ACM , 55(4):77–84, 2012. doi: 10.1145/2133806.2133826

arXiv 2012
[5]

D. M. Blei, A. Y . Ng, and M. I. Jordan. Latent Dirichlet Allocation.J. of Machine Learning Research, 3:993–1022, 2003

work page 2003
[6]

Cavallo and C ¸

M. Cavallo and C ¸. Demiralp. A visual interaction framework for dimen- sionality reduction based data exploration. Proc. SIGCHI Conf. on Human Factors in Computing Systems, 2018

work page 2018
[7]

Z. Chen, A. Mukherjee, B. Liu, M. Hsu, and M. Castellanos. Leveraging multi-domain prior knowledge in topic models. Proc. Int. Joint Conf. on Artiﬁcial Intelligence, pp. 2071–2077, 2013

work page 2013

Show all 59 references

[8]

J. Choo, C. Lee, C. K. Reddy, and H. Park. UTOPIAN: User-driven topic modeling based on interactive nonnegative matrix factorization. IEEE Trans. on Visualization and Computer Graphics, 19(12):1992–2001, Dec. 2013

1992
[9]

Collins, N

C. Collins, N. Andrienko, T. Schreck, J. Yang, J. Choo, U. Engelke, A. Jena, and T. Dwyer. Guidance in the human–machine analytics process. Visual Informatics, 2(3):166–180, 2018

2018
[10]

W. Dou, X. Wang, R. Chang, and W. Ribarsky. ParallelTopics: A proba- bilistic approach to exploring document collections. In Conf. on Visual An- alytics Science and Technology, pp. 231–240, 2011. doi: 10.1109/V AST. 2011.6102461

2011
[11]

W. Dou, L. Yu, X. Wang, Z. Ma, and W. Ribarsky. HierarchicalTopics: Visually exploring large text collections using topic hierarchies. IEEE Trans. on Visualization and Computer Graphics, 19(12):2002–2011, 2013

2002
[12]

Dowling, J

M. Dowling, J. Wenskovitch, P. Hauck, A. Binford, N. Polys, and C. North. A bidirectional pipeline for semantic interaction. In Proc. Workshop on Machine Learning from User Interaction for Visualization and Analytics (at IEEE VIS 2018), vol. 11, 2018

2018
[13]

El-Assady, A

M. El-Assady, A. Hautli-Janisz, V . Gold, M. Butt, K. Holzinger, and D. Keim. Interactive visual analysis of transcribed multi-party discourse. In Proc. of Association for Computational Linguistics, ACL System Demon- strations, pp. 49–54. ACL, 2017. doi: 10.18653/v1/P17-4009

2017 doi
[14]

El-Assady, W

M. El-Assady, W. Jentner, F. Sperrle, R. Sevastjanova, A. Hautli-Janisz, M. Butt, and D. Keim. lingvis.io - A Linguistic Visual Analytics Frame- work. In Proc. of Association for Computational Linguistics, ACL System Demonstrations. ACL, 2019

2019
[15]

El-Assady, R

M. El-Assady, R. Sevastjanova, F. Sperrle, D. Keim, and C. Collins. Pro- gressive learning of topic modeling parameters: A visual analytics frame- work. IEEE Trans. on Visualization and Computer Graphics, 24(1):382– 391, 2018. doi: 10.1109/TVCG.2017.2745080

2018
[17]

El-Assady, F

M. El-Assady, F. Sperrle, R. Sevastjanova, M. Sedlmair, and D. Keim. LTMA: Layered topic matching for the comparative exploration, evalua- tion, and reﬁnement of topic modeling results. In Int. Symp. on Big Data Visual and Immersive Analytics, pp. 1–10, Oct 2018. doi: 10.1109/...

2018
[18]

Endert, L

A. Endert, L. Bradel, and C. North. Beyond control panels: Direct manip- ulation for visual analytics. IEEE Computer Graphics and Applications, 33(4):6–13, 2013

2013
[19]

Endert, R

A. Endert, R. Chang, C. North, and M. Zhou. Semantic interaction: Coupling cognition and computation through usable interactive analytics. IEEE Computer Graphics and Applications, 35(4):94–99, 2015

2015
[20]

Endert, P

A. Endert, P. Fiaux, and C. North. Semantic interaction for visual text analytics. In Proc. SIGCHI Conf. on Human factors in Computing Systems, pp. 473–482. ACM, 2012

2012
[21]

Endert, W

A. Endert, W. Ribarsky, C. Turkay, B. W. Wong, I. Nabney, I. D. Blanco, and F. Rossi. The state of the art in integrating machine learning into visual analytics. Computer Graphics Forum, 36(8):458–486, 2017. doi: 10.1111/cgf.13092

2017 doi
[22]

Federico, M

P. Federico, M. Wagner, A. Rind, A. Amor-Amor ´os, S. Miksch, and W. Aigner. The role of explicit knowledge: A conceptual model of knowledge-assisted visual analytics. In Proc. IEEE Conf. on Visual Ana- lytics Science and Technology (VAST), pp. 92–103. IEEE, 2017

2017
[23]

Felix, S

C. Felix, S. Franconeri, and E. Bertini. Taking word clouds apart: An empirical investigation of the design space for keyword summaries. IEEE Trans. on Visualization and Computer Graphics, 24(1):657–666, 2018. doi: 10.1109/TVCG.2017.2746018

2018
[24]

R. A. Finkel and J. L. Bentley. Quad trees a data structure for retrieval on composite keys. Acta Informatica, 4(1):1–9, Mar 1974. doi: 10.1007/ BF00288933

1974
[25]

S. Fortune. A sweepline algorithm for voronoi diagrams. Algorithmica, 2:153–174, 1987

1987
[26]

Gleicher

M. Gleicher. Considerations for Visualizing Comparison. IEEE Trans. on Visualization and Computer Graphics, 24(1):413–423, 1 2018. doi: 10. 1109/TVCG.2017.2744199

2018
[27]

Halkidi, Y

M. Halkidi, Y . Batistakis, and M. Vazirgiannis. Clustering validity check- ing methods: part ii. ACM Sigmod Record, 31(3):19–27, 2002

2002
[28]

G. E. Hinton and S. T. Roweis. Stochastic neighbor embedding. In Advances in Neural Information Processing Systems, pp. 857–864, 2003

2003
[29]

Hoque and G

E. Hoque and G. Carenini. ConVisIT: Interactive topic modeling for exploring asynchronous online conversations. In Proc. Int. Conf. on Intelligent User Interfaces, pp. 169–180. ACM, 2015

2015
[30]

Y . Hu, J. Boyd-Graber, B. Satinoff, and A. Smith. Interactive topic modeling. Machine learning, 95(3):423–469, 2014

2014
[31]

Isenberg, P

T. Isenberg, P. Isenberg, J. Chen, M. Sedlmair, and T. M¨oller. A system- atic review on the practice of evaluating visualization. IEEE Trans. on Visualization and Computer Graphics, 19(12):2818–2827, Dec. 2013. doi: 10.1109/TVCG.2013.126

2013 doi
[32]

Iv´ancsy, A

R. Iv´ancsy, A. Babos, and C. Leg´any. Analysis and extensions of popular clustering algorithms. In Int. Symposium of Hungarian Researchers on Computational Intelligence, 2005

2005
[33]

Jiang, S

L. Jiang, S. Liu, and C. Chen. Recent research advances on interactive machine learning. J. of Visualization, pp. 1–17, 2018

2018
[34]

L. T. Kaastra and B. Fisher. Field experiment methodology for pair ana- lytics. In Proc. Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization (BELIV), pp. 152–159. ACM Press, 2014

2014
[35]

Levy and Y

O. Levy and Y . Goldberg. Dependency-based word embeddings. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), vol. 2, pp. 302–308, 2014

2014
[36]

C. Li, Y . Duan, H. Wang, Z. Zhang, A. Sun, and Z. Ma. Enhancing topic modeling for short texts with auxiliary word embeddings. ACM Trans. Inf. Syst., 36(2):11, 2017. doi: 10.1145/3091108

2017 doi
[37]

S. Liu, X. Wang, M. Liu, and J. Zhu. Towards better analysis of ma- chine learning models: A visual analytics perspective. Visual Informatics, 1(1):48–56, 2017

2017
[38]

L. v. d. Maaten and G. Hinton. Visualizing data using t-SNE.J. of Machine Learning Research, 9(Nov):2579–2605, 2008

2008
[39]

C. D. Manning and H. Sch¨utze. Foundations of Statistical Natural Lan- guage Processing. MIT Press, Cambridge, MA, USA, 1999

1999
[40]

Mehta, A

H. Mehta, A. Chalbi, F. Chevalier, , and C. Collins. Datatours: A data narratives framework. In Proc. of IEEE Conf. on Information Visualization (InfoVis), Posters, 2017

2017
[41]

Mikolov, I

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Proc. Advances in Neural Information Processing Systems , pp. 3111–3119, 2013

2013
[42]

C. E. Moody. Mixing Dirichlet topic models and word embeddings to make lda2vec. CoRR, abs/1605.02019, 2016

2016 arXiv
[43]

D. Q. Nguyen, R. Billingsley, L. Du, and M. Johnson. Improving topic models with latent feature word representations. Trans. of the Association for Computational Linguistics, 3:299–313, 2015

2015
[44]

D. Park, S. Kim, J. Lee, J. Choo, N. Diakopoulos, and N. Elmqvist. ConceptVector: Text visual analytics via interactive lexicon building using word embedding. IEEE Trans. on Visualization and Computer Graphics, 24(1):361–370, 2017

2017
[45]

Rayson and R

P. Rayson and R. Garside. Comparing corpora using frequency proﬁling. In Proc. Workshop on Comparing Corpora-Volume 9, pp. 1–6. Association for Computational Linguistics, 2000

2000
[46]

Y . Ren, R. Wang, and D. Ji. A topic-enhanced word embedding for Twitter sentiment classiﬁcation. Inf. Sci., 369:188–198, 2016. doi: 10.1016/j.ins. 2016.06.040

2016 doi
[47]

Sadeghi, C

A. Sadeghi, C. Lange, M.-E. Vidal, and S. Auer. Integration of scholarly communication metadata using knowledge graphs. In Proc. Int. Con. on Theory and Practice of Digital Libraries, pp. 328–341. Springer, 2017

2017
[48]

A. I. Schein, A. Popescul, L. H. Ungar, and D. M. Pennock. Methods and metrics for cold-start recommendations. In Proc. of Int. Conf. on Research and Development in Information Retrieval, pp. 253–260. ACM, 2002

2002
[49]

Sevastjanova, F

R. Sevastjanova, F. Beck, B. Ell, C. Turkay, R. Henkin, M. Butt, D. A. Keim, and M. El-Assady. Going beyond visualization: Verbalization as complementary medium to explain machine learning models. In Proc. Workshop on Visualization for AI Explainability, 2018

2018
[50]

P. Y . Simard, S. Amershi, D. M. Chickering, A. E. Pelton, S. Ghorashi, C. Meek, G. Ramos, J. Suh, J. Verwey, M. Wang, et al. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742, 2017

2017 arXiv
[51]

Sp¨arck Jones

K. Sp¨arck Jones. A statistical interpretation of term speciﬁcity and its application in retrieval. J. of Documentation, 28(1):11–21, 1972

1972
[52]

Speer, J

R. Speer, J. Chin, and C. Havasi. ConceptNet 5.5: An open multilingual graph of general knowledge. In Proc. AAAI Conf. on Artiﬁcial Intelligence, 2017

2017
[53]

Spinner, U

T. Spinner, U. Schlegel, H. Sch¨afer, and M. El-Assady. explAIner: A visual analytics framework for interactive and explainable machine learning. IEEE Trans. on Visualization and Computer Graphics, 2019

2019
[54]

Streeb, R

D. Streeb, R. Kehlbeck, D. J¨ackle, and M. El-Assady. Distances, neighbor- hoods, or dimensions? Projection literacy for the analysis of multivariate data. In Proc. Workshop on Visualization for AI Explainability, 2018

2018
[55]

X. Wang, S. Liu, J. Liu, J. Chen, J. Zhu, and B. Guo. TopicPanorama: A full picture of relevant topics. IEEE Trans. on Visualization and Computer Graphics, 2016. doi: 10.1109/TVCG.2016.2515592

2016
[56]

Z. Wang, J. Zhang, J. Feng, and Z. Chen. Knowledge graph and text jointly embedding. In Proc. of Conf. on Empirical Methods in Natural Language Processing, pp. 1591–1601, 2014

2014
[57]

Wyszecki and W

G. Wyszecki and W. Stiles. Color Science: Concepts and Methods, Quan- titative Data and Formulas. Wiley, 1968

1968
[58]

H. Zhao, L. Du, and W. Buntine. A word embeddings informed focused topic model. In Proc. of The 9th Asian Conf. on Machine Learning (ACML), pp. 423–438, 2017

2017
[59]

Y . Zhao, G. Karypis, and U. Fayyad. Hierarchical clustering algorithms for document datasets. Data Mining and Knowledge Discovery, 10(2):141– 168, 2005

2005

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [16]

El-Assady, F

M. El-Assady, F. Sperrle, O. Deussen, D. Keim, and C. Collins. Visual Analytics for Topic Model Optimization based on User-Steerable Specu- lative Execution. IEEE Trans. on Visualization and Computer Graphics, 25(1):374–384, 2019. doi: 10.1109/TVCG.2018.2864769

arXiv 2019

[2] [1]

Alexander, C.-C

E. Alexander, C.-C. Chang, M. Shimabukuro, S. Franconeri, C. Collins, and M. Gleicher. Perceptual biases in font size as a data encoding. IEEE Trans. on Visualization and Computer Graphics, 24(8):2397–2410, 2017. doi: 10.1109/TVCG.2017.2723397

work page arXiv 2017

[3] [2]

Alexander, J

E. Alexander, J. Kohlmann, R. Valenza, M. Witmore, and M. Gleicher. Serendip: Topic model-driven visual exploration of text corpora. In Proc. IEEE Symp. on Visual Analytics Science and Technology (VAST), pp. 173– 182, 2014

work page 2014

[4] [3]

Andrzejewski, X

D. Andrzejewski, X. Zhu, and M. Craven. Incorporating domain knowl- edge into topic modeling via dirichlet forest priors. In Proc. Int. Conf. on Machine Learning, pp. 25–32, 2009

work page 2009

[5] [4]

D. M. Blei. Probabilistic topic models. Communications of the ACM , 55(4):77–84, 2012. doi: 10.1145/2133806.2133826

arXiv 2012

[6] [5]

D. M. Blei, A. Y . Ng, and M. I. Jordan. Latent Dirichlet Allocation.J. of Machine Learning Research, 3:993–1022, 2003

work page 2003

[7] [6]

Cavallo and C ¸

M. Cavallo and C ¸. Demiralp. A visual interaction framework for dimen- sionality reduction based data exploration. Proc. SIGCHI Conf. on Human Factors in Computing Systems, 2018

work page 2018

[8] [7]

Z. Chen, A. Mukherjee, B. Liu, M. Hsu, and M. Castellanos. Leveraging multi-domain prior knowledge in topic models. Proc. Int. Joint Conf. on Artiﬁcial Intelligence, pp. 2071–2077, 2013

work page 2013