Pith. sign in

REVIEW 3 major objections 5 minor 67 references

GraphQAG: A Knowledge-Graph-Guided Visual Analytics Framework for Question-Answer Pairs Generation

T0 review · 3 major / 5 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A document knowledge graph plus interactive views lets people generate, check, and fix QA pairs that cover long-document knowledge better than prompt-only LLM workflows.

desk verdict Solid TVCG systems paper on KG-linked VA for set-level QA curation; the inspection/refinement story holds, the “more comprehensive and trustworthy” abstract claim does not fully match the stats. read the letter →

arxiv 2607.27182 v1 pith:FKC5ES3G submitted 2026-07-29 cs.HC

Yize Li , Ruiqi Yu , Tianya Pan , Ningxin Li , Songyue Li , Xiangyang Wu , Jinchang Li , Zhiguang Zhou This is my paper
classification cs.HC
keywords question-answerpairgenerationknowledgegraphvisualanalyticsdocumentunderstandingLLM-guidedevidencegroundinghuman-AIcollaboration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long documents scatter important facts across paragraphs and link them through multi-entity relations, so ordinary QA-pair generators often miss core content, cross-paragraph links, and multi-hop reasoning. GraphQAG turns the document into a knowledge graph, builds a graph-based generation space of key entities, relations, and multi-hop paths, and steers an LLM to write QA pairs inside that space. The same graph becomes a visual workspace: users see coverage gaps, trace each pair to source passages and subgraph structure, and add, repair, or delete pairs instead of rewriting prompts blindly. A 16-person study, two cases, and expert interviews report better perceived effectiveness and usability than a same-model prompt baseline, with higher expert ratings on reasoning complexity and evidence grounding. The practical claim is that graph scaffolding plus visual analytics makes long-document QA sets more complete and trustworthy for knowledge bases, QA systems, and model training data.

What carries the argument

Graph-based generation space: key entities ranked by paragraph-prior PageRank, local subgraphs retrieved by Personalized PageRank (complexity capped at five hops), reference subgraphs that constrain LLM QA generation, and reverse verification that rebuilds a subgraph from each QA pair and compares it to the reference for validation and repair.

What would settle it

Run the same long-document QA construction task with GraphQAG versus a strong baseline, then measure whether the resulting QA sets improve a fixed downstream use (for example domain QA accuracy or post-training gain) and whether expert coverage and grounding scores still favor GraphQAG when the automatic graph is noisy or incomplete.

Watch

Extended reading notes

Core claim

Combining a document knowledge graph, graph-constrained LLM generation, and coordinated visual analytics supports users in producing QA pair sets from long documents that better cover distributed knowledge, expose multi-hop reasoning, and stay grounded in identifiable source evidence—outperforming prompt-only generation on inspection, refinement, and several expert quality dimensions.

Load-bearing premise

The automatically built document graph is faithful enough that generating and judging QA pairs against it stands in for true document knowledge quality, even without proving the pairs help downstream tasks.

Editorial extensions

If this is right

  • Long-document QAG should be treated as set construction with coverage, complexity, grounding, and redundancy trade-offs, not one-shot question writing.
  • Making generation targets visible as entities, paths, and evidence lets users control scope before and after the LLM writes text.
  • Reverse subgraph rebuild-and-compare gives a concrete hook for spotting unsupported or mis-ordered answers and guiding targeted repair.
  • Human–LLM division of labor shifts user effort from prompt trial-and-error to choosing knowledge value and accepting or rejecting repairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If graph extraction quality is the bottleneck, gains may shrink on messy enterprise docs with weak entity linking even when the interface stays usable.
  • The same scaffold could extend to multimodal documents once tables and figures become first-class graph nodes and evidence cards.
  • Collection-level redundancy cues may need tunable complementarity policies, since similar pairs can be useful training variants rather than pure duplicates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. GraphQAG is a knowledge-graph-guided visual analytics system for generating, inspecting, and refining QA pair sets from long documents. It builds a document KG (GraphRAG extraction, PageRank key entities, PPR local subgraphs), constrains LLM generation with reference subgraphs and evidence, and reverse-verifies pairs by rebuilding subgraphs for comparison. Coordinated views (Knowledge Map, Evidence Bridge, three-level QA Space) support coverage diagnosis, evidence tracing, and targeted add/repair/delete. Evaluation comprises a within-subjects study (n=16, Latin square, same LLM vs. prompt-only baseline), two case studies, and expert interviews. Users rate effectiveness/usability higher; experts rate reasoning complexity and evidence grounding higher, with mixed results on coverage and non-redundancy.

Significance. If the claims are appropriately scoped, this is a solid TVCG-style systems contribution: it fills a real gap between automatic KG-guided QAG and interactive set-level construction/refinement for long documents. Strengths include a requirements-driven design from literature plus four experts, a coherent three-stage pipeline with inspectable generation targets, reverse subgraph verification as an operational quality signal, and a carefully counterbalanced user study with Wilcoxon tests and multi-expert ICC. The work is useful to HCI/VIS audiences building human–LLM authoring tools and to practitioners who need controllable document-grounded QA data. It does not yet establish downstream utility (KB construction, domain QA, LLM post-training), which the authors correctly flag as future work.

major comments (3)
  1. [Abstract; §VI-A; §VIII] Abstract, RQ2, §VI-A quantitative results, and §VIII: the claim that GraphQAG produces “more comprehensive and trustworthy” QA pairs overreaches the reported statistics. After Holm correction, knowledge coverage is not significant (Mdn 4.50 vs 3.50, p_adj=.097), while non-redundancy significantly favors the Baseline (Mdn 5.75 vs 7.00, p_adj=.003). Significant gains are limited to reasoning complexity and evidence grounding (plus subjective Effectiveness/Usability). Please align abstract, contribution bullets, and conclusion with the actual pattern of results, and discuss the coverage/non-redundancy tradeoff explicitly rather than summarizing as uniformly higher quality.
  2. [§IV-D; Eq. (3); §VII-B] §IV-D and Fig. 3: reverse verification defines validity largely as consistency between the rebuilt subgraph and the reference subgraph (fact omission/addition/substitution, etc.). That checks fidelity to the generation scaffold, not independent document faithfulness or answer correctness beyond the extracted KG. Eq. (3)’s C_QA(v) is likewise coverage over generation-eligible KG facts, not over document knowledge outside the graph. Given that the central “trustworthy/comprehensive” framing load-bears on this proxy (§VII-B acknowledges the gap), the paper should (i) state this scope limitation up front in Method/Evaluation, and (ii) either add a modest external check (e.g., human span-level grounding against source text independent of the KG, or error analysis of GraphRAG extraction failures) or systematically soften claims that imply document-complete quality.
  3. [§VI-A; §VII-B] §VI-A Baseline and expert assessment: the prompt-only LLM baseline is appropriate for usability of the visual workflow, but it is a weak foil for attributing QA-set quality gains to the full GraphQAG method versus any structured overview. Combined with only two documents (narrative fiction + one GovReport), moderate ICC on evidence grounding (.59) and non-redundancy (.64), and no downstream task, general claims about set quality remain fragile. At minimum, add a limitations-bound interpretation of external validity; preferably report document-level breakdowns, extraction-error notes, and clearer expert rubrics/calibration for the two weaker dimensions (as §VII-B already suggests).
minor comments (5)
  1. [Fig. 1; Fig. 4–5] Fig. 1 and Fig. 4 captions are dense; a short “how to read” callout for C_QA arcs and QA glyph outline colors would help first-time readers.
  2. [§IV-A–C; §V-A] Algorithm 1 and Eqs. (1)–(2): briefly state default values or ranges used in the user study for d, convergence threshold, τ, and similarity thresholds so the free parameters are reproducible.
  3. [§II-C] Related work §II-C is thorough; a compact comparison table (system × generation-space control × set-level refinement × evidence bridge) would sharpen the claimed gap versus LinkQ, VisPile, XGraphRAG, etc.
  4. [Throughout; References] Typographical/consistency: “DeepSeek-V4” / “DeepSeek-V4-Flash” citations and arXiv-style placeholders should be checked against final bibliographic entries; ensure “GraphQAG” spacing is consistent in running text.
  5. [§VI-B] §VI-B case studies are illustrative; stating approximate interaction times or number of repair cycles would better support transferability claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation circularity: empirical HCI systems paper with external baselines, not self-fitting predictions.

full rationale

GraphQAG is a design-and-evaluation systems paper, not a first-principles derivation. Quality goals come from a literature review plus formative expert interviews (Table I; §III); the system is then built to support those goals (document KG via GraphRAG, PageRank/PPR subgraphs, LLM QAG, reverse subgraph check, coordinated views) and tested against an LLM-prompt baseline in a counterbalanced user study (n=16), case studies, and expert ratings. Reverse verification (§IV-D; Fig. 3) defines a QA pair as valid when a rebuilt subgraph is consistent with the reference subgraph used for generation—this is an explicit internal consistency check and UI repair signal, not a claimed external prediction that reduces to fitted inputs by construction. Coverage C_QA(v) (Eq. 3) is likewise a defined visualization of which generation-eligible KG facts are already used, not a forecast of document truth. Citations for GraphRAG, PageRank, PPR, ICC, and the LLM backend are external standard tools. Evaluation limitations (KG as proxy, two documents, non-significant coverage after Holm, moderate evidence-grounding ICC, no downstream task validation) are external-validity/correctness concerns, not circular derivation. No self-definitional X≡Y claim, no fitted-parameter-as-prediction, and no load-bearing self-citation uniqueness chain.

Assumptions & free parameters 5 free parameters · 6 assumptions · 4 invented entities

Claims rest on standard IR/graph and LLM tooling choices plus design assumptions that extracted graph structure is the right intermediate language for long-document QAG quality. Several knobs (k, hops, similarity thresholds) are user- or author-set; evaluation treats expert Likert aggregates as ground truth for set quality without external task benchmarks.

free parameters (5)
  • Number of key entities k (PageRank top-k) = User-set (e.g., case used 9)
    User-chosen seed count that defines the generation space; study required consistency across conditions but value is not theoretically fixed.
  • Complexity / max relation-chain length (1–5 hops) = Integer 1–5 (case used 2)
    Caps multi-hop subgraph extent; justified by cited accuracy drop with depth, not derived from document structure.
  • Reference-subgraph Jaccard dedup threshold τ = Predefined threshold (value not reported)
    Controls which candidate subgraphs are kept before QAG; predefined but not validated against external quality.
  • Redundancy semantic/structural similarity thresholds = User-adjustable (defaults not specified)
    User-adjustable cutoffs that define pale-blue redundant groups in QA Space and affect non-redundancy outcomes.
  • PageRank damping factor d and convergence threshold = Not numerically specified
    Standard ranking hyperparameters that change which entities are treated as “key.”
assumptions (6)
  • domain assumption GraphRAG-style entity/relation extraction plus cross-paragraph entity merge yields a document KG adequate for coverage and evidence linking.
    Invoked in §IV-A as the foundation of R1 and all later generation/verification.
  • ad hoc to paper PageRank with uniform paragraph priors ranks entities by importance for QAG seeds.
    §IV-A Eqs. (1)–(2); alternative importance notions are not compared.
  • domain assumption Personalized PageRank neighborhoods plus hop limits define appropriate local generation subgraphs.
    §IV-B; hop cap at 5 from external multi-hop QA accuracy observations.
  • domain assumption An LLM can map a reference subgraph + source sentences into a natural-language QA pair whose meaning is captured by reverse entity/fact linking.
    §§IV-C–D generation and reverse verification loop.
  • domain assumption Four expert-prioritized dimensions (coverage, reasoning complexity, evidence grounding, non-redundancy) are the right primary quality objectives for long-document QA sets.
    Formative study §III; Diversity and QA alignment deprioritized by the same experts who later rate outputs.
  • ad hoc to paper A prompt-only LLM baseline without graph structure is a fair foil for claiming GraphQAG usefulness.
    User study §VI-A; no strong structured non-visual baseline (e.g., automatic KG-QAG alone).
invented entities (4)
  • Graph-based generation space (entities, relations, multi-hop paths as controllable QAG scope)
    purpose: Translate high-level QAG goals into inspectable graph structures that constrain the LLM.
    Core framing in Abstract/Intro and R2; operationalized via key entities and reference subgraphs.
  • Reference subgraph vs rebuilt subgraph reverse-verification pair
    purpose: Diagnose omission/addition/substitution and drive LLM repair.
    §IV-D and QA Card Compare view; validity defined relative to the same document KG.
  • Local QA-reference coverage score C_QA(v)
    purpose: Visualize per-entity fraction of generation-eligible facts covered by current QA set.
    Eq. (3) in Knowledge Map design; drives gap-finding interactions.
  • Evidence Bridge + three-level QA Space visual workflow
    purpose: Coordinate set-level and instance-level inspection and refinement.
    §V system design; primary HCI contribution surface.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GraphQAG: A Knowledge-Graph-Guided Visual Analytics Framework for Question-Answer Pairs Generation." pith.science (2026). https://pith.science/paper/FKC5ES3G

@misc{pith2026260727182,
  author       = {Pith},
  title        = {Pith review of: GraphQAG: A Knowledge-Graph-Guided Visual Analytics Framework for Question-Answer Pairs Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FKC5ES3G}},
  note         = {Machine review of arXiv:2607.27182}
}
read the original abstract

Question-answer (QA) pairs are widely used in knowledge base construction, question-answering systems, and the post-training of large language models (LLMs). However, important knowledge in long documents is often distributed across multiple paragraphs and connected through complex entity relationships. Such fragmented and relational knowledge poses substantial challenges for existing QA generation methods, which often fail to adequately cover core document content, cross-paragraph semantic connections, and multi-entity relationships. We present GraphQAG, a knowledge graph-guided visual analytics framework for generating high-quality QA pairs from long documents. GraphQAG follows a three-stage workflow. First, it constructs a document knowledge graph by segmenting the document into paragraphs and extracting salient entities and relations. Second, it builds a graph-based generation space from entities, relations, and multi-hop paths to constrain and guide LLM-based QA generation. Third, it uses the knowledge graph as an interactive visual representation, enabling users to explore document knowledge structures, inspect the coverage and evidence provenance of generated QA pairs, and iteratively refine the QA pair set through graph-based interactions. We evaluated GraphQAG through a user study with 16 participants, two case studies, and expert interviews. The results indicate that GraphQAG effectively supports users in identifying knowledge coverage gaps, examining generated QA pairs, and refining the QA pair set. These findings demonstrate the usefulness of combining knowledge graphs, LLM-based generation, and visual analytics for producing more comprehensive and trustworthy QA pairs from long documents.

Figures

Figures reproduced from arXiv: 2607.27182 by the authors.

Figure 1
Figure 1. The interface of GraphQAG. A The Control Panel allows users to upload documents and adjust the system parameters for QAG. B The Document View presents the source text, where dark-blue entities indicate knowledge covered by the current QA pair set, while light-blue underlined entities indicate uncovered knowledge. C The Knowledge Map visualizes the document KG and its current QA coverage. D The Evidence Bridge links … view at source ↗
Figure 2
Figure 2. The pipeline of GraphQAG. Given a source document and user-specified generation parameters, the system constructs a document KG, identifies and ranks key entities, expands their local graph structures to retrieve generation subgraphs, and uses them to guide LLM-based QAG. It then assesses the generated QA pairs along four quality dimensions and supports iterative refinement through visual interaction. B. Local Subgr… view at source ↗
Figure 3
Figure 3. Comparison and repair of QA subgraphs. The [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: A The Knowledge Map organizes entities by source paragraph and highlights a selected reference subgraph, and users can explore the reasoning structure on it a . B The Evidence Bridge presents supporting source passages in linked b Evidence Cards . C The QA Space organi…
Figure 5
Figure 5. Figure 5: The QA Space supports progressive inspection of a QA pair set at three semantic zoom levels. a The Distribution Level groups QA glyphs by key entity. Sector angle encodes QA count. Radial distance encodes the number of relations in the reference subgraph. Outline color…
Figure 6
Figure 6. Figure 6: User study results. The left panel summarizes participants’ responses on Effectiveness ( [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Case 1: Charting knowledge gaps for targeted QA refinement. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Case 2: Constructing and inspecting cross-paragraph QA pairs. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 2 linked inside Pith

  1. [1]

    A comprehensive survey on automatic knowledge graph construction,

    L. Zhong, J. Wu, Q. Li, H. Peng, and X. Wu, “A comprehensive survey on automatic knowledge graph construction,”ACM Computing Surveys, vol. 56, no. 4, pp. 1–62, 2023

    2023

  2. [2]

    Dense passage retrieval for open-domain question answering,

    V . Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question answering,” inProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 2020, pp. 6769–6781

    2020

  3. [3]

    Squad: 100,000+ questions for machine comprehension of text,

    P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for machine comprehension of text,” inProceedings of the 2016 conference on empirical methods in natural language processing, 2016, pp. 2383–2392

    2016

  4. [4]

    Phi- 4-reasoning technical report,

    M. Abdin, S. Agarwal, A. Awadallah, V . Balachandran, H. Behl, L. Chen, G. de Rosa, S. Gunasekar, M. Javaheripi, N. Joshiet al., “Phi- 4-reasoning technical report,” 2025

    2025

  5. [5]

    Deepseek-v4: Towards highly efficient million- token context intelligence,

    A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Linget al., “Deepseek-v4: Towards highly efficient million- token context intelligence,” 2026

    2026

  6. [6]

    A survey on neural question generation: Methods, applications, and prospects,

    S. Guo, L. Liao, C. Li, and T. S. Chua, “A survey on neural question generation: Methods, applications, and prospects,” inProceedings of the 33rd International Joint Conference on Artificial Intelligence, IJCAI

  7. [7]

    Question generation for question answering,

    N. Duan, D. Tang, P. Chen, and M. Zhou, “Question generation for question answering,” inProceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 866–874

    2017

  8. [8]

    Learning to ask: Neural question gen- eration for reading comprehension,

    X. Du, J. Shao, and C. Cardie, “Learning to ask: Neural question gen- eration for reading comprehension,” inProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1342–1352

    2017

Show all 67 references
  1. [9]

    Lost in the middle: How language models use long contexts,

    N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Transactions of the association for computational linguistics, vol. 12, pp. 157–173, 2024

    2024

  2. [10]

    A survey on knowledge graphs: Representation, acquisition, and applications,

    S. Ji, S. Pan, E. Cambria, P. Marttinen, and P. S. Yu, “A survey on knowledge graphs: Representation, acquisition, and applications,”IEEE transactions on neural networks and learning systems, vol. 33, no. 2, pp. 494–514, 2021

    2021

  3. [11]

    Graph chain-of-thought: Augmenting large language models by reasoning on graphs,

    B. Jin, C. Xie, J. Zhang, K. K. Roy, Y . Zhang, Z. Li, R. Li, X. Tang, S. Wang, Y . Menget al., “Graph chain-of-thought: Augmenting large language models by reasoning on graphs,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 163–184

    2024

  4. [12]

    Toward subgraph-guided knowledge graph question generation with graph neural networks,

    Y . Chen, L. Wu, and M. J. Zaki, “Toward subgraph-guided knowledge graph question generation with graph neural networks,”IEEE Transac- tions on Neural Networks and Learning Systems, vol. 35, no. 9, pp. 12 706–12 717, 2023

    2023

  5. [13]

    Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications,

    N. Mulla and P. Gharpure, “Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications,”Progress in Artificial Intelligence, vol. 12, no. 1, pp. 1–32, 2023

    2023

  6. [14]

    Quizbot: A dialogue-based adaptive learning system for factual knowledge,

    S. Ruan, L. Jiang, J. Xu, B. J.-K. Tham, Z. Qiu, Y . Zhu, E. L. Murnane, E. Brunskill, and J. A. Landay, “Quizbot: A dialogue-based adaptive learning system for factual knowledge,” inProceedings of the 2019 CHI conference on human factors in computing systems, 2019, pp. 1–13

    2019

  7. [15]

    Liquid: A framework for list question answering dataset generation,

    S. Lee, H. Kim, and J. Kang, “Liquid: A framework for list question answering dataset generation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 11, 2023, pp. 13 014–13 024

    2023

  8. [16]

    An empirical comparison of lm-based question and answer generation methods,

    A. Ushio, F. Alva-Manchego, and J. Camacho-Collados, “An empirical comparison of lm-based question and answer generation methods,” in Findings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 14 262–14 272

    2023

  9. [17]

    Graph guided question answer generation for procedural question-answering,

    H. Pham, I. Hadji, X. Xu, Z. Degutyte, J. Rainey, E. Kazakos, A. Fazly, G. Tzimiropoulos, and B. Martinez, “Graph guided question answer generation for procedural question-answering,” inProceedings of the 18th Conference of the European Chapter of the Association for Com- puta...

    2024

  10. [18]

    A-SEA3L-QA: A fully automated self-evolving, adversarial workflow for arabic long- context question-answer generation,

    K. Wang, D. Toibazar, and P. J. M. Mengibar, “A-SEA3L-QA: A fully automated self-evolving, adversarial workflow for arabic long- context question-answer generation,” inProceedings of The Third Arabic Natural Language Processing Conference, 2025, pp. 107–116

    2025

  11. [19]

    Automatic dataset generation for knowledge intensive question answering tasks,

    S. Yuen, T. Su, Z. Wang, Y . Du, and A. J. Sobey, “Automatic dataset generation for knowledge intensive question answering tasks,”arXiv preprint arXiv:2505.14212, 2025

    2025 arXiv

  12. [20]

    Evaluation of question generation needs more references,

    S. Oh, H. Go, H. Moon, Y . Lee, M. Jeong, H. S. Lee, and S. Choi, “Evaluation of question generation needs more references,” inFindings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 6358–6367. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 16

    2023

  13. [21]

    Rquge: Reference-free metric for evaluating question generation by answering the question,

    A. Mohammadshahi, T. Scialom, M. Yazdani, P. Yanki, A. Fan, J. Hen- derson, and M. Saeidi, “Rquge: Reference-free metric for evaluating question generation by answering the question,” inFindings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 6845– 6867

    2023

  14. [22]

    Qgeval: benchmarking multi-dimensional evaluation for question generation,

    W. Fu, B. Wei, J. Hu, Z. Cai, and J. Liu, “Qgeval: benchmarking multi-dimensional evaluation for question generation,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 11 783–11 803

    2024

  15. [23]

    Ragas: Automated evaluation of retrieval augmented generation,

    S. Es, J. James, L. E. Anke, and S. Schockaert, “Ragas: Automated evaluation of retrieval augmented generation,” inProceedings of the 18th conference of the european chapter of the association for computational linguistics: system demonstrations, 2024, pp. 150–158

    2024

  16. [24]

    Ragchecker: A fine-grained framework for diagnosing retrieval-augmented generation,

    D. Ru, L. Qiu, X. Hu, T. Zhang, P. Shi, S. Chang, C. Jiayang, C. Wang, S. Sun, H. Liet al., “Ragchecker: A fine-grained framework for diagnosing retrieval-augmented generation,”Advances in Neural Information Processing Systems, vol. 37, pp. 21 999–22 027, 2024

    2024

  17. [25]

    Minicheck: Efficient fact-checking of llms on grounding documents,

    L. Tang, P. Laban, and G. Durrett, “Minicheck: Efficient fact-checking of llms on grounding documents,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 8818– 8847

    2024

  18. [26]

    Towards diverse and effective question-answer pair generation from children storybooks,

    S. Eo, H. Moon, J. Kim, Y . Hur, J. Kim, S. Lee, C. Chun, S. Park, and H.-S. Lim, “Towards diverse and effective question-answer pair generation from children storybooks,” inFindings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 6100–6115

    2023

  19. [27]

    Explicit over implict: Explicit diversity conditions for effective question answer gen- eration,

    V . Yadav, H. joon Kwon, V . Srinivasan, and H. Jin, “Explicit over implict: Explicit diversity conditions for effective question answer gen- eration,” inProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LRE...

    2024

  20. [28]

    Kg-qagen: A knowledge- graph-based framework for systematic question generation and long- context llm evaluation,

    N. Tatarinov, V . Kannan, H. Srinivasa, A. Raj, H. Singh Anand, V . Singh, A. Luthra, R. Lade, A. Shah, and S. Chava, “Kg-qagen: A knowledge- graph-based framework for systematic question generation and long- context llm evaluation,” pp. arXiv–2505, 2025

    2025

  21. [29]

    Knight: Knowledge graph-driven multiple-choice question generation with adaptive hardness calibration,

    M. Amanlou, E. S. Moghaddam, M. Nouri, Y . A. Jafary, F. Farsi, and B. Bahrak, “Knight: Knowledge graph-driven multiple-choice question generation with adaptive hardness calibration,” inThird Conference on Parsimony and Learning. PMLR, 2026, pp. 989–1024

    2026

  22. [30]

    Linkq: An llm-assisted visual interface for knowledge graph question-answering,

    H. Li, G. Appleby, and A. Suh, “Linkq: An llm-assisted visual interface for knowledge graph question-answering,” in2024 IEEE Visualization and Visual Analytics (VIS). IEEE, 2024, pp. 116–120

    2024

  23. [31]

    Netme 2.0: a web-based platform for extracting and modeling knowledge from biomedical literature as a labeled graph,

    A. Di Maria, L. Bellomo, F. Billeci, A. Cardillo, S. Alaimo, P. Ferragina, A. Ferro, and A. Pulvirenti, “Netme 2.0: a web-based platform for extracting and modeling knowledge from biomedical literature as a labeled graph,”Bioinformatics, vol. 40, no. 5, p. btae194, 2024

    2024

  24. [32]

    Knownet: Guided health information seeking from llms via knowledge graph integration,

    Y . Yan, Y . Hou, Y . Xiao, R. Zhang, and Q. Wang, “Knownet: Guided health information seeking from llms via knowledge graph integration,” IEEE Transactions on Visualization and Computer Graphics, vol. 31, no. 1, pp. 547–557, 2024

    2024

  25. [33]

    Healthgenie: A knowledge-driven llm framework for tailored dietary guidance,

    F. Gao, X. Zhao, D. Xia, Z. Zhou, R. Yang, J. Lu, H. Jiang, C. Park, and I. Li, “Healthgenie: A knowledge-driven llm framework for tailored dietary guidance,” inProceedings of the 34th ACM International Con- ference on Information and Knowledge Management, 2025, pp. 6639– 6643

    2025

  26. [34]

    Kragen: a knowledge graph-enhanced rag framework for biomedical problem solving using large language models,

    N. Matsumoto, J. Moran, H. Choi, M. E. Hernandez, M. Venkatesan, P. Wang, and J. H. Moore, “Kragen: a knowledge graph-enhanced rag framework for biomedical problem solving using large language models,”Bioinformatics, vol. 40, no. 6, p. btae353, 2024

    2024

  27. [35]

    Vispile: A visual analytics system for analyzing multiple text documents with large language models and knowledge graphs,

    A. Coscia and A. Endert, “Vispile: A visual analytics system for analyzing multiple text documents with large language models and knowledge graphs,” inProceedings of the 59th Hawaii International Conference on System Sciences, ser. HICSS-59, 2026

    2026

  28. [36]

    Ragviz: Diagnose and visualize retrieval- augmented generation,

    T. Wang, J. He, and C. Xiong, “Ragviz: Diagnose and visualize retrieval- augmented generation,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demon- strations, 2024, pp. 320–327

    2024

  29. [37]

    Ragtrace: Understanding and re- fining retrieval-generation dynamics in retrieval-augmented generation,

    S. Cheng, J. Li, H. Wang, and Y . Ma, “Ragtrace: Understanding and re- fining retrieval-generation dynamics in retrieval-augmented generation,” inProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, 2025, pp. 1–20

    2025

  30. [38]

    Xgraphrag: Interactive visual analysis for graph-based retrieval- augmented generation,

    K. Wang, B. Pan, Y . Feng, Y . Wu, J. Chen, M. Zhu, and W. Chen, “Xgraphrag: Interactive visual analysis for graph-based retrieval- augmented generation,” in2025 IEEE 18th Pacific Visualization Con- ference (PacificVis). IEEE, 2025, pp. 1–11

    2025

  31. [39]

    Ragexplorer: A visual analytics system for the comparative diagnosis of rag systems,

    H. Tian, Y . Feng, Z. Wen, H. Li, M. Zhu, and W. Chen, “Ragexplorer: A visual analytics system for the comparative diagnosis of rag systems,” IEEE Transactions on Visualization and Computer Graphics, vol. 32, no. 6, pp. 4807–4817, 2026

    2026

  32. [40]

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering,

    Z. Yang, P. Qi, S. Zhang, Y . Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning, “Hotpotqa: A dataset for diverse, explainable multi-hop question answering,” inProceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp. 2369–2380

    2018

  33. [41]

    Musique: Multihop questions via single-hop question composition,

    H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal, “Musique: Multihop questions via single-hop question composition,”Transactions of the Association for Computational Linguistics, vol. 10, pp. 539–554, 2022

    2022

  34. [42]

    Difficulty-controllable multi-hop question generation from knowledge graphs,

    V . Kumar, Y . Hua, G. Ramakrishnan, G. Qi, L. Gao, and Y .-F. Li, “Difficulty-controllable multi-hop question generation from knowledge graphs,” inInternational Semantic Web Conference. Springer, 2019, pp. 382–398

    2019

  35. [43]

    On the importance of diversity in question generation for qa,

    M. A. Sultan, S. Chandel, R. F. Astudillo, and V . Castelli, “On the importance of diversity in question generation for qa,” inProceedings of the 58th annual meeting of the association for computational linguistics, 2020, pp. 5651–5656

    2020

  36. [44]

    From local to global: A graph rag approach to query-focused summarization,

    D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson, “From local to global: A graph rag approach to query-focused summarization,”arXiv preprint arXiv:2404.16130, 2024

    2024 arXiv

  37. [45]

    The anatomy of a large-scale hypertextual web search engine,

    S. Brin and L. Page, “The anatomy of a large-scale hypertextual web search engine,”Computer networks and ISDN systems, vol. 30, no. 1-7, pp. 107–117, 1998

    1998

  38. [46]

    Scaling personalized web search,

    G. Jeh and J. Widom, “Scaling personalized web search,” inProceedings of the 12th international conference on World Wide Web, 2003, pp. 271– 279

    2003

  39. [47]

    GRADE: Generating multi-hop QA and fine-gRAined difficulty matrix for RAG evaluation,

    J. Lee, D. Kwon, and K. Jin, “GRADE: Generating multi-hop QA and fine-gRAined difficulty matrix for RAG evaluation,” inFind- ings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Peng, Eds. Suzhou, China: Ass...

    2025

  40. [48]

    Guidelines for using multiple views in information visualization,

    M. Q. Wang Baldonado, A. Woodruff, and A. Kuchinsky, “Guidelines for using multiple views in information visualization,” inProceedings of the working conference on Advanced visual interfaces, 2000, pp. 110– 119

    2000

  41. [49]

    Arc diagrams: Visualizing structure in strings,

    M. Wattenberg, “Arc diagrams: Visualizing structure in strings,” inIEEE Symposium on Information Visualization, 2002. INFOVIS 2002.IEEE, 2002, pp. 110–116

    2002

  42. [50]

    Network visualization by semantic substrates,

    B. Shneiderman and A. Aris, “Network visualization by semantic substrates,”IEEE transactions on visualization and computer graphics, vol. 12, no. 5, pp. 733–740, 2006

    2006

  43. [51]

    More flexible radial layout

    U. Brandes and C. Pich, “More flexible radial layout.”Journal of Graph Algorithms and Applications, vol. 15, no. 1, pp. 157–173, 2011

    2011

  44. [52]

    Arcs, angles, or areas: Individual data encodings in pie and donut charts,

    D. Skau and R. Kosara, “Arcs, angles, or areas: Individual data encodings in pie and donut charts,”Computer Graphics Forum, vol. 35, no. 3, pp. 121–130, 2016. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.12888

    2016 doi

  45. [53]

    Automating the design of graphical presentations of relational information,

    J. Mackinlay, “Automating the design of graphical presentations of relational information,”Acm Transactions On Graphics (Tog), vol. 5, no. 2, pp. 110–141, 1986

    1986

  46. [54]

    Context-preserving visual links,

    M. Steinberger, M. Waldner, M. Streit, A. Lex, and D. Schmalstieg, “Context-preserving visual links,”IEEE transactions on visualization and computer graphics, vol. 17, no. 12, pp. 2249–2258, 2011

    2011

  47. [55]

    Jigsaw: supporting investigative analysis through interactive visualization,

    J. Stasko, C. G ¨org, and Z. Liu, “Jigsaw: supporting investigative analysis through interactive visualization,”Information visualization, vol. 7, no. 2, pp. 118–132, 2008

    2008

  48. [56]

    The eyes have it: A task by data type taxonomy for information visualizations,

    B. Shneiderman, “The eyes have it: A task by data type taxonomy for information visualizations,” inThe craft of information visualization. Elsevier, 2003, pp. 364–371

    2003

  49. [57]

    Pad++: A zoomable graphical sketchpad for exploring alternate interface physics,

    B. B. Bederson, J. D. Hollan, K. Perlin, J. Meyer, D. Bacon, and G. Furnas, “Pad++: A zoomable graphical sketchpad for exploring alternate interface physics,”Journal of Visual Languages & Computing, vol. 7, no. 1, pp. 3–32, 1996

    1996

  50. [58]

    Visual comparison for information visualization,

    M. Gleicher, D. Albers, R. Walker, I. Jusufi, C. D. Hansen, and J. C. Roberts, “Visual comparison for information visualization,”Information Visualization, vol. 10, no. 4, pp. 289–309, 2011

    2011

  51. [59]

    D 3 data-driven documents,

    M. Bostock, V . Ogievetsky, and J. Heer, “D 3 data-driven documents,” IEEE transactions on visualization and computer graphics, vol. 17, no. 12, pp. 2301–2309, 2011

    2011

  52. [60]

    Making monolingual sentence embed- dings multilingual using knowledge distillation,

    N. Reimers and I. Gurevych, “Making monolingual sentence embed- dings multilingual using knowledge distillation,” inProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 2020, pp. 4512–4525. IEEE TRANSACTIONS ON VISUALIZATION AND COMPU...

    2020

  53. [61]

    Mpnet: Masked and permuted pre-training for language understanding,

    K. Song, X. Tan, T. Qin, J. Lu, and T.-Y . Liu, “Mpnet: Masked and permuted pre-training for language understanding,”Advances in neural information processing systems, vol. 33, pp. 16 857–16 867, 2020

    2020

  54. [62]

    A. C. Doyle and L. Sloane,A scandal in Bohemia. Pasa, 2007

    2007

  55. [63]

    Efficient attentions for long document summarization,

    L. Huang, S. Cao, N. Parulian, H. Ji, and L. Wang, “Efficient attentions for long document summarization,” inProceedings of the 2021 confer- ence of the north American chapter of the association for computational linguistics: Human language technologies, 2021, pp. 1419–1436

    2021

  56. [64]

    Latin square design,

    J. Salinas Ru ´ız, O. A. Montesinos L ´opez, and J. Crossa, “Latin square design,” inIntroduction to Experimental Designs with PROC GLIMMIX of SAS: Applications in Food Science and Agricultural Science. Cham: Springer Nature Switzerland, 2024, pp. 167–195

    2024

  57. [65]

    Intraclass correlation for reliability assessment: the introduction of a validated program in sas (icc6),

    V . Senthil Kumar and S. Shahraz, “Intraclass correlation for reliability assessment: the introduction of a validated program in sas (icc6),”Health Services and Outcomes Research Methodology, vol. 24, no. 1, pp. 1–13, 2024

    2024

  58. [66]

    Interval estimation of the intra- class correlation in general linear mixed effects models,

    X. Feng, T. Mathew, and K. Adragni, “Interval estimation of the intra- class correlation in general linear mixed effects models,”Journal of Statistical Theory and Practice, vol. 15, no. 3, p. 65, 2021. IX. BIOGRAPHYSECTION Yize Liis currently a Ph.D. student in the School of C...

    2021

  59. [2024]

    8038–8047

    International Joint Conferences on Artificial Intelligence, 2024, pp. 8038–8047

    2024

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.