Pith. sign in

REVIEW 2 major objections 5 minor 65 references

CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that grounding conversational AI in a per-designer heterogeneous knowledge graph, with a Heterogeneous Graph Transformer selecting which nodes to inject, outperforms both ungrounded LLM interaction and naive full-graph…

desk verdict A well-built system with an honest limitations section, but the benchmark evaluation leaks the answer into the HGT's training and selection, so the headline claim about selective grounding isn't supported by the technical eval. read the letter →

arxiv 2608.13216 v1 pith:RSB6QLSE submitted 2026-08-13 cs.HC

Jiin Choi , Kyung Hoon Hyun This is my paper · ORCID
classification cs.HC
keywords knowledgegraphheterogeneoustransformerconversationalAIdesigncognitioncontextgroundingprobingquestionspersonalizationlong-termdialoguememory
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CogChat builds a personal heterogeneous knowledge graph from each designer's utterances, extracts typed entities and relations, and uses a Heterogeneous Graph Transformer (HGT) to choose which graph nodes are injected into the LLM's prompt at each turn. The paper argues that preserving relational context requires structure, not capacity: exhaustive graph injection (KG-only) overdraws on noise, while HGT-selected top-k grounding (KG+HGT) beats both unfiltered graph memory and the ungrounded baseline. Technical evaluation on ASQA, RewardBench, LoCoMo, and LongMemEval reports KG+HGT highest on all four, with the largest gaps on ambiguous and relationally complex subsets. A within-subjects study with nine professional designers reports fewer turns, more consistent vocabulary reuse, higher probing success, lower reported cognitive load, and deeper engagement for the HGT-grounded condition. If correct, the central consequence is that the bottleneck in multi-turn design conversation is context curation, deciding which relations to foreground, rather than context-window capacity.

What carries the argument

The load-bearing object is the Heterogeneous Graph Transformer (HGT), a graph neural layer with type-specific query, key, and value projections and relation-specific attention, used to embed a heterogeneous knowledge graph whose nodes carry types such as CONCEPT, DESIGN_ELEMENT, MATERIAL, PROPERTY, and ACTION, initialized from CLIP embeddings. The graph is built per turn via a GraphRAG-style extraction pipeline; positive and negative link sets are sampled from co-occurrence and designer feedback, and the HGT is retrained incrementally on them. At response time, nodes are ranked by cosine similarity between the current message embedding and HGT node embeddings, and the top roughly twenty entities, with their relations, enter the LLM prompt. That selection step is what the paper contrasts against KG-only, which injects all extracted entities, to isolate selective grounding as the cause of the observed gains.

What would settle it

Re-run the four benchmarks with graphs built only from the question and prior conversation turns, never from the reference or preferred answer. If the KG+HGT advantage over both Baseline and KG-only collapses or drops substantially, the reported gains came from exposing the preferred response's entities rather than from selective relational grounding. For the user study, a blind expert rating of the final design artifacts across conditions would settle whether the self-reported depth translates into output quality.

Watch

Extended reading notes

Core claim

The central claim is that grounding a conversational LLM in a selectively injected, relationally structured personal knowledge graph improves multi-turn design conversation over both no grounding and unfiltered graph grounding. The paper's own framing is that recency-based context decays cross-turn relations no matter how large the window, and that injecting the full graph adds noise that competes with salient nodes; HGT-based ranking fixes that by foregrounding structurally relevant entities. Because the KG+HGT condition outperforms KG-only on every tested benchmark, and because the KG-only versus KG+HGT gap is significant in the user study on semantic similarity, token overlap, probing success, and task time, the paper concludes that selective graph-grounded context injection, not memory capacity, is the operative mechanism. The paper further claims that the same ordering generalizes from ambiguity and preference benchmarks to long-term memory benchmarks, motivating its use in design practice.

Load-bearing premise

The technical results stand or fall on whether per-query knowledge graph construction from questions plus reference passages produces the same signal a live designer's graph would, with no answer leakage; the user-experience results additionally rest on a nine-person, single-session study that the authors themselves caution is not evidence of long-term personalization.

Editorial extensions

If this is right

  • Context-window growth alone will not fix multi-turn failure: the Baseline received full conversation history, yet HGT-selected context still won.
  • Naive knowledge-graph augmentation can hurt: KG-only injected noise that compounded errors on ambiguous queries, so adding graph memory without selection is not an improvement.
  • Selective grounding gives its largest gains on relationally dense or ambiguous tasks, with RewardBench improvements of 23.2 points on Chat Hard and 22.4 points on Reasoning, so the method should be expected to help wherever utterances are polysemous and relational.
  • Long-horizon dialogue benefits: the ranking held on LongMemEval and LoCoMo, indicating the mechanism transfers beyond design-style ambiguity to multi-session recall.
  • In user interaction, the graph-grounded system cut turns to completion by 45.9%, nearly doubled token-level vocabulary reuse relative to KG-only, raised probing question answer rates from 0.17 to 0.70, and lowered reported mental demand, effort, and frustration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that any selective retriever, not specifically HGT, might reproduce part of the effect; a fair test would replace HGT with a simpler context selector, such as plain CLIP similarity without graph training, and compare the three-condition ordering.
  • Because the technical benchmarks construct per-query knowledge graphs from questions plus reference or preferred-answer passages, part of the reported edge may reflect entity overlap with the gold response; my reading is that the benchmark numbers are strong evidence for the value of relevant context access, but weaker evidence for relational grounding per se.
  • A testable extension is to run the same three-condition comparison on a multi-session design project spanning days or weeks, since the paper's own limitations note that graph decay, contradictory constraints, and intent pivots remain open challenges that short single-session tasks cannot expose.
  • The probing feedback loop suggests a possible positive spiral: better questions produce denser graphs, and denser graphs produce better questions; if that holds, gains should compound across turns, a prediction the current 30-minute sessions only hint at.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. CogChat is a chat framework that constructs a personal heterogeneous knowledge graph from a designer's utterances, embeds it with a Heterogeneous Graph Transformer, and uses HGT-based ranking to select a small set of entities for prompt injection and to generate intentional and exploratory probing questions. The paper compares three conditions—Baseline (LLM-only), KG-only (naive full-graph injection), and KG+HGT (full pipeline)—on four technical benchmarks (ASQA, RewardBench, LoCoMo, LongMemEval) and in a within-subjects study with nine professional designers. The central claim is that selective relational grounding, rather than context capacity or naive graph augmentation, is what improves multi-turn design conversation and generalizes to ambiguous and long-horizon QA.

Significance. If the claims hold, the work is significant for UIST: it offers a concrete architecture that moves conversational context from recency-based token memory to a designer-specific structured relational memory, with a clean three-condition ablation and a mixed-methods user study. The interface features, especially in-context knowledge lookup and probing questions, are thoughtful and well-motivated by prior design-cognition literature. The paper also states its limitations candidly. However, the technical evaluation is seriously compromised by the way benchmark knowledge graphs are constructed from reference/gold passages, and the reported pairwise p-values in the user study are statistically impossible with nine participants. Both issues are load-bearing: the first affects the claim that HGT-based selection outperforms alternatives, and the second affects the reported significance of the user-study findings. These are correctable in principle, so the appropriate outcome is a major revision rather than rejection.

major comments (2)
  1. [Section 4, Setup; Section 3.2.2; Section 3.2.4] The benchmark knowledge graphs are built 'per-query from questions and reference passages using the same extraction pipeline' (Section 4, Setup). For ASQA and especially RewardBench, the only available reference passages are gold long-form or preferred responses. Section 3.2.2 creates positive links from entity pairs that co-occur 'within the same utterance,' so during evaluation the gold response's entities become positive training links for the HGT. Section 3.2.4 then ranks entities by cosine similarity between the query embedding and the HGT embeddings (Eq. 2) and injects the top-k into the prompt, while Baseline receives no passages. As a result, the large reported gains on Chat Hard (91.6 vs. 68.4) and Reasoning (94.2 vs. 71.8) may reflect answer leakage into both the retriever's training and its inputs, rather than evidence for selective relational grounding. I ask the authors to rerun the technical evaluation with knowledge graphs built only from the user-visible query and the model's own prior outputs, and to include an ablation that withholds entities appearing in the gold response from the graph, reporting how much of the HGT advantage remains.
  2. [Section 6, Analysis; Section 5.1] With n=9 participants, the minimum achievable two-sided exact p-value for a Wilcoxon signed-rank test is 2/512 = 0.0039. The paper repeatedly reports p<.001 for pairwise Wilcoxon tests (e.g., turn count, BERTScore, token-level IoU, negative feedback count, CUQ in Sections 6.1 and 6.2). These values cannot come from an exact signed-rank test; they appear to be artifacts of a normal approximation that is unreliable at this sample size. The authors should report exact permutation-based p-values, exact confidence intervals, or use small-sample-valid methods, and should verify that the reported p<.001 results survive the correction. This does not necessarily overturn the direction of the effects, but it invalidates the statistical significance claims as stated.
minor comments (5)
  1. [Section 3.2.4, Eq. (2)] The text says HGT scores nodes by 'representation quality (embedding norm) and consistency with neighbors (inter-node similarity),' but Eq. (2) defines relevance only as cosine similarity between the query embedding and the node embedding. If norm or inter-node consistency affect ranking, they should be defined formally; if not, the sentence should be corrected.
  2. [Section 3.2.4, Eq. (3)] The continuity score includes a parameter alpha, but the manuscript never states its value or reports sensitivity analyses; please specify how alpha is chosen.
  3. [Section 4, Setup] The benchmark setup constructs KGs 'per-query,' whereas the deployed system updates and retrains the graph incrementally at every turn; the paper should clarify how the per-query benchmark KGs relate to the per-turn mechanism, including how many HGT training updates are run per query and whether the query's own reference passage is used during training.
  4. [Section 7, Discussion] The phrase 'cognitive grounding' may overstate the contribution, since the paper explicitly disclaims modeling the designer's internal cognition; consider using 'relational grounding' or 'structured grounding' consistently to avoid confusion.
  5. [Section 6 and Figure 4] Several caption and text claims report p<.001 for pairwise Wilcoxon tests; after replacing them with exact small-sample tests, please update all significance symbols and any related effect-size statements consistently.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: benchmark KGs are built from the reference/preferred answers used for scoring, so HGT ranking is trained and prompted on the evaluation signal.

  1. fitted input called prediction [Section 4 (Technical Evaluation, Setup) with Sections 3.2.2–3.2.4]
    "For all four, KGs were constructed per-query from questions and reference passages using the same extraction pipeline. ... Positive link sets are drawn from entity pairs that co-occur within the same utterance ... The model trains at each turn on the current link sets ... The top-k (k≈20) entities are injected into the LLM prompt as structured context."

    The 'reference passages' are the gold/preferred answers that define the benchmark scores. Because the extraction pipeline in Sec. 3.2.1 ingests the assistant response into the graph, the correct answer's entities become nodes; Sec. 3.2.2 turns entity pairs co-occurring inside that passage into positive HGT training links; Sec. 3.2.3 retrains the HGT on these links every turn; Sec. 3.2.4 injects the top-k ranked nodes into the prompt. The HGT ranking is therefore optimized to surface exactly the entities of the reference answer, while Baseline receives no passage and KG-only injects all entities with noise. The largest deltas (Chat Hard +23.2, Reasoning +22.4) occur where preferred-response entities are most diagnostic.

full rationale

The central circularity is in the technical evaluation. Section 4 says KGs are constructed per-query from 'questions and reference passages' using the same extraction pipeline; for ASQA and RewardBench, those passages are the gold long-form answers or preferred responses. Section 3.2.1 extracts entities from the assistant response as well as the user utterance, so the answer's entities enter the graph. Section 3.2.2 defines positive links from entity pairs co-occurring in the same utterance, and Section 3.2.3 retrains the HGT on these links at every turn; Section 3.2.4 injects the HGT top-k nodes into the prompt. The HGT is thus fitted to surface the very entities that define correctness, while Baseline receives no such leakage. The reported ordering H > K > B, especially Chat Hard +23.2 and Reasoning +22.4, follows directly from this construction; it tests context access to the answer, not selective relational grounding. This is a fitted-input/prediction reduction. The user study is not circular in this sense: graphs are built from the designers' own utterances and feedback, and the HGT is not trained on the outcome measures. Its small sample (n=9) and the shared probing/preference components are acknowledged in Section 7 as requiring 'targeted ablations,' which is a confound rather than a construction-level circularity. I found no load-bearing self-citation chain: HGT is an external method [15], and no 'uniqueness' argument is imported from the authors' own prior work. Overall, one central benchmark construction reduces by construction, giving a partial circularity score of 6; the strongest independent content, the user study, does not share the leak.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claims rest on two families of premises the paper does not independently verify: the faithfulness of LLM-based extraction and self-supervised relevance labels built from the same texts being evaluated, plus the small-sample statistical platform. No new physical or conceptual entities beyond the software components (personal heterogeneous graph, probing zones) are postulated, and those are described with their interfaces rather than assumed to exist independently.

free parameters (5)
  • top-k context budget = k approximately 20
    Number of graph entities injected into the LLM prompt each turn (Section 3.2.4); fixed by hand with no sensitivity analysis reported.
  • continuity weighting alpha = not specified (stated to favor embedding similarity)
    Equation (3) blends node-overlap and embedding-similarity terms; alpha is chosen by hand and its value is not reported.
  • probing zone thresholds = mean +/- 1 standard deviation of normalized similarities
    Equation (4) partitions nodes into intentional, exploratory, and excluded zones; the mu-plus-sigma boundaries are a hand-chosen heuristic that adapts to graph size.
  • cold-start bypass size = fewer than 5 nodes
    HGT is bypassed below 5 nodes (Section 3.2.3); threshold chosen by hand.
  • HGT per-turn trained weights = trained each turn on co-occurrence link sets
    HGT embeddings that drive ranking are refit at every turn on positive and negative links derived from the current conversation (Section 3.2.2), so ranking quality is a fitted quantity, not a frozen model.
assumptions (6)
  • domain assumption Typed entity and relation extraction by GraphRAG/GPT-4o faithfully captures the designer's expressed concepts and relations.
    Section 3.2.1 builds the entire graph from this extraction; Section 7 concedes extraction relies on the underlying LLM and may fail on abstract or purely visual content, yet the system's value depends on it.
  • domain assumption CLIP text embeddings (ViT-B/32) provide semantically meaningful initial features for design concepts such as 'warm' or 'heavy'.
    Section 3.2.1 initializes node features with 512-dimensional CLIP embeddings; the meaning resolution claimed in RQ2 depends on these embeddings separating designer-specific senses.
  • domain assumption Co-occurrence within an utterance and explicit feedback define valid positive links for learning entity relevance.
    Section 3.2.2 constructs the HGT training signal from co-occurrence pairs; the benchmarks inherit this definition of relevance from the reference passages.
  • domain assumption Friedman and Wilcoxon inference with Benjamini-Hochberg correction at n=9 yields stable effect-size estimates.
    Section 5.2 Analysis; several reported pairwise p<.001 values are below the exact Wilcoxon minimum at n=9, implying reliance on an approximation.
  • standard math Standard attention and softmax machinery of HGT (Hu et al. 2020) is correct as used.
    Equation (1) imports HGT attention without re-derivation; treated as background.
  • domain assumption LLM-based judgment of extraction precision is a valid accuracy measure.
    Section 7 reports 88.6% precision under strict LLM-based judgment with no human agreement metric; the defensive claim about error propagation depends on this proxy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation." pith.science (2026). https://pith.science/paper/RSB6QLSE

@misc{pith2026260813216,
  author       = {Pith},
  title        = {Pith review of: CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RSB6QLSE}},
  note         = {Machine review of arXiv:2608.13216}
}
read the original abstract

LLM-based chat systems have become valuable tools for design practice, enabling rapid ideation and flexible task support. Yet these systems process designer utterances as generic sequences, maintaining context through recency rather than through any model of how the speaker organizes knowledge. In design conversation, this gap compounds as relational context decays between turns, identical words go unresolved across designers, and the conversation loops or restarts rather than deepens. We present CogChat, a real-time chat framework that grounds conversational AI in a personal heterogeneous knowledge graph constructed from each designer's input. The system extracts typed entities and relations into a heterogeneous graph, then applies a HGT (Heterogeneous Graph Transformer) to select structurally relevant nodes for response generation and to generate both intentional and exploratory probing questions. Technical evaluation shows that HGT-based entity selection outperforms both ungrounded LLM interaction and naive KG augmentation, which introduces noise that degrades response quality. A within-subjects study with nine professional designers indicates that grounding conversation in a relationally structured, designer-specific semantic context improves context retention, personalized intent interpretation, and conversational depth while reducing cognitive load. These findings suggest that structuring a designer's expressed concepts and relations as a dynamic knowledge graph can preserve relational context that fades across turns, pointing toward a graph-grounded approach to long-term context management in LLM-based interaction.

Figures

Figures reproduced from arXiv: 2608.13216 by the authors.

Figure 1
Figure 1. Overview of CogChat. (a) Chat interaction with in-context knowledge lookup. (b) Probing questions with suggested answers and progressively refined outputs across turns. (c) Cognitive knowledge graph constructed from designer input. Abstract LLM-based chat systems have become valuable tools for design practice, enabling rapid ideation and flexible task support. Yet these systems process designer utterances as generic… view at source ↗
Figure 2
Figure 2. CogChat interface. (a) Node and edge counts with extracted entity chips. (b) Double-clicking a term reveals its graph relations. (c) Text input with image generation toggle. (d) Intentional and exploratory questions with suggested answers. personalization has been approached through user profile embed￾dings [47] and preference-conditioned generation [29]. These mod￾els operate on aggregate behavioral patterns rather… view at source ↗
Figure 3
Figure 3. System pipeline. (a) Entity and relation extraction. (b) Positive/negative link set creation with subgraph sampling. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Quantitative results across three conditions. Top [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: P6’s interaction timeline across three conditions. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: (a) P4’s conversation excerpt with CogChat and cognitive knowledge graph snapshots at turn 3 (b1) and turn 5 (b2). Intentional questions narrowed design specifications; an exploratory question introduced “futuristic” as an alternative direction. The graph densifies as …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 27 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023). doi:10.48550/arXiv.2303.08774

    arXiv doi:10.48550/arxiv.2303.08774 2023

  2. [2]

    Krisztian Balog and Tom Kenter. 2019. Personal knowledge graphs: A research agenda. InProceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval. 217–220. doi:10.1145/3341981.3344241

    arXiv 2019

  3. [3]

    Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing.Journal of the Royal Statistical Society: Series B (Methodological)57, 1 (1995), 289–300. doi:10.1111/j. 2517-6161.1995.tb02031.x

    arXiv 1995

  4. [4]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psy- chology.Qualitative Research in Psychology3, 2 (2006), 77–101. doi:10.1191/ 1478088706qp063oa UIST ’26, November 02–05, 2026, Detroit, MI, USA Jiin Choi and Kyung Hoon Hyun

    work page 2006

  5. [5]

    Liwei Chan, Yi-Chi Liao, George B Mo, John J Dudley, Chun-Lien Cheng, Per Ola Kristensson, and Antti Oulasvirta. 2022. Investigating positive and negative qualities of human-in-the-loop optimization for designing interaction techniques. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–14. doi:10.1145/3491102.3501850

    arXiv 2022

  6. [7]

    Nigel Cross. 2011. Understanding design thinking.Notes on doctoral re-search in design: Contributions from the Politecnico Di Milano. FrancoAngeli(2011), p19–37. doi:10.5040/9781474293884

    doi:10.5040/9781474293884 2011

  7. [8]

    Vardhan Dongre, Ryan A Rossi, Viet Dac Lai, David Seunghyun Yoon, Dilek Hakkani-Tür, and Trung Bui. 2025. Drift No More? Context Equilibria in Multi- Turn LLM Interactions.arXiv preprint arXiv:2510.07777(2025). doi:10.48550/ arXiv.2510.07777

    work page doi:10.48550/arxiv.2510.07777 2025

  8. [9]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130(2024). doi:10.48550/arXiv.2404. 16130

    arXiv doi:10.48550/arxiv.2404.16130 2024

Show all 65 references
  1. [10]

    Frederic Gmeiner, Kaitao Luo, Ye Wang, Kenneth Holstein, and Nikolas Martelaro

  2. [11]

    2014.Linkography: unfolding the design process

    Gabriela Goldschmidt. 2014.Linkography: unfolding the design process. Mit Press

    2014

  3. [12]

    Aditya Gunturu, Ben Pearman, Keiichi Ihara, Morteza Faraji, Bryan Wang, Ruba- iat Habib Kazi, and Ryo Suzuki. 2025. MapStory: Prototyping Editable Map Animations with LLM Agents. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 1–20. do...

    2025

  4. [13]

    Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.Advances in Psychology 52 (1988), 139–183. doi:10.1016/s0166-4115(08)62386-9

    1988 doi

  5. [14]

    Sam Holmes, Anne Moorhead, Raymond Bond, Huiru Zheng, Vivien Coates, and Michael McTear. 2019. Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces?. InProceedings of the 31st European Conference on Cognitive Ergon...

    2019

  6. [15]

    Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous graph transformer. InProceedings of the web conference 2020. 2704–2710. doi:10. 48550/arXiv.2003.01332

    2020 doi arXiv

  7. [16]

    Youwen Kang, Zhida Sun, Sitong Wang, Zeyu Huang, Ziming Wu, and Xiao- juan Ma. 2021. MetaMap: Supporting visual metaphor ideation through multi- dimensional example-based exploration. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15. doi:10....

    2021

  8. [17]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907(2016). doi:10.48550/ arXiv.1609.02907

    2016 doi arXiv

  9. [18]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. Llms get lost in multi-turn conversation.arXiv preprint arXiv:2505.06120(2025). doi:10. 48550/arXiv.2505.06120

    2025 doi arXiv

  10. [19]

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, Lester James Validad Mi- randa, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. 2025. Rewardbench: Evaluating reward models for language modeling. InFindings of the Association for Comp...

    2025

  11. [20]

    2006.How designers think

    Bryan Lawson. 2006.How designers think. Routledge

    2006

  12. [21]

    doi:10.18653/v1/2025.findings-naacl.96

    1755–1797. doi:10.18653/v1/2025.findings-naacl.96

    2025 doi

  13. [22]

    Hyunseung Lim, Ji Yong Cho, Taewan Kim, Jeongeon Park, Hyungyu Shin, Seulgi Choi, Sunghyun Park, Kyungjae Lee, Juho Kim, Moontae Lee, et al . 2024. Co- Creating Question-and-Answer Style Articles with Large Language Models for Research Promotion. InProceedings of the 2024 ACM ...

    2024

  14. [23]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al . 2020. Retrieval-augmented generation for knowledge- intensive nlp tasks.Advances in neural information processing ...

    2020

  15. [24]

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating Very Long-Term Conversational Memory of LLM Agents.arxiv(2024). doi:10.48550/arXiv.2402.17753

    2024 doi arXiv

  16. [25]

    Hyunseung Lim, Dasom Choi, and Hwajung Hong. 2024. Identify Design Prob- lems Through Questioning: Exploring Role-playing Interactions with Large Lan- guage Models to Foster Design Questioning Skills. InCompanion Publication of the 2024 Conference on Computer-Supported Coopera...

    2024

  17. [26]

    Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. 2023. MemGPT: towards LLMs as operating systems. (2023). doi:10.48550/arXiv.2310.0856

    2023 doi

  18. [28]

    Kevin Pu, Ting Zhang, Naveen Sendhilnathan, Sebastian Freitag, Raj Sodhi, and Tanya R Jonker. 2025. Promemassist: Exploring timely proactive assistance through working memory modeling in multi-modal wearable devices. InProceed- ings of the 38th Annual ACM Symposium on User Int...

    2025

  19. [30]

    Apoorv Saxena, Soumen Chakrabarti, and Partha Talukdar. 2021. Question answering over temporal knowledge graphs. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing...

    2021

  20. [31]

    Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. Lamp: When large language models meet personalization. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 7370–7392. doi:10.18653/v1/2024...

    2024 doi

  21. [32]

    D Schön. 1983. The reflective practitioner. New York, NY: Basic Book. (1983). doi:10.2307/j.ctvpbnpjg.24

    1983 doi

  22. [33]

    Sina Semnani, Violet Yao, Heidi Zhang, and Monica Lam. 2023. WikiChat: Stop- ping the hallucination of large language model chatbots by few-shot grounding on Wikipedia. InFindings of the association for computational linguistics: EMNLP

    2023

  23. [34]

    Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. InEuropean semantic web conference. Springer, 593–607. doi:10.1007/ 978-3-319-93417-4_38

    2018

  24. [35]

    Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. ASQA: Factoid questions meet long-form answers. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 8273–8288. doi:10.18653/ v1/2022.emnlp-main.566

    2022

  25. [36]

    Maciej Tomczak and Ewa Tomczak. 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. https: //api.semanticscholar.org/CorpusID:73706075

    2014

  26. [37]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023). doi:10.48550/...

    2023 doi arXiv

  27. [38]

    Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. InProceedings of the AAAI conference on artificial intelligence, Vol. 31. doi:10.1609/aaai.v31i1.11164

    2017 doi

  28. [39]

    Viswanath Venkatesh and Hillol Bala. 2008. Technology acceptance model 3 and a research agenda on interventions.Decision sciences39, 2 (2008), 273–315. doi:10.1111/j.1540-5915.2008.00192.x

    2008

  29. [40]

    Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase.Commun. ACM57, 10 (2014), 78–85. doi:10.1145/2629489

    2014 doi

  30. [41]

    Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2024. Longmemeval: Benchmarking chat assistants on long-term interactive memory.arXiv preprint arXiv:2410.10813(2024). doi:10.48550/arXiv.2410.10813

    2024 doi arXiv

  31. [42]

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks.arXiv preprint arXiv:1710.10903(2017). doi:10.48550/arXiv.1710.10903

    2017 doi arXiv

  32. [43]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations

    2022

  33. [44]

    Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–98...

    2018 doi

  34. [45]

    Andrew Zhai, Dmitry Kislyuk, Yushi Jing, Michael Feng, Eric Tzeng, Jeff Donahue, Yue Li Du, and Trevor Darrell. 2017. Visual discovery at pinterest. InProceedings of the 26th international conference on world wide web companion. 515–524. doi:10. 1145/2783258.2788621

    2017

  35. [46]

    Youfu Yan, Yu Hou, Yongkang Xiao, Rui Zhang, and Qianwen Wang. 2024. Knownet: Guided health information seeking from llms via knowledge graph integration.IEEE Transactions on Visualization and Computer Graphics31, 1 (2024), 547–557. doi:10.1109/tvcg.2024.3456364/mm1

    2024

  36. [47]

    Yinhe Zheng, Rongsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020. A pre- training based personalized dialogue generation model with persona-sparse data. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 9693–9700. doi:10.1609/aaai.v34i05.6518

    2020 doi

  37. [48]

    {user_message}

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Mem- orybank: Enhancing large language models with long-term memory. InPro- ceedings of the AAAI conference on artificial intelligence, Vol. 38. 19724–19731. doi:10.48550/arXiv.2305.10250 Appendices A System ...

    2024 doi arXiv

  38. [50]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi

  39. [54]

    Use 1-hop and 2-hop graph connections to suggest related options

  40. [55]

    questions

    Build on previous choices to explore variations Return JSON format: { "questions": [ { "question": "specific clarifying question to resolve ambiguity", "suggested_answers": ["option 1", "option 2", "option 3"] } ] } Exploratory probing prompt (KG-only) User said: "{user_messag...

  41. [56]

    Use 2-3 hop graph connections to find creative alternatives

  42. [57]

    Suggest complementary or contrasting options

  43. [58]

    questions

    Expand into related but different directions CREATIVE EXPLORATION RULES: - Look beyond direct connections to find interesting combinations - Introduce related concepts from the broader graph Return JSON format: { "questions": [ { "question": "creative exploration question usin...

  44. [59]

    Transform abstract concepts into concrete options

  45. [60]

    questions

    Use HGT embeddings to find semantically related but more specific options Return JSON format: { "questions": [ { "question": "specific clarifying question to resolve ambiguity", "suggested_answers": ["option 1", "option 2", "option 3"] } ] } Exploratory probing prompt (KG+HGT)...

  46. [61]

    Use semantic embeddings to discover unexpected but relevant connections

  47. [62]

    feel related

    Go beyond direct graph connections to find creative alternatives CREATIVE EXPANSION RULES: - Use semantic similarity to find concepts that "feel related" - Balance novelty with relevance to the conversation Return JSON format: { "questions": [ { UIST ’26, November 02–05, 2026,...

    2026

  48. [64]

    Uses the conversation context to understand what the user wants

  49. [65]

    {message}

    Creates a cohesive prompt that combines all their stated preferences Return ONLY the image generation prompt, nothing else. A.3.2KG-only.Image generation prompt (KG-only) You are helping with an ongoing conversation. The user has requested a new image. IMPORTANT CONTEXT: [Hist...

  50. [68]

    {message}

    Combines user preferences with graph-based design knowledge Return ONLY the image generation prompt, nothing else. A.3.3CogChat. Image generation prompt (KG+HGT) You are helping with an ongoing conversation. The user has requested a new image. IMPORTANT CONTEXT: [History conte...

    2026

  51. [69]

    MUST include ALL user choices from Q&A (colors, styles, materials, etc.)

  52. [70]

    Incorporates knowledge graph concepts that are relevant

  53. [71]

    B Participant Demographics and Survey Instruments Table B1 summarizes participant backgrounds

    Combines user preferences with graph-based design knowledge Return ONLY the image generation prompt, nothing else. B Participant Demographics and Survey Instruments Table B1 summarizes participant backgrounds. All nine designers were active LLM users (𝑀= 5.33/7,𝑆𝐷= 1.66) with ...

    2026

  54. [2020]

    InInternational Confer- ence on Learning Representations

    BERTScore: Evaluating text generation with BERT. InInternational Confer- ence on Learning Representations. doi:10.48550/arXiv.1904.09675 CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation UIST ...

    1904 doi arXiv

  55. [2023]

    doi:10.18653/v1/2023.findings-emnlp.157

    2387–2413. doi:10.18653/v1/2023.findings-emnlp.157

    2023 doi

  56. [2025]

    InProceedings of the 2025 ACM Designing Interactive Systems Conference

    Exploring the potential of metacognitive support agents for human-AI co- creation. InProceedings of the 2025 ACM Designing Interactive Systems Conference. 1244–1269. doi:10.1145/3715336.3735785

    2025

  57. [6676]

    doi:10.48550/arXiv.2106.01515

    doi arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.