Promoting Real-Time Reflection in Synchronous Communication with Generative AI

Meng Xia; Yi Wen

REVIEW 2 major objections 6 minor 20 references

Promoting Real-Time Reflection in Synchronous Communication with Generative AI

T0 review · 2 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A review of 11 systems argues that generative AI can support real-time reflection during synchronous communication if designers add lightweight explanations, proactive notifications, and richer persona grounding.

desk verdict Useful compact review of 11 systems, but the design implications are presented as if they rest on a user study that never appears in the paper. read the letter →

arxiv 2504.15647 v2 pith:FTOSBNSE submitted 2025-04-22 cs.HC

Yi Wen , Meng Xia This is my paper · ORCID

classification cs.HC

keywords real-timereflectionsynchronouscommunicationgenerativeAIdesignimplicationshuman-AIinteractionambientinformationsystemsrole-playingagentsproactivenotification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper reviews 11 systems for supporting reflection during synchronous communication—live meetings, online classes, presentations, and practice talks—and tries to establish that generative AI can make real-time reflection practical if the interaction is designed a certain way. It sorts current systems by how they support reflection (increasing contextual awareness versus evaluating performance and giving suggestions) and by interaction paradigm (user-initiated, system-initiated, continuous display). From that synthesis it derives three design implications: make AI-generated reflective output interpretable with lightweight explanations, let generative AI proactively deliver well-timed notifications, and ground role-play personas in richer context. The sympathetic reading is that these implications chart a path for future systems; the text itself signals that the implications are based on the findings of the user study, but it does not report or cite such a study, so the evidence base is an open question.

What carries the argument

The central machinery is a three-part analytical map of the reviewed systems. First, a dichotomy of support strategies: increasing contextual awareness (simulating audience feedback, aggregating audience status, summarizing past conversation) versus evaluating performance and offering expert suggestions. Second, a trichotomy of interaction paradigms—user-initiated, system-initiated (proactive), and continuous display—together with the notification levels those choices imply, a categorization borrowed from the ambient-information-systems taxonomy. Third, the role of generative AI in each cell, from not needed for simple statistics to understanding the conversation and generating feedback for expert and audience simulation. This map does the argument's work: it turns individual systems into design patterns that make the three implications look like natural corrections to observed limitations.

What would settle it

Find the user study in Section 4: a search of the manuscript shows no user study is described or cited, so the stated basis of the implications is unverifiable as written. A stronger check would be a controlled experiment in which novice tutors run the same online lesson with a TutorUp-style proactive feedback system, the same system with lightweight explanations added, and no system; if explained proactive feedback does not improve reflection quality or lower perceived disruption, the implication fails.

Watch

Extended reading notes

Core claim

The central claim is that real-time reflection in synchronous communication—the ability to evaluate and adjust one's communication while it is still happening—can be supported by generative AI without disrupting the ongoing conversation, provided the support follows three principles: explainable, lightweight AI output; proactive rather than user-initiated delivery at critical moments; and persona-grounded role-play feedback agents. The paper grounds this claim in a structured review of 11 existing systems, mapping them onto two support strategies and three interaction paradigms, and using an ambient-information taxonomy to characterize notification levels. It further argues that generative AI changes what is possible in this space: instead of simple statistics about audience status, LLM/VLM systems can summarize conversation structure, extract consensus and key opinions, integrate multimodal cues, and simulate an audience member or an expert. The design implications are presented as the paper's main result, with each tied to a perceived shortcoming of current systems.

Load-bearing premise

The load-bearing premise is that a user study exists whose findings support the three design implications; the paper refers to the user study in Section 4 but neither reports nor cites one, and if the intended study is the TutorUp pilot, it covers only one system.

Editorial extensions

If this is right

A reflection-support system that adds brief annotations or visual cues explaining how an AI result was generated should reduce user distrust and confusion without adding much cognitive load.
Generative AI that proactively detects critical moments can lower the number of manual steps users must take, as long as notification timing and relevance are carefully controlled.
Role-play agents such as simulated audience members or simulated tutors need rich, context-sensitive persona modeling and clear evaluation criteria; without these, their feedback will be too generic or inconsistent to help reflection.
With LLM/VLM support, audience-status displays can move from statistical charts to summaries of conversation structure, key opinions, consensus points, and multimodal cues, making reflection richer in real time.
Designers must choose their notification level deliberately—change-blind displays, make-aware notifications, interruptive alerts, or attention-demanding textual channels—because the same reflective information can either support or disrupt the primary communication task depending on that choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One testable extension is direct comparison: deliver the same reflective content as continuous display, proactive notification, and user-initiated lookup in a controlled communication task, then measure reflection depth, interruption, trust, and user agency; the paper implies but does not test which paradigm wins in which scenario.
The Section 4 claim that the implications are based on the findings of the user study points to an evidence gap: no user study is reported or cited in the manuscript, and the only plausible candidate is the single-system TutorUp pilot [14]. If that is the intended source, the field-wide implications would be an extrapolation from one system, not a validated result.
The review's categories suggest a neighboring design question the paper leaves implicit: whether generative AI should act as a separate reflection channel alongside the conversation or be woven into the existing communication interface, for example as subtle inline cues in a shared transcript. The interaction-paradigm trichotomy could be used as a design space for that choice.
Because the corpus was restricted to the last five years and to a single bibliographic database, the patterns identified may miss reflection systems from other venues; a broader corpus would be a direct way to check whether the three implications generalize.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth reading as a compact map of 11 systems, but don't take Section 4's design implications as evidence-based: they claim to rest on "the findings of the user study," and no user study appears anywhere in the paper.

The best part is Table 1. Grouping systems by whether they increase contextual awareness or evaluate performance and give suggestions, then tagging design pattern, notification level, and role of GenAI, is a clear and mostly accurate synthesis. The three interaction paradigms (user-initiated, system-initiated, continuous display) map cleanly onto the examples, and the point that GenAI opens up role-play of audience/expert personas is genuinely useful. For a six-page workshop piece, the descriptive review does its job.

The soft spots are real, though. Section 4 says the implications are based on a user study, but the manuscript has no method section for it, no participants, no results, and no citation. If the intended study is TutorUp [14], that's the authors' own prior work, not an independent empirical basis. The implications themselves — add lightweight explanations, use proactiveness carefully, ground personas — are plausible and consistent with prior HCI work, but they are expert opinions dressed up as findings. The search method is also thin: ACM DL only, a few keywords, "last five years," no inclusion/exclusion criteria, and the table includes systems from 2018 and 2019, which sits awkwardly with the stated filter. The citation for "Joshua [13]" points to a design-fiction paper, not a system called Joshua; that looks like a broken reference.

None of this is fatal for a position paper, but the authors need to either include the study, cite a public version of it, or rephrase Section 4 as design recommendations derived from the review, which would be honest and still useful.

Who should read it: HCI folks working on real-time reflection, meeting analytics, or AI feedback in communication. They'll get a good related-works starting point and a citable taxonomy, but not a definitive result. If I were a serious editor, I'd send it to reviewers with a clear request to fix the evidence claim; it's salvageable. I would not cite it as empirical support for anything.

Referee Report

2 major / 6 minor

Summary. This position paper reviews systems that support real-time reflection in synchronous communication (meetings, online classes, presentations, practice talks) and proposes design implications for incorporating generative AI into such systems. The authors searched the ACM Digital Library for the last five years, identified 11 papers, and organized them in Table 1 by the way reflection is supported (increasing contextual awareness vs. evaluating performance and giving suggestions), by interaction paradigm (user-initiated, system-initiated, continuous display), and by notification level. Section 3 argues that generative AI can move beyond simple statistics to produce nuanced summaries, role-play audiences or experts, and deliver timely, contextual feedback. Section 4 presents three design implications: add lightweight explanations, leverage proactive notifications to reduce workload, and improve persona grounding for role-play agents.

Significance. If the claims are taken as design proposals, the paper offers a useful organizing taxonomy for a small but growing design space, and its three implications align with broader HCI findings on explanation, interruption, and AI persona credibility. The paper is honest about being a position piece, and its mapping of existing systems in Table 1 is a convenient starting point. However, the central Section 4 explicitly attributes the design implications to "findings of the user study," and no such study is reported or cited; the only candidate, the authors' own TutorUp [14], is cited as a system, not as a user evaluation. In addition, the literature review is not reproducible and its inclusion criterion is not consistently applied. These issues mean the paper's main prescriptive conclusions are not empirically grounded as written, although they could become defensible if reframed as design recommendations based on the review and prior work.

major comments (2)

[Section 4, first paragraph] The paper states: "We analyze the limitations of current systems based on the findings of the user study and propose the following design implications." No user study is described anywhere in the manuscript, in Section 2's method, or in the reference list. The only possible source, TutorUp [14], is cited as a system description (an arXiv preprint) and no participants, procedure, measures, or results are reported. Consequently, the three implications in §4.1–§4.3 are presented as evidence-based but actually rest on an unverifiable empirical foundation. This is a load-bearing issue because the paper's strongest claim is that these improvements will make real-time reflection less disruptive. The authors should either include a summary of the study (and a citation to a permanent report), or reword Section 4 to present the implications as design proposals derived from the review and prior literature, not from an inaccessible user study.
[Section 2 and Table 1] The literature-search method is not reproducible. The authors list only broad keywords ('meeting', 'reflection', 'online classes', 'presentation', 'practice', 'training') with no search string, no database query syntax, no inclusion/exclusion criteria beyond 'last five years', and no screening procedure. More seriously, the stated five-year filter is violated by the selected corpus: TalkTraces [3] (2019), Joshua [13] (2018), Coco [18] (2018), and the audience-flow study [19] (2019) all fall outside 2020–2025. In addition, MeetScript [5] is discussed in §3.2 as a continuous-display system but is omitted from Table 1, making the claimed total of 11 papers unverifiable from the table alone. These inconsistencies undermine the representativeness premise on which the review's design-space conclusions are built.

minor comments (6)

[Title page] The title contains an erroneous line break within the word "Communication" ("Communicati on") in the running header; please fix the typography.
[Section 3.2] The phrase "These systems can be change blind" should read "can be change-blind" or "can suffer from change blindness" for grammatical clarity.
[Reference [13]] The text describes "Joshua [13]" as a VR system for speech visualization, but the reference cited is titled "Immersive design fiction: Using VR to prototype speculative interfaces and interaction rituals within a virtual storyworld" and does not name a system called Joshua; the citation appears to be mismatched.
[Section 2] The sentence "Finally, there are 11 papers that satisfy the conditions" refers to unspecified conditions; the authors should enumerate the exact inclusion and exclusion criteria used in the search.
[Section 2] The paper says the taxonomy is "proposed by Zachary et al. [16]", but reference [16] is by Pousman and Stasko; the in-text author name "Zachary" appears to be a mistake.
[General] The manuscript still contains the placeholder ACM DOI and the note about "acm-jdslogo.png"; these should be removed or resolved in a camera-ready version.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the review is self-contained, but Section 4's appeal to an unnamed user study is an evidentiary gap, not a circular step.

full rationale

This is a position paper/review with no equations or formal derivation chain. The central contribution is a synthesis of 11 systems (Sections 2-3) and three design implications (Section 4). The implications are not predictions and are not obtained by fitting parameters or by renaming inputs; they are reasoned recommendations grounded in the reviewed systems. The one self-citation, TutorUp [14] (co-authored by Meng Xia), appears as an instance of system-initiated interaction in Section 3.2 and as an example in Table 1. It is illustrative rather than load-bearing: removing TutorUp would not collapse the argument, since AudiLens [15] and other reviewed systems independently instantiate proactive and role-play patterns. No uniqueness theorem or prior-author assumption is invoked to force a conclusion. The only notable issue is Section 4's sentence, "We analyze the limitations of current systems based on the findings of the user study and propose the following design implications," while no user study is included, described in Section 2, or cited in the references. This is an omitted-proof and missing-evidence problem, not a circularity: the implications do not reduce by construction to the unstated study. It weakens empirical grounding but does not make the derivation circular. The score reflects one minor self-citation and the missing-study caveat, not load-bearing circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no free parameters, new entities, or formal derivations. Its central contribution is a synthesis of existing systems plus design recommendations. The recommendations rest on the domain assumptions listed above, particularly the unstated user study and the representativeness of the 11-paper sample.

assumptions (5)

domain assumption Real-time reflection is a key mechanism for improving communication effectiveness and is feasible with AI assistance.
Asserted in the abstract and introduction with citations but not empirically tested in this paper.
domain assumption Synchronous communication leaves insufficient cognitive bandwidth and feedback for speakers to reflect unaided.
Motivates the entire design space; cited to [17,18] but not established by this review.
ad hoc to paper The 11 papers selected from ACM DL over the last five years provide a representative sample of the design space.
Section 2's informal search has no protocol, inclusion/exclusion criteria, or quality assessment; the representativeness of the sample is assumed.
domain assumption The taxonomy of ambient information systems by Zachary et al. [16] is an appropriate lens for classifying the reviewed systems.
Invoked in Section 2 without justification; the taxonomy comes from prior literature and may not fit all systems.
domain assumption LLMs can act as believable audience or expert personas when given domain context.
Discussed in Section 3.3 and Section 4.3; the reliability of role-play agents is a known open problem, not proven here.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Promoting Real-Time Reflection in Synchronous Communication with Generative AI." pith.science (2026). https://pith.science/paper/FTOSBNSE

@misc{pith2026250415647,
  author       = {Pith},
  title        = {Pith review of: Promoting Real-Time Reflection in Synchronous Communication with Generative AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FTOSBNSE}},
  note         = {Machine review of arXiv:2504.15647}
}

read the original abstract

Real-time reflection plays a vital role in synchronous communication. It enables users to adjust their communication strategies dynamically, thereby improving the effectiveness of their communication. Generative AI holds significant potential to enhance real-time reflection due to its ability to comprehensively understand the current context and generate personalized and nuanced content. However, it is challenging to design the way of interaction and information presentation to support the real-time workflow rather than disrupt it. In this position paper, we present a review of existing research on systems designed for reflection in different synchronous communication scenarios. Based on that, we discuss design implications on how to design human-AI interaction to support reflection in real time.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 18 canonical work pages

[14]

Sitong Pan, Robin Schmucker, Bernardo Garcia Bulle Bue no, Salome Aguilar Llanes, Fernanda Albo Alarcón, Hangxiao Zhu, Adam Teo, and Meng Xia. 2025. TutorUp: What If Your Students Were Simulated? Training Tutors to Address Engagement Challenges in Online Learning. arXiv preprint arXiv:2502.16178 (2025)

work page Pith review arXiv 2025
[13]

Joshua McVeigh-Schultz, Max Kreminski, Keshav Prasad , Perry Hoberman, and Scott S Fisher. 2018. Immersive designﬁction: Using VR to prototype speculative interfaces and interaction rituals within a virtual storyworld. In Proceedings of the 2018 designing interactive systems conference. 817–829

work page 2018
[3]

Senthil Chandrasegaran, Chris Bryan, Hidekazu Shidara , Tung-Yen Chuang, and Kwan-Liu Ma. 2019. TalkTraces: Real- time capture and visualiza- tion of verbal content in meetings. In Proceedings of the 2019 CHI conference on human factors in com puting systems. 1–14

work page 2019
[18]

Samiha Samrose, Ru Zhao, Jeﬀery White, Vivian Li, Luis N ova, Yichen Lu, Mohammad Rafayet Ali, and Mohammed Ehsan Hoq ue. 2018. Coco: Collaboration coach for understanding team dynamics durin g video conferencing. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 1, 4 (2018), 1–24

work page 2018
[19]

Wei Sun, Yunzhi Li, Feng Tian, Xiangmin Fan, and Hongan W ang. 2019. How presenters perceive and react to audience ﬂow prediction in-situ: An explorative study of live online lectures. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–19. Manuscript submitted to ACM 6 Wen et al

work page 2019
[5]

Xinyue Chen, Shuo Li, Shipeng Liu, Robin Fowler, and Xu Wa ng. 2023. Meetscript: designing transcript-based interac tions to support active participation in group video meetings. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1–32

work page 2023
[1]

Bon Adriel Aseniero, Marios Constantinides, Sagar Jogl ekar, Ke Zhou, and Daniele Quercia. 2020. MeetCues: Supporting online meetings experience. In 2020 IEEE Visualization Conference (VIS) . IEEE, 236–240

work page 2020
[2]

Eric PS Baumer, Vera Khovanskaya, Mark Matthews, Lindsa y Reynolds, Victoria Schwanda Sosik, and Geri Gay. 2014. Rev iewing reﬂection: on the use of reﬂection in interactive system design. In Proceedings of the 2014 conference on Designing interactive systems. 93–102

work page 2014

Show all 20 references

[4]

Chaoran Chen, Bingsheng Yao, Ruishi Zou, Wenyue Hua, Wei min Lyu, Yanfang Ye, Toby Jia-Jun Li, and Dakuo Wang. 2025. To wards a Design Guideline for RPA Evaluation: A Survey of Large Language Mod el-Based Role-Playing Agents. arXiv preprint arXiv:2502.13012 (2025)

2025 arXiv
[6]

Xinyue Chen, Nathan Yap, Xinyi Lu, Aylin Gunal, and Xu Wan g. 2025. MeetMap: Real-Time Collaborative Dialogue Mappin g with LLMs in Online Meetings. arXiv preprint arXiv:2502.01564 (2025)

2025 arXiv
[7]

Hyunsung Cho, DaEun Choi, Donghwi Kim, Wan Ju Kang, Eun Ky oung Choe, and Sung-Ju Lee. 2021. Reﬂect, not regret: Unders tanding regretful smartphone use with app feature-level analysis. Proceedings of the ACM on human-computer interaction 5, CSCW2 (2021), 1–36

2021
[8]

DaEun Choi, Sumin Hong, Jeongeon Park, John Joon Young Ch ung, and Juho Kim. 2024. CreativeConnect: Supporting Refer ence Recombination for Graphic Design Ideation with Generative AI. In Proceedings of the 2024 CHI Conference on Human Factors in Com puting Systems. 1–25

2024
[9]

Jingchao Fang, Jeongeon Park, Juho Kim, and Hao-Chuan Wa ng. 2024. EduLive: Re-Creating Cues for Instructor-Learne rs Interaction in Educa- tional Live Streams with Learners’ Transcript-Based Annot ations. Proc. ACM Hum.-Comput. Interact. 8, CSCW2, Article 421 (Nov. 2024), 33 ...

2024 doi
[10]

Ian Li, Anind K Dey, and Jodi Forlizzi. 2011. Understand ing my data, myself: supporting self-reﬂection with ubicom p technologies. In Proceedings of the 13th international conference on Ubiquitous computi ng. 405–414

2011
[11]

Yuhan Luo, Young-Ho Kim, Bongshin Lee, Naeemul Hassan, and Eun Kyoung Choe. 2021. Foodscrap: Promoting rich data ca pture and reﬂective food journaling through speech input. In Proceedings of the 2021 ACM Designing Interactive Systems Co nference. 606–618

2021
[12]

Shuai Ma, Taichang Zhou, Fei Nie, and Xiaojuan Ma. 2022. Glancee: An adaptable system for instructors to grasp stude nt learning status in synchronous online classes. In Proceedings of the 2022 CHI conference on human factors in com puting systems. 1–25

2022
[15]

Jeongeon Park and DaEun Choi. 2023. AudiLens: Conﬁgura ble LLM-Generated Audiences for Public Speech Practice(UIST ’23 Adjunct). Association for Computing Machinery, New York, NY, USA. doi:10.1145/35 86182.3625114

2023
[16]

Zachary Pousman and John Stasko. 2006. A taxonomy of amb ient information systems: four patterns of design. In Proceedings of the working conference on Advanced visual interfaces . 67–74

2006
[17]

Samiha Samrose, Daniel McDuﬀ, Robert Sim, Jina Suh, Kae l Rowan, Javier Hernandez, Sean Rintel, Kevin Moynihan, and Mary Czerwinski. 2021. Meetingcoach: An intelligent dashboard for supporting eﬀe ctive & inclusive meetings. In Proceedings of the 2021 CHI Conference on Human F...

2021
[20]

acm-jdslogo.png

Ashley Ge Zhang, Yan Chen, and Steve Oney. 2023. Vizprog : Identifying misunderstandings by visualizing students’ coding progress. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Syst ems. 1–16. Manuscript submitted to ACM This figure "acm-jdslogo.png" ...

2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [14]

Sitong Pan, Robin Schmucker, Bernardo Garcia Bulle Bue no, Salome Aguilar Llanes, Fernanda Albo Alarcón, Hangxiao Zhu, Adam Teo, and Meng Xia. 2025. TutorUp: What If Your Students Were Simulated? Training Tutors to Address Engagement Challenges in Online Learning. arXiv preprint arXiv:2502.16178 (2025)

work page Pith review arXiv 2025

[2] [13]

Joshua McVeigh-Schultz, Max Kreminski, Keshav Prasad , Perry Hoberman, and Scott S Fisher. 2018. Immersive designﬁction: Using VR to prototype speculative interfaces and interaction rituals within a virtual storyworld. In Proceedings of the 2018 designing interactive systems conference. 817–829

work page 2018

[3] [3]

Senthil Chandrasegaran, Chris Bryan, Hidekazu Shidara , Tung-Yen Chuang, and Kwan-Liu Ma. 2019. TalkTraces: Real- time capture and visualiza- tion of verbal content in meetings. In Proceedings of the 2019 CHI conference on human factors in com puting systems. 1–14

work page 2019

[4] [18]

Samiha Samrose, Ru Zhao, Jeﬀery White, Vivian Li, Luis N ova, Yichen Lu, Mohammad Rafayet Ali, and Mohammed Ehsan Hoq ue. 2018. Coco: Collaboration coach for understanding team dynamics durin g video conferencing. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 1, 4 (2018), 1–24

work page 2018

[5] [19]

Wei Sun, Yunzhi Li, Feng Tian, Xiangmin Fan, and Hongan W ang. 2019. How presenters perceive and react to audience ﬂow prediction in-situ: An explorative study of live online lectures. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–19. Manuscript submitted to ACM 6 Wen et al

work page 2019

[6] [5]

Xinyue Chen, Shuo Li, Shipeng Liu, Robin Fowler, and Xu Wa ng. 2023. Meetscript: designing transcript-based interac tions to support active participation in group video meetings. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1–32

work page 2023

[7] [1]

Bon Adriel Aseniero, Marios Constantinides, Sagar Jogl ekar, Ke Zhou, and Daniele Quercia. 2020. MeetCues: Supporting online meetings experience. In 2020 IEEE Visualization Conference (VIS) . IEEE, 236–240

work page 2020

[8] [2]

Eric PS Baumer, Vera Khovanskaya, Mark Matthews, Lindsa y Reynolds, Victoria Schwanda Sosik, and Geri Gay. 2014. Rev iewing reﬂection: on the use of reﬂection in interactive system design. In Proceedings of the 2014 conference on Designing interactive systems. 93–102

work page 2014