pith. sign in

arxiv: 2604.15800 · v1 · submitted 2026-04-17 · 💻 cs.HC · cs.AI· cs.CL

From Intention to Text: AI-Supported Goal Setting in Academic Writing

Yueling Fan , Richard Lee Davis , Olga Viberg This is my paper

Pith reviewed 2026-05-10 08:00 UTC · model grok-4.3

classification 💻 cs.HC cs.AIcs.CL
keywords academic writingAI writing assistantgoal settingmetacognitionreflective writinghuman-AI collaborationWizard-of-Oz
0
0 comments X

The pith

WriteFlow supports metacognitive regulation in academic writing by enabling iterative goal refinement and alignment with text through voice-based dialogue.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces WriteFlow, an AI voice-based writing assistant that frames interaction as a dialogic space for articulating, monitoring, and negotiating writing goals. It establishes that this design scaffolds metacognitive regulation by supporting iterative goal refinement, maintaining alignment between goals and text, and prompting writers to evaluate goal fulfillment. A Wizard-of-Oz study with expert users demonstrates these benefits, contrasting with efficiency-focused AI tools that offer less support for reflection and agency. Sympathetic readers would care because this approach could help writers maintain control over their intentions rather than letting tools dictate the output.

Core claim

WriteFlow frames AI interaction as a dialogic space for ongoing goal articulation, monitoring, and negotiation grounded in writers' intentions. Findings show that it scaffolds metacognitive regulation and reflection-in-action by supporting iterative goal refinement, maintaining goal-text alignment during drafting, and prompting evaluation of goal fulfillment.

What carries the argument

WriteFlow's goal-oriented dialogic interaction mechanism that turns writing assistance into a conversation about evolving intentions.

If this is right

  • Iterative goal refinement becomes integrated into the drafting process.
  • Goal-text alignment is actively maintained through AI prompts.
  • Evaluation of whether goals are fulfilled is prompted as part of the workflow.
  • AI writing systems benefit from prioritizing reflective dialogue and flexible goal structures over pure text generation efficiency.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This approach might apply to other creative or professional writing tasks where intention alignment is key.
  • Real-world deployment could test if the benefits persist without the Wizard-of-Oz simulation.
  • It points toward AI designs that act as partners in thinking rather than just producers of text.

Load-bearing premise

Insights from a small sample of 12 expert users in a simulated Wizard-of-Oz environment generalize to real AI systems and broader populations of academic writers.

What would settle it

A larger study with a fully implemented AI version of WriteFlow that measures increases in goal revisions and reflection compared to standard AI writing tools.

Figures

Figures reproduced from arXiv: 2604.15800 by Olga Viberg, Richard Lee Davis, Yueling Fan.

Figure 1
Figure 1. Figure 1: The overview of WriteFlow, a Google Docs add-on for goal-oriented academic writing. WriteFlow interface consists of a voice agent (A) and a sidebar panel with three pages: Writing Task (B), AI Chat (C), and My Goals (D). Users can upload Google Docs and communicate with the voice agent at any stage of writing to discuss their writing directions. The agent then generates writing goals to help them plan, tra… view at source ↗
Figure 2
Figure 2. Figure 2: Procedure of the user study. writing conventions and the design and evaluation of interactive systems enabled an informed assessment of WriteFlow’s support for goal articulation, authorial voice, and user control. Also, participants exhibited diverse AI use practices: five reported very frequent use of AI for writing, four reported frequent but more limited use, and three reported occasional use for specif… view at source ↗
read the original abstract

This study presents WriteFlow, an AI voice-based writing assistant designed to support reflective academic writing through goal-oriented interaction. Academic writing involves iterative reflection and evolving goal regulation, yet prior research and a formative study with 17 participants show that writers often struggle to articulate and manage changing goals. While commonly used AI writing tools emphasize efficiency, they offer limited support for metacognition and writer agency. WriteFlow frames AI interaction as a dialogic space for ongoing goal articulation, monitoring, and negotiation grounded in writers' intentions. Findings from a Wizard-of-Oz study with 12 expert users show that WriteFlow scaffolds metacognitive regulation and reflection-in-action by supporting iterative goal refinement, maintaining goal-text alignment during drafting, and prompting evaluation of goal fulfillment. We discuss design implications for AI writing systems that prioritize reflective dialogue, flexible goal structures, and multi-perspective feedback to support intentional and agentic writing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces WriteFlow, a voice-based AI writing assistant that frames interaction as a dialogic space for goal articulation, monitoring, and negotiation to support reflective academic writing. Drawing on a formative study (N=17) that identifies challenges in managing evolving goals and a Wizard-of-Oz study (N=12 expert users), it claims that the system scaffolds metacognitive regulation and reflection-in-action by enabling iterative goal refinement, maintaining goal-text alignment during drafting, and prompting evaluation of goal fulfillment, with associated design implications for AI writing tools.

Significance. If the empirical observations hold under more rigorous conditions, the work offers a meaningful contribution to HCI by contrasting efficiency-oriented AI tools with those emphasizing writer agency and metacognition. The emphasis on flexible goal structures and multi-perspective feedback provides concrete design directions that could inform more intentional writing support systems.

major comments (2)
  1. [Wizard-of-Oz Study] Wizard-of-Oz Study section: The central claim that WriteFlow scaffolds metacognitive regulation rests on qualitative findings from a sample of only 12 expert users in a simulated environment without reported quantitative metrics, baselines, or control conditions (e.g., comparison to standard non-dialogic AI tools). This setup leaves open whether observed benefits in iterative refinement and goal-text alignment arise specifically from the goal-oriented design or from the human-like responsiveness and participant expertise.
  2. [Findings] Findings section: Without controls or real AI deployment, the attribution of benefits to WriteFlow's features (iterative goal refinement, alignment maintenance, fulfillment evaluation) is not fully secured, as the simulated setup may not capture actual system behaviors such as response variability or error handling that could affect reflection-in-action.
minor comments (2)
  1. [Abstract] Abstract and Introduction: Clarify the exact criteria used to classify participants as 'expert users' (e.g., years of academic writing experience, disciplines) to help readers assess the scope of the reported patterns.
  2. [Discussion] Discussion: The design implications could be more explicitly linked back to specific observed interaction patterns from the study to strengthen the connection between findings and recommendations.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive and detailed feedback. We agree that the Wizard-of-Oz study is exploratory and qualitative, and we will revise the manuscript to more explicitly frame its scope, strengthen the limitations discussion, and use appropriately cautious language when attributing benefits to specific design features. Below we respond point by point to the major comments.

read point-by-point responses

  1. Referee: [Wizard-of-Oz Study] Wizard-of-Oz Study section: The central claim that WriteFlow scaffolds metacognitive regulation rests on qualitative findings from a sample of only 12 expert users in a simulated environment without reported quantitative metrics, baselines, or control conditions (e.g., comparison to standard non-dialogic AI tools). This setup leaves open whether observed benefits in iterative refinement and goal-text alignment arise specifically from the goal-oriented design or from the human-like responsiveness and participant expertise.

    Authors: We acknowledge the limitations of the sample size, qualitative approach, and absence of quantitative metrics or control conditions. This study was intentionally designed as a formative Wizard-of-Oz exploration to surface rich, expert-driven insights into goal-oriented voice interaction rather than to demonstrate causal efficacy or generalizability. The consistent patterns of iterative refinement and alignment we observed were linked by participants to the dialogic goal-negotiation structure. We will revise the Wizard-of-Oz Study section to (1) explicitly describe the work as formative, (2) note participant expertise as enabling identification of subtle metacognitive needs, and (3) add a dedicated limitations paragraph addressing the lack of baselines and the potential role of simulated responsiveness. We cannot add quantitative metrics or control conditions without a new controlled experiment, which we will flag as necessary future work. revision: partial

  2. Referee: [Findings] Findings section: Without controls or real AI deployment, the attribution of benefits to WriteFlow's features (iterative goal refinement, alignment maintenance, fulfillment evaluation) is not fully secured, as the simulated setup may not capture actual system behaviors such as response variability or error handling that could affect reflection-in-action.

    Authors: This is a fair observation. The Wizard-of-Oz protocol enabled focused testing of the intended interaction model while avoiding current AI limitations, but it cannot fully replicate response variability or error handling that real systems would introduce. We will revise the Findings section to replace definitive phrasing with more measured language (e.g., “observations suggest” and “participants reported”), explicitly discuss how real deployment might affect reflection-in-action, and present the design implications as directions for future systems rather than established outcomes. These textual clarifications will better reflect the study’s exploratory character. revision: partial

standing simulated objections not resolved
  • Introduction of quantitative metrics, baselines, or control conditions, which would require a separate controlled experiment not feasible within the current qualitative Wizard-of-Oz study.

Circularity Check

0 steps flagged

No circularity: empirical HCI study with independent user observations

full rationale

This paper reports qualitative findings from a formative study (17 participants) and a Wizard-of-Oz study (12 expert users) on an AI writing assistant. There are no equations, derivations, fitted parameters, or predictions that reduce by construction to inputs. Central claims about metacognitive scaffolding rest on direct observation of interaction patterns rather than self-definition, self-citation chains, or renamed known results. The work is self-contained against external benchmarks of user-study data and does not invoke uniqueness theorems or ansatzes from prior author work as load-bearing justification.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim depends on domain assumptions about writers' goal struggles and the value of dialogic AI support, drawn from cited prior research and the small formative study; no free parameters or invented entities are introduced.

axioms (2)
  • domain assumption Academic writers often struggle to articulate and manage changing goals during the writing process.
    Stated as motivation based on prior research and the formative study with 17 participants.
  • domain assumption Framing AI interaction as a dialogic space supports ongoing goal articulation, monitoring, and negotiation.
    Core design premise of WriteFlow presented without independent validation beyond the user studies.

pith-pipeline@v0.9.0 · 5452 in / 1473 out tokens · 47767 ms · 2026-05-10T08:00:46.298074+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages

  1. [1]

    Journal of Educa- tional Psychology103(2), 415 (2011)

    Beauvais, C., othersl: Why are some texts good and others not? relationship be- tween text quality and management of the writing processes. Journal of Educa- tional Psychology103(2), 415 (2011). https://doi.org/10.1037/a0022545

    work page doi:10.1037/a0022545 2011

  2. [2]

    Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges

    Becker,J.,Wahle,J.P.,Gipp,B.,Ruas,T.:Textgeneration:Asystematicliterature reviewoftasks,evaluation,andchallenges.arXivpreprintarXiv:2405.15604(2024). https://doi.org/10.48550/arXiv.2405.15604

    work page doi:10.48550/arxiv.2405.15604 2024

  3. [3]

    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tech- nologies6(1), 1–27 (2022)

    Bentvelzen, M., Woźniak, P.W., Herbes, P.S., Stefanidi, E., Niess, J.: Revisiting reflection in hci: Four design resources for technologies that support reflection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tech- nologies6(1), 1–27 (2022). https://doi.org/10.1145/3517233

    work page doi:10.1145/3517233 2022

  4. [4]

    Curriculum inquiry17(1), 9–30 (1987)

    Bereiter, C., Scardamalia, M.: An attainable version of high literacy: Approaches to teaching higher-order skills in reading and writing. Curriculum inquiry17(1), 9–30 (1987). https://doi.org/10.1080/03626784.1987.11075275

    work page doi:10.1080/03626784.1987.11075275 1987

  5. [5]

    Reading and Writing34(7), 1885–1913 (2021)

    Chung, H.Q., Chen, V., Olson, C.B.: The impact of self-assessment, planning and goal setting, and reflection before and after revision on student self-efficacy and writing performance. Reading and Writing34(7), 1885–1913 (2021). https://doi. org/10.1007/s11145-021-10186-x 14 Y. Fan et al

    work page doi:10.1007/s11145-021-10186-x 1913

  6. [6]

    In: Encyclopedia of critical psychology, pp

    Clarke, V., Braun, V.: Thematic analysis. In: Encyclopedia of critical psychology, pp. 1947–1952. Springer, New York, NY, USA (2014). https://doi.org/10.1007/ 978-1-4614-5583-7_311

    work page 1947

  7. [7]

    In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

    Dang, H., Benharrak, K., Lehmann, F., Buschek, D.: Beyond text generation: Sup- porting writers with continuous automatic text summaries. In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. pp. 1–13 (2022). https://doi.org/10.1145/3526113.3545672

    work page doi:10.1145/3526113.3545672 2022

  8. [8]

    the tone TONE

    E, J.L., et al.: When to give feedback: Exploring tradeoffs in the timing of design feedback. In: Proceedings of the 16th Conference on Creativity & Cognition. ACM, New York, NY, USA (2024). https://doi.org/10.1145/3635636.3656183

    work page doi:10.1145/3635636.3656183 2024

  9. [9]

    Beware of metacognitive lazi- ness: Effects of generative artificial intelligence on learning motivation, processes, and performance

    Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., Gašević, D.: Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology56(2), 489–530 (2025). https://doi.org/10.1111/bjet.13544

    work page doi:10.1111/bjet.13544 2025

  10. [10]

    College Composition & Communication32(4), 365–387 (1981)

    Flower, L., Hayes, J.R.: A cognitive process theory of writing. College Composition & Communication32(4), 365–387 (1981). https://doi.org/10.58680/ccc198115885

    work page doi:10.58680/ccc198115885 1981

  11. [11]

    Freeman, J.: Student generative AI survey 2025. Tech. rep., Higher Edu- cation Policy Institute, London, UK (2025), https://www.hepi.ac.uk/reports/ student-generative-ai-survey-2025/

    work page 2025

  12. [12]

    American psychologist 41(10), 1106 (1986)

    Hayes, J.R., Flower, L.S.: Writing research and the writer. American psychologist 41(10), 1106 (1986). https://doi.org/10.1037/0003-066X.41.10.1106

    work page doi:10.1037/0003-066x.41.10.1106 1986

  13. [13]

    Learning and Instruction89, 101847 (2024)

    Jansen, T., Meyer, J., Fleckenstein, J., Horbach, A., Keller, S., Möller, J.: Individ- ualizing goal-setting interventions using automated writing evaluation to support secondary school students’ text revisions. Learning and Instruction89, 101847 (2024). https://doi.org/10.1016/j.learninstruc.2023.101847

    work page doi:10.1016/j.learninstruc.2023.101847 2024

  14. [14]

    In: Proceedings of the 2023 ACM Designing Interactive Systems Conference

    Karolus, J., Feger, S.S., Schmidt, A., Woźniak, P.W.: Your text is hard to read: facilitating readability awareness to support writing proficiency in text production. In: Proceedings of the 2023 ACM Designing Interactive Systems Conference. pp. 147–160 (2023). https://doi.org/10.1145/3563657.3596052

    work page doi:10.1145/3563657.3596052 2023

  15. [15]

    In: Proceedings of the 15th In- ternational Learning Analytics and Knowledge Conference

    Li, T., et al.: Turning real-time analytics into adaptive scaffolds for self-regulated learning using generative artificial intelligence. In: Proceedings of the 15th In- ternational Learning Analytics and Knowledge Conference. pp. 667–679 (2025). https://doi.org/10.1145/3706468.3706559

    work page doi:10.1145/3706468.3706559 2025

  16. [16]

    Proceedings of the ACM on Human-Computer Interaction9(2), 1–32 (2025)

    Neshaei, S.P., et al.: Leveraging learner errors in digital argumentation learning: How ALure helps students learn from their mistakes and write better arguments. Proceedings of the ACM on Human-Computer Interaction9(2), 1–32 (2025). https: //doi.org/10.1145/3711023, https://dl.acm.org/doi/10.1145/3711023

    work page doi:10.1145/3711023 2025

  17. [17]

    International Journal of Higher Education3, 12–22 (09 2013)

    Pineteh, E.A.: The academic writing challenges of undergraduate students: A south african case study. International Journal of Higher Education3, 12–22 (09 2013). https://doi.org/10.5430/ijhe.v3n1p12

    work page doi:10.5430/ijhe.v3n1p12 2013

  18. [18]

    In: International Conference on Human- Computer Interaction

    Radwan, A.Y., Alasmari, K.M., Abdulbagi, O.A., Alghamdi, E.A.: Sard: A human-ai collaborative story generation. In: International Conference on Human- Computer Interaction. pp. 94–105. Springer (2024). https://doi.org/10.1007/ 978-3-031-76806-4_7

    work page 2024

  19. [19]

    Guilford Publications, New York, NY, USA, 2 edn

    Reid, R., et al.: Strategy instruction for students with learning disabilities. Guilford Publications, New York, NY, USA, 2 edn. (2013)

    work page 2013

  20. [20]

    SAGE Open9, 215824401989428 (07 2019)

    Sabti, A.A., Md Rashid, S., Nimehchisalem, V., Darmi, R.: The impact of writ- ing anxiety, writing achievement motivation, and writing self-efficacy on writing performance: A correlational study of iraqi tertiary efl learners. SAGE Open9, 215824401989428 (07 2019). https://doi.org/10.1177/2158244019894289 From Intention to Text: AI-Supported Goal Setting ...

    work page doi:10.1177/2158244019894289 2019

  21. [21]

    Schön.The Reflective Practitioner: How Professionals Think in Action

    Schön, D.A.: The reflective practitioner: How professionals think in action. Rout- ledge, London (2017). https://doi.org/10.4324/9781315237473

    work page doi:10.4324/9781315237473 2017

  22. [22]

    Educational psychologist25(1), 71–86 (1990)

    Schunk, D.H.: Goal setting and self-efficacy during self-regulated learn- ing. Educational psychologist25(1), 71–86 (1990). https://doi.org/10.1207/ s15326985ep2501_6

    work page 1990

  23. [23]

    In: Writing as a learning tool: Integrating theory and practice, pp

    Tynjälä,P.:Writing,learningandthedevelopmentofexpertiseinhighereducation. In: Writing as a learning tool: Integrating theory and practice, pp. 37–56. Springer, Dordrecht (2001). https://doi.org/10.1007/978-94-010-0740-5_4

    work page doi:10.1007/978-94-010-0740-5_4 2001

  24. [24]

    In: Proceedings of the tenth international conference on learning analytics & knowledge

    Viberg, O., Khalil, M., Baars, M.: Self-regulated learning and learning analytics in online learning environments: A review of empirical research. In: Proceedings of the tenth international conference on learning analytics & knowledge. pp. 524–533 (2020). https://doi.org/10.1145/3375462.3375483

    work page doi:10.1145/3375462.3375483 2020

  25. [25]

    In: LAK Workshops

    Wong, J., Viberg, O.: Supporting self-regulated learning with generative ai: A case of two empirical studies. In: LAK Workshops. pp. 223–229 (2024)

    work page 2024

  26. [26]

    Are large language models really good logical reasoners? a comprehensive evaluation and beyond

    Xu, F., et al.: Are large language models really good logical reasoners? a com- prehensive evaluation and beyond. IEEE Transactions on Knowledge and Data Engineering (2025). https://doi.org/10.1109/TKDE.2025.3536008

    work page doi:10.1109/tkde.2025.3536008 2025

  27. [27]

    In: Proceedings of the 15th international learning analytics and knowledge conference

    Yang, K., Raković, M., Liang, Z., Yan, L., Zeng, Z., Fan, Y., Gašević, D., Chen, G.: Modifying ai, enhancing essays: How active engagement with generative ai boosts writing quality. In: Proceedings of the 15th international learning analytics and knowledge conference. pp. 568–578 (2025). https://doi.org/10.1145/3706468. 3706544

    work page doi:10.1145/3706468 2025

  28. [28]

    Rzes- zotarski

    Zhang, C., et al.: Friction: Deciphering writing feedback into writing revisions through llm-assisted reflection. In: Proceedings of the 2025 CHI Conference on Hu- man Factors in Computing Systems. CHI ’25, ACM, New York, NY, USA (2025). https://doi.org/10.1145/3706598.3714316

    work page doi:10.1145/3706598.3714316 2025

  29. [29]

    Setlur, A

    Zhang, Z., Gao, J., Dhaliwal, R.S., Li, T.J.J.: Visar: A human-ai argumentative writing assistant with visual programming and rapid draft prototyping. In: Pro- ceedings of the 36th annual ACM symposium on user interface software and tech- nology. pp. 1–30 (2023). https://doi.org/10.1145/3586183.3606800

    work page doi:10.1145/3586183.3606800 2023

  30. [30]

    Zimmerman

    Zimmerman, B.J.: Becoming a self-regulated learner: An overview. Theory into practice41(2), 64–70 (2002). https://doi.org/10.1207/s15430421tip4102_2

    work page doi:10.1207/s15430421tip4102_2 2002

  31. [31]

    (eds.): Handbook of Self-Regulation of Learn- ing and Performance

    Zimmerman, B.J., Schunk, D.H. (eds.): Handbook of Self-Regulation of Learn- ing and Performance. Routledge, New York, NY (2011). https://doi.org/10.4324/ 9780203839010

    work page 2011