pith. sign in

arxiv: 2605.17263 · v1 · pith:OOLLWAIXnew · submitted 2026-05-17 · 💻 cs.HC

Expert Cognition Dashboard: From Learning Analytics to Cognition Intelligence in AI-Driven Education

Annie Yuan This is my paper

Pith reviewed 2026-05-19 23:15 UTC · model grok-4.3

classification 💻 cs.HC
keywords Expert Cognition DashboardLearning AnalyticsAI-driven EducationCognition IntelligenceAI TwinsAdaptive InterventionsCognitive StructuresDashboard Architecture
0
0 comments X

The pith

Expert Cognition Dashboards let AI systems interpret learner behaviors through expert-like cognitive structures rather than raw data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current AI education tools track clicks and scores but miss how experts actually read learner progress, spot misconceptions, and decide on next steps. The paper introduces the Expert Cognition Dashboard to fill that gap by turning student interactions into layered cognitive models that AI can reason over. Its architecture splits cognition into individual learner views, whole-class patterns, and an AI Twin expert layer that aggregates insights across groups. This setup lets AI Twins detect recurring difficulties, create targeted tasks, and deliver interventions grounded in modeled expert reasoning instead of surface metrics. The result is a proposed move from simple learning analytics to a fuller Cognition Intelligence where dashboards act as embedded reasoning tools for future AI-native classrooms.

Core claim

The Expert Cognition Dashboard models expert cognition inside dashboard systems so that learner behaviors are read through structures of interpretation, identity cognition, value recognition, misconception patterns, and learning tension. Its three-layer design—individual cognition dashboards, class cognition dashboards, and AI Twin expert dashboards—aggregates interactions for cross-group reasoning and adaptive intervention, redefining dashboards as cognitive middleware that connects raw behaviors to AI-driven expert decisions.

What carries the argument

The three-layer Expert Cognition Dashboard architecture that converts learner interactions into interpretable cognition structures for AI Twin analysis and intervention.

If this is right

  • AI Twins gain the ability to identify recurring learner difficulties from modeled misconception patterns rather than performance scores alone.
  • Adaptive tasks and personalised interventions can be generated directly from cognitive structures instead of raw interaction logs.
  • Dashboards function as cognitive middleware that links learner behaviors to expert reasoning across individual, class, and cross-group levels.
  • Education systems can shift from visualising data for humans to embedding expert reasoning as infrastructure for AI-native decision making.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Human instructors might use the same layered views to align their own judgments with the AI Twin outputs, reducing miscommunication in hybrid teaching settings.
  • Validation of the modeled cognition would require direct comparison against real expert educators' interpretations of the same learner data.
  • The framework could extend to other AI domains where dashboards need to encode domain-expert reasoning rather than just displaying metrics.

Load-bearing premise

Expert cognition including interpretation, misconception patterns, and learning tension can be captured and operationalized through the three-layer dashboard architecture to support AI adaptive interventions.

What would settle it

A side-by-side trial in which one AI education system uses the Expert Cognition Dashboard and another uses only behavioral analytics, then measures whether the first system identifies learner misconceptions and selects interventions more accurately than the second.

Figures

Figures reproduced from arXiv: 2605.17263 by Annie Yuan.

Figure 1
Figure 1. Figure 1: Paradigm shift from learning analytics to Cognition intelligence. Traditional learning analytics transforms [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Three-level architecture of the Expert Cognition Dashboard. Learner behaviours are interpreted through AI [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Prototype interface of the Expert Cognition Dashboard System. The interface demonstrates how cognition [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: AI Expert Feedback Ecology. The Expert Cognition Dashboard operates within a closed-loop educational [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Expert Cognition Dashboard as a cognition-centred infrastructure. The dashboard mediates between learner [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Personal transformation dashboard as an extension of cognition-centred infrastructure. The dashboard [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗
read the original abstract

Current AI-driven educational systems primarily rely on behavioural analytics, performance metrics, and content-level interactions to model learning. While these approaches provide useful indicators of learner activity, they are insufficient for representing the expert cognition used to interpret learner development, identify misconceptions, and make adaptive pedagogical decisions. Existing learning analytics dashboards largely visualise learner behaviour for human instructors, rather than embody expert cognition as a reasoning infrastructure for AI-native education. This paper introduces the Expert Cognition Dashboard (ECD), a cognition-centred reporting infrastructure for AI Twin-driven education systems. ECD models expert cognition within dashboard systems, enabling learner behaviours to be interpreted through expert-like cognitive structures rather than treated as raw behavioural signals. The proposed framework transforms student interactions into interpretable cognition structures through AI Tutor analysis and multi-level dashboard aggregation. Its architecture organises cognition across three layers: individual cognition dashboards, class cognition dashboards, and AI Twin expert dashboards for cross-group reasoning and adaptive intervention. Building on the AI Expert Feedback Ecology framework, ECD redefines dashboards as cognitive middleware that connects learner behaviours with AI-driven expert reasoning. By modelling interpretation, identity cognition, value recognition, misconception patterns, and learning tension, ECD enables AI Twins to identify recurring learner difficulties, generate adaptive tasks, and support personalised intervention. The paper argues for a shift from learning analytics toward Cognition Intelligence, positioning dashboards as foundational cognition infrastructures that embed expert reasoning into future AI-native education systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper proposes the Expert Cognition Dashboard (ECD) as a cognition-centred reporting infrastructure for AI Twin-driven education systems. It argues that current AI-driven educational systems rely on behavioural analytics and performance metrics that are insufficient to represent expert cognition for interpreting learner development, identifying misconceptions, and making adaptive decisions. ECD transforms student interactions into interpretable cognition structures (interpretation, identity cognition, value recognition, misconception patterns, and learning tension) via AI Tutor analysis, aggregated across three layers—individual cognition dashboards, class cognition dashboards, and AI Twin expert dashboards—to enable cross-group reasoning and personalised interventions, shifting the field from learning analytics toward Cognition Intelligence.

Significance. If the framework can be operationalized with concrete mechanisms and validation, it could advance HCI and AI in education by repositioning dashboards as cognitive middleware that embeds expert reasoning directly into AI-native systems. This builds explicitly on the AI Expert Feedback Ecology framework and offers a conceptual foundation for moving beyond raw behavioural signals to structured cognitive models that support adaptive tasks and intervention.

major comments (1)
  1. [Abstract and framework architecture description] The three-layer architecture (individual cognition dashboards, class cognition dashboards, and AI Twin expert dashboards) is described at the conceptual level in the abstract and framework sections, but the manuscript supplies neither data schemas, transformation rules, nor worked examples showing how a concrete interaction trace is mapped by the AI Tutor to modeled elements such as misconception patterns or learning tension values. This is load-bearing for the central claim that ECD operationalizes expert cognition rather than merely relabeling analytics.
minor comments (1)
  1. [Abstract] The abstract is information-dense; breaking the description of the three layers and their intended functions into separate sentences would improve readability.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thorough review and valuable suggestions. We agree that providing more concrete details on the operationalization of the Expert Cognition Dashboard is essential to substantiate our claims. Below, we address the major comment point by point and outline the revisions we will make to the manuscript.

read point-by-point responses

  1. Referee: The three-layer architecture (individual cognition dashboards, class cognition dashboards, and AI Twin expert dashboards) is described at the conceptual level in the abstract and framework sections, but the manuscript supplies neither data schemas, transformation rules, nor worked examples showing how a concrete interaction trace is mapped by the AI Tutor to modeled elements such as misconception patterns or learning tension values. This is load-bearing for the central claim that ECD operationalizes expert cognition rather than merely relabeling analytics.

    Authors: We appreciate this observation and agree that the manuscript would benefit from greater specificity in demonstrating the mapping process. In the revised version, we will expand the framework architecture section to include: (1) explicit data schemas defining the structure of each cognition element (e.g., fields for misconception patterns including type, frequency, and associated learner behaviors); (2) transformation rules outlining how the AI Tutor analyzes interaction traces, such as rule-based or model-driven mappings from raw logs to interpretation and learning tension scores; and (3) a detailed worked example tracing a sample student interaction sequence through the AI Tutor analysis to the resulting cognition structures in the individual dashboard, with aggregation steps to class and AI Twin levels. These additions will provide concrete evidence that ECD embeds expert reasoning mechanisms rather than simply relabeling existing analytics. We believe this addresses the concern directly while maintaining the conceptual focus of the paper. revision: yes

Circularity Check

0 steps flagged

Conceptual framework paper with no derivations, equations or fitted predictions

full rationale

The manuscript introduces the Expert Cognition Dashboard as a high-level architectural proposal for AI-driven education. It describes three layers (individual, class, AI Twin) and concepts such as interpretation, misconception patterns and learning tension, but supplies no equations, no parameter fitting, no predictive claims, and no transformation rules that could reduce outputs to inputs by construction. The text references building on the AI Expert Feedback Ecology framework at a conceptual level without invoking uniqueness theorems, self-citations as load-bearing premises, or ansatzes smuggled from prior work. No self-definitional loops or renaming of known results appear. This is a standard non-finding for a purely descriptive framework paper whose central claim remains unevaluated rather than circular.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claim rests on domain assumptions about the feasibility of modeling expert cognition from behavioral data and introduces new conceptual entities without independent evidence or validation.

axioms (1)
  • domain assumption Expert cognition can be modeled as interpretable structures (interpretation, identity cognition, value recognition, misconception patterns, learning tension) from learner behaviors
    Invoked as the foundation for transforming interactions into cognition structures via AI Tutor analysis.
invented entities (2)
  • Expert Cognition Dashboard (ECD) no independent evidence
    purpose: Cognitive middleware connecting learner behaviors with AI-driven expert reasoning
    Introduced as the core new infrastructure for AI Twin-driven education systems.
  • AI Twin expert dashboards no independent evidence
    purpose: Enable cross-group reasoning and adaptive intervention
    Part of the three-layer architecture for class and cross-group cognition.

pith-pipeline@v0.9.0 · 5772 in / 1256 out tokens · 39302 ms · 2026-05-19T23:15:03.715697+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages

  1. [1]

    Learning and individual differences , volume=

    ChatGPT for good? On opportunities and challenges of large language models for education , author=. Learning and individual differences , volume=. 2023 , publisher=

    work page 2023

  2. [2]

    Complexity , volume=

    A Review of Artificial Intelligence (AI) in Education from 2010 to 2020 , author=. Complexity , volume=. 2021 , publisher=

    work page 2010

  3. [3]

    Proceedings of the 2nd international conference on learning analytics and knowledge , pages=

    Learning analytics and educational data mining: towards communication and collaboration , author=. Proceedings of the 2nd international conference on learning analytics and knowledge , pages=

    work page

  4. [4]

    International journal of technology enhanced learning , volume=

    Learning analytics: drivers, developments and challenges , author=. International journal of technology enhanced learning , volume=. 2012 , publisher=

    work page 2012

  5. [5]

    TechTrends , volume=

    Let’s not forget: Learning analytics are about learning , author=. TechTrends , volume=. 2015 , publisher=

    work page 2015

  6. [6]

    Journal of educational data mining , volume=

    The state of educational data mining in 2009: A review and future visions , author=. Journal of educational data mining , volume=

    work page 2009

  7. [7]

    Proceedings of the 1st international conference on learning analytics and knowledge , pages=

    Attention please! Learning analytics for visualization and recommendation , author=. Proceedings of the 1st international conference on learning analytics and knowledge , pages=

    work page

  8. [8]

    Personal and Ubiquitous Computing , volume=

    Learning dashboards: an overview and future research opportunities , author=. Personal and Ubiquitous Computing , volume=. 2014 , publisher=

    work page 2014

  9. [9]

    Journal of educational technology & society , volume=

    Social learning analytics , author=. Journal of educational technology & society , volume=. 2012 , publisher=

    work page 2012

  10. [10]

    Proceedings of the 8th international conference on learning analytics and knowledge , pages=

    Open learner models and learning analytics dashboards: a systematic review , author=. Proceedings of the 8th international conference on learning analytics and knowledge , pages=

    work page

  11. [11]

    The journal of the learning sciences , volume=

    Cognitive tutors: Lessons learned , author=. The journal of the learning sciences , volume=. 1995 , publisher=

    work page 1995

  12. [12]

    International journal of artificial intelligence in education , volume=

    The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching , author=. International journal of artificial intelligence in education , volume=. 2014 , publisher=

    work page 2014

  13. [13]

    ACM Transactions on Interactive Intelligent Systems (TiiS) , volume=

    AutoTutor and affective AutoTutor: Learning by talking with cognitively and emotionally intelligent computers that talk back , author=. ACM Transactions on Interactive Intelligent Systems (TiiS) , volume=. 2013 , publisher=

    work page 2013

  14. [14]

    Advances in intelligent tutoring systems , pages=

    Rule-based cognitive modeling for intelligent tutoring systems , author=. Advances in intelligent tutoring systems , pages=. 2010 , publisher=

    work page 2010

  15. [15]

    Educational psychologist , volume=

    The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems , author=. Educational psychologist , volume=. 2011 , publisher=

    work page 2011

  16. [16]

    User Modeling and User-Adapted Interaction , volume=

    A review of recent advances in learner and skill modeling in intelligent learning environments , author=. User Modeling and User-Adapted Interaction , volume=. 2012 , publisher=

    work page 2012

  17. [17]

    Advances in intelligent tutoring systems , pages=

    Open learner models , author=. Advances in intelligent tutoring systems , pages=. 2010 , publisher=

    work page 2010

  18. [18]

    Psychonomic bulletin & review , volume=

    Cognitive tutor: Applied research in mathematics education , author=. Psychonomic bulletin & review , volume=. 2007 , publisher=

    work page 2007

  19. [19]

    International Conference on Artificial Intelligence in Education , pages=

    Enhancing llm-based feedback: Insights from intelligent tutoring systems and the learning sciences , author=. International Conference on Artificial Intelligence in Education , pages=. 2024 , organization=

    work page 2024

  20. [20]

    International Conference on Artificial Intelligence in Education , pages=

    OLiMent: Conversations About Open Learner Modelling to Help Learners Understand and Self-assess Learning Goals , author=. International Conference on Artificial Intelligence in Education , pages=. 2025 , organization=

    work page 2025

  21. [21]

    2022 , publisher=

    Human-centered AI , author=. 2022 , publisher=

    work page 2022

  22. [22]

    Artificial intelligence , volume=

    Explanation in artificial intelligence: Insights from the social sciences , author=. Artificial intelligence , volume=. 2019 , publisher=

    work page 2019

  23. [23]

    interactions , volume=

    Value-sensitive design , author=. interactions , volume=. 1996 , publisher=

    work page 1996

  24. [24]

    2007 , publisher=

    Human-machine reconfigurations: Plans and situated actions , author=. 2007 , publisher=

    work page 2007

  25. [25]

    Proceedings of the SIGCHI conference on Human Factors in Computing Systems , pages=

    Principles of mixed-initiative user interfaces , author=. Proceedings of the SIGCHI conference on Human Factors in Computing Systems , pages=

    work page

  26. [26]

    Proceedings of the 2019 chi conference on human factors in computing systems , pages=

    Guidelines for human-AI interaction , author=. Proceedings of the 2019 chi conference on human factors in computing systems , pages=

    work page 2019

  27. [27]

    1991 , publisher=

    Situated learning: Legitimate peripheral participation , author=. 1991 , publisher=

    work page 1991

  28. [28]

    Educational researcher , volume=

    Situated cognition and the culture of learning , author=. Educational researcher , volume=. 1989 , publisher=

    work page 1989

  29. [29]

    1986 , publisher=

    Mind over machine , author=. 1986 , publisher=

    work page 1986

  30. [30]

    2017 , publisher=

    Sources of power: How people make decisions , author=. 2017 , publisher=

    work page 2017

  31. [31]

    2018 , publisher=

    The Cambridge handbook of expertise and expert performance , author=. 2018 , publisher=

    work page 2018

  32. [32]

    2017 , publisher=

    The reflective practitioner: How professionals think in action , author=. 2017 , publisher=

    work page 2017

  33. [33]

    AI magazine , volume=

    The problem of extracting the knowledge of experts from the perspective of experimental psychology , author=. AI magazine , volume=

    work page

  34. [34]

    1994 , publisher=

    Unified theories of cognition , author=. 1994 , publisher=

    work page 1994

  35. [35]

    ACM Computing Surveys (CSUR) , volume=

    Imitation learning: A survey of learning methods , author=. ACM Computing Surveys (CSUR) , volume=. 2017 , publisher=

    work page 2017

  36. [36]

    1998 , publisher=

    Reinforcement learning: An introduction , author=. 1998 , publisher=

    work page 1998

  37. [37]

    Advances in neural information processing systems , volume=

    Deep reinforcement learning from human preferences , author=. Advances in neural information processing systems , volume=

    work page

  38. [38]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

    work page

  39. [39]

    Decision and organization , volume=

    Theories of bounded rationality , author=. Decision and organization , volume=. 1972 , publisher=

    work page 1972

  40. [40]

    1987 , publisher=

    Plans and situated actions: The problem of human-machine communication , author=. 1987 , publisher=

    work page 1987

  41. [41]

    ACM Transactions on Computer-Human Interaction (TOCHI) , volume=

    Distributed cognition: toward a new foundation for human-computer interaction research , author=. ACM Transactions on Computer-Human Interaction (TOCHI) , volume=. 2000 , publisher=

    work page 2000

  42. [42]

    Philosophy , volume=

    The logic of tacit inference , author=. Philosophy , volume=. 1966 , publisher=

    work page 1966

  43. [43]

    Handbook of intuition research , year=

    Expert intuition and naturalistic decision making , author=. Handbook of intuition research , year=

    work page

  44. [44]

    Harvard business review , volume=

    The knowledge-creating company , author=. Harvard business review , volume=

    work page

  45. [45]

    2001 , publisher=

    The practice turn in contemporary theory , author=. 2001 , publisher=

    work page 2001

  46. [46]

    , author=

    Identities and interactions. , author=. 1966 , publisher=

    work page 1966

  47. [47]

    Handbook of sociological theory , pages=

    Traditional symbolic interactionism, role theory, and structural symbolic interactionism: The road to identity theory , author=. Handbook of sociological theory , pages=. 2001 , publisher=

    work page 2001

  48. [48]

    , author=

    The dynamic self-concept: A social psychological perspective. , author=. Annual review of psychology , year=

    work page

  49. [49]

    Journal of Consumer Psychology , volume=

    Identity-based motivation: Implications for action-readiness, procedural-readiness, and consumer behavior , author=. Journal of Consumer Psychology , volume=. 2009 , publisher=

    work page 2009

  50. [50]

    Advances in experimental social psychology , volume=

    Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries , author=. Advances in experimental social psychology , volume=. 1992 , publisher=

    work page 1992

  51. [51]

    Early engagement and new technologies: Opening up the laboratory , pages=

    Value sensitive design and information systems , author=. Early engagement and new technologies: Opening up the laboratory , pages=. 2013 , publisher=

    work page 2013

  52. [52]

    Foundations and Trends

    A survey of value sensitive design methods , author=. Foundations and Trends. 2017 , publisher=

    work page 2017

  53. [53]

    Proceedings of the 31st International Conference on Intelligent User Interfaces , pages=

    Mental Models in Human-AI Interaction: Systematic Review of Empirical Methodologies and Guidelines , author=. Proceedings of the 31st International Conference on Intelligent User Interfaces , pages=

    work page

  54. [54]

    Extended abstracts of the 2023 CHI conference on human factors in computing systems , pages=

    Unlocking the tacit knowledge of data work in machine learning , author=. Extended abstracts of the 2023 CHI conference on human factors in computing systems , pages=

    work page 2023