Pith. sign in

REVIEW 5 major objections 5 minor 91 references

AI-Driven Contribution Evaluation and Conflict Resolution: A Framework & Design for Group Workload Investigation

T0 review · 5 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes a design for an AI-assisted tool that organizes group-work evidence into nine benchmarks, flags workload inequality with a Gini index, and produces advisory judgments through a validated LLM analysis — all within current

desk verdict A thoughtful but unvalidated design for an AI-assisted group-work dispute investigation tool; the framework and tool survey are useful, and the LLM 'expert analysis' stage is the unsupported hinge. read the letter →

arxiv 2511.07667 v2 pith:IQBPVKS7 submitted 2025-11-10 cs.AI

Jakub Slapek , Mir Seyedebrahimi , Jianhua Yang This is my paper
classification cs.AI
keywords AI-assistedassessmentcontributionevaluationconflictresolutiongroupworkpeerlearninganalyticsGiniindexlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Equitable assessment of individual contribution in team projects is a persistent, labor-intensive problem, and existing peer-assessment tools do little to help instructors investigate disputes. This paper proposes a framework and implementation design for an AI-enhanced system that does: it sorts heterogeneous evidence — submissions, messages, meeting logs, peer reviews — into three dimensions (Contribution, Interaction, Role) with nine benchmarks, computes objective metrics from each, normalizes and aggregates them, and uses the Gini index to flag unequal workloads. The aggregated measures feed an LLM stage that produces interpretable, advisory judgments about what happened in the team, backed by evidence and a double-pass validation. The authors argue the design is feasible under current UK and EU data-protection and AI rules, provided the tool stays advisory, transparent, and compliant with the required impact assessments. A sympathetic reader would care because the framework directly targets the gap between automated peer scoring and the manual, costly dispute adjudication instructors currently face.

What carries the argument

The load-bearing object is the three-dimension/nine-benchmark conflict framework (Contribution, Interaction, Role), which labels what evidence means. The Gini index computed on each base measure provides the conflict-marker mechanism — pairing inequality with individual scores to distinguish over-centralisation from social loafing. The LLM expert-analysis stage, using hierarchical prompts and double-pass validation, is the machinery that converts metrics into an interpretable advisory judgment. The final piece is the AutoRating-style normalisation used to make heterogeneous metrics comparable.

What would settle it

Take a corpus of group projects with known ground-truth individual contributions (e.g., version-control history plus peer ratings) and run the proposed pipeline end-to-end; compare the LLM's advisory judgments against a panel of experienced instructors adjudicating the same evidence. If the LLM systematically misclassifies conflict scenarios, reflects bias toward particular communication styles, or produces judgments that the panel cannot reproduce, the paper's reliability claim fails.

Watch

Extended reading notes

Core claim

The central claim is that conflict over individual contribution can be investigated systematically rather than adjudicated by intuition: the paper defines a three-dimensional, nine-benchmark taxonomy of evidence (Contribution: quantity, quality, relevance; Interaction: tone, effectiveness, presence; Role: adherence, organisation, support), with concrete metrics per dimension. These metrics are normalised and aggregated into three objective measures, and the Gini index is computed per base measure to surface conflict markers — e.g., high Gini with a standout score signals over-centralisation, low Gini with a low score signals social loafing. An LLM then performs hierarchical, decomposed reaso

Load-bearing premise

The entire advisory-judgment stage rests on the unverified assumption that an LLM, with only a double-pass validation, can reliably read the computed metrics and produce accurate, unbiased, hallucination-free judgments — an assumption the paper does not test.

Editorial extensions

If this is right

  • Instructors receive objective metrics and conflict flags before or during a dispute, so investigation starts from organised evidence rather than anecdotes.
  • The tool's advisory stance means it can operate as profiling under UK GDPR, not automated decision-making, so students retain the right to human review.
  • Because the framework is implementation-agnostic, tool developers can instantiate the nine benchmarks with whatever evidence is available, scaling investigative power with data.
  • Existing platforms can adopt the design to close the gap the survey identifies: none offer AI-assisted conflict investigation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's weakest point is the untested LLM stage; before any deployment, the double-pass validation would need to be benchmarked against human expert adjudication on real group-work data — a study the authors do not provide.
  • If the LLM stage proves reliable, the same three-dimension/nine-benchmark architecture could generalise beyond education to workplace team evaluation, where similar contribution disputes arise under different legal regimes.
  • A natural testable extension is to run the pipeline on archival project data (e.g., public Git repos with team metadata) to see whether Gini-based conflict markers predict actual team breakdowns or instructor interventions as recorded.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a framework and implementation design for an AI-assisted tool intended to help instructors investigate disputes over individual contribution in group work. It surveys 11 existing contribution-assessment tools, identifies a gap in conflict-resolution methods and AI integration, and introduces three evaluation dimensions (Contribution, Interaction, Role) subdivided into nine benchmarks. The proposed pipeline extracts metrics from submissions, communications, and coordination records; normalises and aggregates them; computes Gini-based inequality and conflict markers; and uses a hierarchical LLM 'expert analysis' stage with 'double-pass validation' to produce advisory judgments. The authors argue that the design is feasible under UK and University of Warwick data-protection and AI policies provided consent, DPIA/IDG approval, and an advisory stance are maintained. The paper is a design/feasibility argument with no implementation or empirical evaluation; the authors explicitly acknowledge several accuracy-limiting cases in §5.6.

Significance. If operationalised and validated, the framework would address a genuine gap: existing peer-assessment tools flag conflict but do not assist with evidence-based investigation, and the proposed taxonomy of heterogeneous artefacts into nine conflict-related benchmarks is a useful conceptual contribution. The survey of 11 tools and the policy analysis are practical and grounded, and the authors are candid about limitations such as gaming, missing evidence, and cultural/linguistic differences. However, the paper's central claims go beyond what is demonstrated: the LLM expert-analysis stage is not validated, the 'objective' metrics partly depend on the same LLM, and the feasibility conclusion is anchored to a single institution's policies. The strengths are the structured evidence taxonomy, the conflict-marker design, and the explicit limitations section; the weaknesses are the absence of a prototype, error analysis, or reproducibility materials.

major comments (5)
  1. [§5.4 / Abstract] The abstract and §7 state that the LLM stage 'performs validated and contextual analysis' and that the architecture 'enhances reliability', but no validation is reported. 'Double-pass validation' is described only by name: no protocol, accuracy, calibration, bias, or hallucination measurements are provided, and §5.4 itself defers to 'further research' for prompt engineering and fine-tuning. Since the final advisory judgment is the load-bearing output an instructor would act on, this is a central unsupported claim. The paper needs either an implementation study with error analysis or a precise re-scoping of the wording from 'validated' to 'proposed'.
  2. [§5.1–5.2] The aggregation chain treats the resulting measures as 'objective', but several are produced by LLM extraction/grading: task fidelity, assignment fidelity, relevance via hypothetical documents, and subjective quality evaluation. Errors or biases in these extraction steps propagate into the base measures and conflict markers, and then into the same LLM's final judgment in §5.4. The paper does not discuss error propagation, validation of the extraction prompts, or agreement between deterministic and LLM-produced metrics. This matters because the design's objectivity claim depends on these metrics being trustworthy.
  3. [§5.2] The weighted mask for aggregation is described as 'adjustable to emphasise different aspects of performance', but the paper gives no default weights, no sensitivity analysis, and no guidance for instructors. The same applies to normalisation thresholds and to the conflict-marker rules in §5.3, where 'high Gini' and 'above average individual score' are not operationally defined. Because the final objective measures and Gini markers depend on these choices, the framework's outputs are underdetermined. At minimum, the authors should provide default configurations and a sensitivity-testing protocol.
  4. [§6 / §7] The conclusion claims feasibility 'under current statutory and institutional policy', but the policy review is limited to UK law and University of Warwick/WMG policies, and the paper itself notes Warwick prohibits uploading student data to AI/online platforms without consent. The design does not include a consent-management or data-minimisation mechanism that would operationalise these constraints. The feasibility conclusion should be explicitly scoped to the reviewed jurisdiction, and the consent requirement should be treated as a design component rather than an external assumption.
  5. [§5.6] The limitations section lists accuracy-reducing cases (missing evidence, gaming strategies, large cultural/linguistic/personal differences) and then asserts these 'won't significantly affect feasibility' without argument or evidence. Gaming is directly in scope for a dispute-investigation tool, so the effect of adversarial or misleading input on metrics and the LLM judgment needs to be analysed or at least treated as an open risk rather than dismissed. This is a load-bearing issue for the central feasibility claim.
minor comments (5)
  1. [§4, Table 2] The word 'signiy' appears in the Conflicts column; should be 'signal'.
  2. [Table 5] The Conversation group contains 'g. Code Standard', which appears to be a copy/paste error from the Submission group.
  3. [Figure 8] The caption reads 'Feauters in contribution assessment tools' and does not match the surrounding text on subjective/contextual data.
  4. [Abstract and §5.6] 'sentimental' should be 'sentiment' (e.g., 'sentiment analysis').
  5. [§6] The sentence 'Given that our team operates outside of the University of Warwick and WMG' is confusing; if the authors are outside Warwick, the focus on Warwick policies needs a clearer rationale.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a framework/design proposal; the unvalidated LLM stage is a correctness risk, not a derivation-level circularity.

full rationale

The paper does not derive predictions from fitted inputs. Its metrics (Submission, Conversation, Coordination) are computed from raw evidence via deterministic counts, version-control logs, and LLM-assisted extraction, then normalised by AutoRating and aggregated with an adjustable but not data-fitted weighted mask. No parameter is calibrated to an outcome and then used to 'predict' that same outcome. The Gini-based conflict markers are rule-based scenario mappings (high/low Gini against above/below-average scores) and are not defined in terms of the final LLM judgment. The final Expert Analysis stage (§5.4) uses hierarchical prompts over the same metrics, but the paper explicitly defers validation ('This stage could benefit from further research into maximising LLM effectiveness ... to fine-tune judgments'), so the LLM's reliability is an unverified assumption, not a circular construction. Feasibility under policy is argued from external institutional and statutory sources (Warwick policies, UK GDPR, EU AI Act), and the references to the authors' own institution are policy citations, not load-bearing self-citations to their own prior results. The framework's benchmarks are explicitly inspired by existing instruments such as CATME rather than being renamed as independent derivations. The main weakness—lack of evidence for LLM accuracy, bias, or hallucination—is an empirical and methodological validity gap, which falls under correctness risk rather than circularity. Accordingly, no circular step meets the evidentiary standard of reducing a claimed result to its inputs by definition or by self-citation chain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework rests on the validity of proxy metrics, Gini-based conflict markers, and LLM reliability, plus a feasibility claim based on a narrow policy review. Several weights and factors are adjustable without guidance, though none are fitted to data.

free parameters (3)
  • Weighted mask for aggregation
    In §5.2, each category's metrics are aggregated with a weighted mask that sums to 1, described as 'adjustable to emphasise different aspects of performance'. No values or validation are provided; the final objective measures depend on these weights.
  • Contextual adjustment factor
    In §5.5, past grades and personal circumstances are converted to numeric values and 'normalized and aggregated into an adjustment factor that directly affects the students’ final objective measures'. The conversion and normalization are unspecified.
  • Normalization thresholds and scaling for metrics
    Metrics such as code line count, word count, response time, etc. are normalized via AutoRating, but the scaling, handling of outliers, and treatment of zero baselines are not specified.
assumptions (4)
  • domain assumption Objective metrics (line count, word count, message count, sentiment, etc.) are valid proxies for contribution and conflict.
    The entire metrics stage (§5.1) assumes these quantitative signals reflect individual contribution; no validation against ground truth or instructor judgment is provided.
  • domain assumption Gini index computed on base measures surfaces meaningful conflict markers.
    §5.3 uses Gini inequality to flag conflicts; the mapping from Gini values to conflict scenarios A/B is asserted without empirical support.
  • domain assumption LLM-based feature extraction and judgment with 'double-pass validation' yields reliable and unbiased evaluations.
    The expert analysis stage (§5.4) relies on LLMs to infer situations and produce advisory judgments; the paper acknowledges hallucination risk but provides no evaluation or benchmark.
  • domain assumption The policy review of UK and University of Warwick generalizes to other institutions and jurisdictions.
    §6 argues feasibility under current statutory and institutional policy, but only examines UK and Warwick, with 'brief coverage elsewhere'; broader generalizability is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI-Driven Contribution Evaluation and Conflict Resolution: A Framework & Design for Group Workload Investigation." pith.science (2026). https://pith.science/paper/IQBPVKS7

@misc{pith2026251107667,
  author       = {Pith},
  title        = {Pith review of: AI-Driven Contribution Evaluation and Conflict Resolution: A Framework & Design for Group Workload Investigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IQBPVKS7}},
  note         = {Machine review of arXiv:2511.07667}
}
read the original abstract

The equitable assessment of individual contribution in teams remains a persistent challenge, where conflict and disparity in workload can result in unfair performance evaluation, often requiring manual intervention - a costly and challenging process. We survey existing tool features and identify a gap in conflict resolution methods and AI integration. To address this, we propose a framework and implementation design for a novel AI-enhanced tool that assists in dispute investigation. The framework organises heterogeneous artefacts - submissions (code, text, media), communications (chat, email), coordination records (meeting logs, tasks), peer assessments, and contextual information - into three dimensions with nine benchmarks: Contribution, Interaction, and Role. Objective measures are normalised, aggregated per dimension, and paired with inequality measures (Gini index) to surface conflict markers. A Large Language Model (LLM) architecture performs validated and contextual analysis over these measures to generate interpretable and transparent advisory judgments. We argue for feasibility under current statutory and institutional policy, and outline practical analytics (sentimental, task fidelity, word/line count, etc.), bias safeguards, limitations, and practical challenges.

Figures

Figures reproduced from arXiv: 2511.07667 by the authors.

Figure 1
Figure 1. Bar graph of feature adoption in contribution assessment tools. Of the tools listed above, the majority (91%) used an AutoRating-derived normalisation to determine scores based on peer evaluations, naming the metric “relative performance factor” or “group factor”. Variations include square-rooting the weight factor (FeedbackFruits), and constant sum allocation (iPeer). Additionally, 82% constructed rubrics based on … view at source ↗
Figure 2
Figure 2. High-level architecture overview. As seen above, the system accepts evidence to produce metrics and conflict markers that act as the foundation for feature inference and expert analysis. Optionally, a contextual stage can be integrated to affect calculations and conclusions. We discuss each stage in more detail below. [5.1] Metrics Our base design calculates analytics based on data collected from (1) submitted work … view at source ↗
Figure 3
Figure 3. Deriving abstract metrics via LLMs. Coordination. As displayed above, we use LLMs to extract goals, tasks and work summaries from the task description, meeting logs, and contributions to compute task fidelity (the rele￾vance of meeting outcomes to the original project goals), task diversity (the spread of tasks), and assignment fidelity (whether assigned tasks were completed). Presence is gauged from attendance and … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Compiling metrics into base measures. We group the initial input metrics according to the category they align with in the conflict framework to derive objective metrics from collected evidence. In every category, each metric is normalized via AutoRating, and then aggre…
Figure 5
Figure 5. Figure 5: Base measures with semantic and hypothetical embedding. AI graders improve with access to a rubric and examples of grading[86]. Thus, we produce an assessment guide to help with subjective quality evaluation performed by an AI grader for each 9 [PITH_FULL_IMAGE:figure…
Figure 6
Figure 6. Figure 6: Compiling objective measures from base measures. Finally, all the base measures are aggregated into three holistic objective measures. We posit that having this select number of values prevents the instructor from being overwhelmed by statistics, whilst still conveying…
Figure 7
Figure 7. Figure 7: Generating expert opinion via LLMs. This stage could benefit from further research into maximising LLM effectiveness (prompt engi￾neering, Chain-of-Thought, etc.) to fine-tune judgments and improve summarisation. Further, validation passes and filtering should maintain…
Figure 8
Figure 8. Figure 8: Feauters in contribution assessment tools. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

91 extracted references · 4 linked inside Pith

  1. [1]

    Arthars, K

    N. Arthars, K. Thompson, H. Huijser, S. Kickbusch, S. Cunningham, G. Winter, R. Cook, and L. Lockyer. Formative assessment of group work skills: An analytics-enabled concep- tual framework.Australasian Journal of Educational Technology, 40(4):138–153, 2024

    2024

  2. [2]

    T. Bisoux. Harnessing AI to assess team-based performance. AACSB Insights, Jan. 2025. Accessed: 10 November 2025

    2025

  3. [3]

    D. Boud, R. Cohen, and J. Sampson, editors.Peer Learning in Higher Education: Learning from and with Each Other. Routledge, London, 1 edition, 2001

    2001

  4. [4]

    Brindley and S

    C. Brindley and S. Scoffield. Peer assessment in undergraduate programmes.Teaching in Higher Education, 3(1):79–90, 1998

    1998

  5. [5]

    R. W. Brown. Autorating: Getting individual marks from team marks and enhancing teamwork. InProceedings of the 25th Annual Frontiers in Education Conference – En- gineering Education for the 21st Century, pages 3c2.15–3c2.18, Atlanta, GA, USA, Nov

  6. [6]

    Buddycheck: Peer evaluation tool, 2025

    BuddyCheck. Buddycheck: Peer evaluation tool, 2025. Accessed 24 July 2025

    2025

  7. [7]

    Buffardi

    K. Buffardi. Assessing individual contributions to software engineering projects with git logsanduserstories. InProceedings of the 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE), pages 650–656. IEEE/ACM, 2020

    2020

  8. [8]

    Cho and C

    K. Cho and C. D. Schunn. Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system.Computers & Education, 48(3):409–426, Apr. 2007. Source: DBLP

    2007

Show all 91 references
  1. [9]

    IncorporatingAIandlearninganalytics to build trustworthy peer assessment systems.British Journal of Educational Technology, 53(3):429–446, May 2022

    A.Darvishi, H.Khosravi, S.Sadiq, andD.Gašević. IncorporatingAIandlearninganalytics to build trustworthy peer assessment systems.British Journal of Educational Technology, 53(3):429–446, May 2022

    2022

  2. [10]

    Generative artificial intelligence (AI) in education

    Department for Education. Generative artificial intelligence (AI) in education. Policy paper, HM Government, London, Mar. 2023. First published 29 March 2023; last updated 10 June 2025

    2023

  3. [11]

    Apro-innovationapproachtoAIregula- tion

    DepartmentforScience, Innovation&Technology. Apro-innovationapproachtoAIregula- tion. Policy paper (White Paper) CP815, HM Government, London, Mar. 2023. Presented to Parliament 29 March 2023; last updated 3 August 2023

    2023

  4. [12]

    Dougiamas

    M. Dougiamas. Moodle learning management system.https://moodle.org, 2025. Open-source software; accessed 2025-07-30

    2025

  5. [13]

    Dwork, M

    C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. S. Zemel. Fairness through awareness. arXiv preprint arXiv:1104.3913, 2011

    2011 arXiv

  6. [14]

    Eduflow: Social & collaborative learning platform, 2025

    EduFlow. Eduflow: Social & collaborative learning platform, 2025. Accessed 24 July 2025

    2025

  7. [15]

    AI act: Regulatory framework on artificial intelligence.https:// digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, 2025

    European Commission. AI act: Regulatory framework on artificial intelligence.https:// digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, 2025. Ac- cessed 2025-07-06

    2025

  8. [16]

    EU AI Act: First Regulation on Artificial Intelligence

    European Parliament. EU AI Act: First Regulation on Artificial Intelligence. European Parliament, 2023. Last updated 2025-02-19. 14

    2023

  9. [17]

    Fairlearn: A python toolkit to assess and improve fairness in ai

    Fairlearn contributors. Fairlearn: A python toolkit to assess and improve fairness in ai. https://fairlearn.org. Python package

  10. [18]

    Falchikov.Improving Assessment Through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education

    N. Falchikov.Improving Assessment Through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education. RoutledgeFalmer, London, 1 edition, 2005

    2005

  11. [19]

    Falchikov and J

    N. Falchikov and J. Goldfinch. Student peer assessment in higher education: A meta- analysis comparing peer and teacher marks.Review of Educational Research, 70(3):287– 322, 2000

    2000

  12. [20]

    Feedbackfruits: Assessment & engagement tools for education, 2025

    FeedbackFruits. Feedbackfruits: Assessment & engagement tools for education, 2025. Accessed 24 July 2025

    2025

  13. [21]

    S. Feng, D. C. Gibson, and D. Gašević. Analyzing students’ emerging roles based on quantity and heterogeneity of individual contributions in small group online collaborative learning using bipartite network analysis.arXiv preprint arXiv:2502.19112, 2025

    2025 arXiv

  14. [22]

    Freeman and J

    M. Freeman and J. McKenzie. Spark, a confidential web-based template for self and peer assessment of student teamwork: Benefits of evaluating across different subjects.British Journal of Educational Technology, 33(5):551–569, 2002

    2002

  15. [23]

    Annex III: High-Risk AI Systems Referred to in Article 6(2)

    Future of Life Institute. Annex III: High-Risk AI Systems Referred to in Article 6(2). EU Artificial Intelligence Act Explorer, 2025. Based on Regulation (EU) 2024/1689 Artificial Intelligence Act, Official Journal version of 13 June 2024

    2025

  16. [24]

    Goldfinch

    J. Goldfinch. Further developments in peer assessment of group projects.Assessment & Evaluation in Higher Education, 19(1):29–35, Jan. 1994

    1994

  17. [25]

    Goldfinch and R

    J. Goldfinch and R. Raeside. Development of a peer assessment technique for obtaining individual marks on a group project.Assessment & Evaluation in Higher Education, 15(3):210–231, Sept. 1990

    1990

  18. [26]

    Goodyear, L

    P. Goodyear, L. Carvalho, and P. Yeoman. Activity-centred analysis and design (acad): Core purposes, distinctive qualities and current developments.Educational Technology Research and Development, 69(2):445–464, 2021

    2021

  19. [27]

    Greller and H

    W. Greller and H. Drachsler. Translating learning into numbers: A generic framework for learning analytics.Educational Technology & Society, 15(3):42–57, July 2012. ISSN 1436-4522

    2012

  20. [28]

    Gyamfi, B

    G. Gyamfi, B. E. Hanna, and H. Khosravi. The effects of rubrics on evaluative judge- ment: A randomised controlled experiment.Assessment & Evaluation in Higher Educa- tion, 47(1):126–143, 2021

    2021

  21. [29]

    Hooshyar, K

    D. Hooshyar, K. Tammets, T. Ley, K. Aus, and K. Kollom. Learning analytics in support- ing student agency: A systematic review.Sustainability, 15(18):13662, 2023

    2023

  22. [30]

    Combininggithub, chat, andpeer evaluation data to assess individual contributions to team software development projects

    C.Hundhausen, P.Conrad, O.O.Adesope, andA.Tariq. Combininggithub, chat, andpeer evaluation data to assess individual contributions to team software development projects. ACM Transactions on Computing Education, 23(3):33, Sept. 2023

    2023

  23. [31]

    A guide to the data protection principles

    Information Commissioner’s Office. A guide to the data protection principles. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/ 15 data-protection-principles/a-guide-to-the-data-protection-principles/,

  24. [32]

    Data protection impact assessments (dpias)

    Information Commissioner’s Office. Data protection impact assessments (dpias). https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/ accountability-and-governance/data-protection-impact-assessments-dpias/ what-is-a-dpia/, n.d. Accessed: 2025-07-06

    2025

  25. [33]

    What is automated in- dividual decision-making and profiling?https://ico

    Information Commissioner’s Office. What is automated in- dividual decision-making and profiling?https://ico. org.uk/for-organisations/uk-gdpr-guidance-and-resources/ individual-rights/automated-decision-making-and-profiling/ what-is-automated-individual-decision-making-and-pro...

    2025

  26. [34]

    Canvas learning management system.https://www.instructure.com/ canvas, 2025

    Instructure,Inc. Canvas learning management system.https://www.instructure.com/ canvas, 2025. Open-source software; accessed 2025-07-30

    2025

  27. [35]

    Intedashboard: Team-based learning management platform, 2025

    InteDashboard. Intedashboard: Team-based learning management platform, 2025. Ac- cessed 24 July 2025

    2025

  28. [36]

    ipeer: Online peer & self-evaluation tool, 2006

    iPeer. ipeer: Online peer & self-evaluation tool, 2006. Open-source peer evaluation plat- form

    2006

  29. [37]

    i hate group work

    M. Isaac. “i hate group work”: Social loafers, indignant peers, and the drama of the classroom.The English Journal, 101(4):83–89, Mar. 2012

    2012

  30. [38]

    Building the future intelligent campus: Using data to make smarter use of your university estate.https://www.jisc.ac.uk/guides/ building-the-future-intelligent-campus, May 2023

    Jisc. Building the future intelligent campus: Using data to make smarter use of your university estate.https://www.jisc.ac.uk/guides/ building-the-future-intelligent-campus, May 2023. Accessed: 2025-07-03

    2023

  31. [39]

    Jonsson and G

    A. Jonsson and G. Svingby. The use of scoring rubrics: Reliability, validity and educational consequences.Educational Research Review, 2(2):130–144, 2007

    2007

  32. [40]

    Katsenos and C

    I. Katsenos and C. Pierrakeas. Assessing individual contributions of team members to team achievement by combining peer assessments and digital presence in an academic environment.Education Sciences, 15(3):279, 2025

    2025

  33. [41]

    D. B. Kaufman, R. M. Felder, and H. Fuller. Accounting for individual effort in cooperative learning teams.Journal of Engineering Education, 89(2):133–140, 2000

    2000

  34. [42]

    Khosravi, K

    H. Khosravi, K. Kitto, and J. J. Williams. Ripple: A crowdsourced adaptive platform for recommendation of learning activities.Journal of Learning Analytics, 6(3):91–105, 2019

    2019

  35. [43]

    Kiafar, P

    B. Kiafar, P. U. Ravva, A. A. Joy, S. Daher, and R. L. Barmaki. MENA: Multimodal epistemic network analysis for visualizing competencies and emotions.arXiv preprint arXiv:2504.02794, Apr. 2025

    2025 arXiv

  36. [44]

    R. A. Layton and M. W. Ohland. Peer evaluations in teams of predominantly minority students. InProceedings of the 2000 ASEE Annual Conference, St. Louis, Missouri, June

    2000

  37. [45]

    L. Li, P. Louis, Z. Maniadis, and D. Xefteris. Reciprocity in peer assessments, Dec. 2024. SSRN working paper. 16

    2024

  38. [46]

    L. K. Y. Li. Some refinements on peer assessment of group projects.Assessment & Evaluation in Higher Education, 26(1):5–18, 2001

    2001

  39. [47]

    Liang, J

    J. Liang, J. M. Stephens, and G. T. L. Brown. A systematic review of the early impact of artificialintelligenceonhighereducationcurriculum, instruction, andassessment.Frontiers in Education, 10:1522841, Apr. 2025

    2025

  40. [48]

    Loddington, K

    S. Loddington, K. Pond, N. Wilkinson, and P. Willmot. A case study of the develop- ment of webpa: An online peer-moderated marking tool.British Journal of Educational Technology, 40(2):329–341, 2009

    2009

  41. [49]

    Long and G

    P. Long and G. Siemens. Penetrating the fog: Analytics in learning and education.EDU- CAUSE Review, 46(5):31–40, Sept. 2011

    2011

  42. [50]

    M. L. Loughry, M. W. Ohland, and D. Moore. Development of a theory-based assessment of team member effectiveness.Educational and Psychological Measurement, 67(3):505–524, 2007

    2007

  43. [51]

    V. N. Lunetta. Cooperative learning in science, mathematics, and computer problem solv- ing. In M. Gardner, J. G. Greeno, F. Reif, A. H. Schoenfeld, and A. A. di Sessa, editors, Toward a Scientific Practice of Science Education, pages 235–249. Lawrence Erlbaum As- sociates, Hi...

    1990

  44. [52]

    Martinez-Maldonado, D

    R. Martinez-Maldonado, D. Gašević, V. Echeverría, G. Fernández Nieto, Z. Swiecki, and S. Buckingham Shum. What do you mean by collaboration analytics? a conceptual model. Journal of Learning Analytics, 8(1):126–153, Apr. 2021

    2021

  45. [53]

    Miao and W

    F. Miao and W. Holmes. Guidance for generative AI in education and research. Guidance document, United Nations Educational, Scientific and Cultural Organization (UNESCO), Paris, Sept. 2023. Published 7 September 2023; last updated 14 April 2025

    2023

  46. [54]

    Mitra and E

    J. Mitra and E. A. Gerber. Analyzing individual contribution in team-based software engineering projects. InASEE Annual Conference & Exposition, United States, July

  47. [55]

    Mohammadi, E

    M. Mohammadi, E. Tajik, R. Martinez-Maldonado, S. Sadiq, W. Tomaszewski, and H. Khosravi. Artificial intelligence in multimodal learning analytics: A systematic lit- erature review.Computers & Education: Artificial Intelligence, 8:100426, 2025

    2025

  48. [56]

    Nikolopoulou

    K. Nikolopoulou. Generative artificial intelligence in higher education: Exploring ways of harnessing pedagogical practices with the assistance of chatgpt.International Journal of Changes in Education, Mar. 2024

    2024

  49. [57]

    Oakley, R

    B. Oakley, R. M. Felder, R. Brent, and I. Elhajj. Turning student groups into effective teams.Journal of Student Centered Learning, 2(1):9–34, 2004

    2004

  50. [58]

    M. W. Ohland, M. L. Loughry, D. J. Woehr, L. G. Bullard, R. M. Felder, C. J. Finelli, R. A. Layton, H. R. Pomeranz, and D. G. Schmucker. The comprehensive assessment of team member effectiveness: Development of a behaviorally anchored rating scale for self- and peer evaluation...

    2012

  51. [59]

    Panadero, H

    E. Panadero, H. Andrade, and S. M. Brookhart. Fusing self-regulated learning and forma- tive assessment: a roadmap of where we are, how we got there, and where we are going. The Australian Educational Researcher, 45(1):13–31, 2018. 17

    2018

  52. [60]

    T. Panitz. Collaborative versus cooperative learning: A comparison of the two concepts which will help us understand the underlying nature of interactive learning. ERIC Docu- ment ED448443, Dec. 1999. Opinion paper, 13 pp. Accessed 18 July 2025

    1999

  53. [61]

    Perkins, J

    M. Perkins, J. Roe, and L. Furze. The AI assessment scale revisited: A framework for educational assessment. arXiv preprint arXiv:2412.09029, Dec. 2024. cs.CY

    2024 arXiv

  54. [62]

    Ridwan, A

    M. Ridwan, A. Najib, H. Ruzakki, Rofiqi, and Muslimin. Bias in peer assessment: Chal- lenges, solutions, and best practices for fair student evaluation.Al-Ishlah: Jurnal Pen- didikan, 17(2):2663–2677, June 2025

    2025

  55. [63]

    What is cooperative learn- ing?https://serc.carleton.edu/introgeo/cooperative/whatis.html, 2006

    Science Education Resource Center at Carleton College. What is cooperative learn- ing?https://serc.carleton.edu/introgeo/cooperative/whatis.html, 2006. Start- ing Point - Teaching Entry Level Geoscience. Accessed 18 July 2025

    2006

  56. [64]

    Slade and P

    S. Slade and P. Prinsloo. Learning analytics: Ethical issues and dilemmas.American Behavioral Scientist, 57(10):1510–1529, 2013

    2013

  57. [65]

    R. E. Slavin. Research on cooperative learning and achievement: What we know, what we need to know.Contemporary Educational Psychology, 21(1):43–69, 1996

    1996

  58. [66]

    J. H. Stonewall.Development and Evaluation of Training to Increase Student Perceptions of Fairness in Peer Assessment. Phd dissertation, Iowa State University, Ames, IA, 2022. Industrial & Manufacturing Systems Engineering / Human–Computer Interaction

    2022

  59. [67]

    H. Sung, M. L. Bernacki, J. A. Greene, L. Yu, and R. D. Plumley. Beyond frequency: Using epistemic network analysis and multimodal traces to understand temporal dynamics of self- regulated learning.Journal of Science Education and Technology, 2024. Advance online publication, ...

    2024

  60. [68]

    TEAMMATES: Online peer feedback / peer evaluation system, 2010

    TEAMMATES. TEAMMATES: Online peer feedback / peer evaluation system, 2010. Accessed on 2025-07-24

    2010

  61. [69]

    Tempelaar, B

    D. Tempelaar, B. Rienties, and B. Giesbers. Dispositional learning analytics and formative assessment: an inseparable twinship.International Journal of Educational Technology in Higher Education, 21(1):57, 2024. Published 9 October 2024

    2024

  62. [70]

    Y. Tong, C. D. Schunn, and H. Wang. Why increasing the number of raters only helps sometimes: Reliability and validity of peer assessment across tasks of different complexity. Studies in Educational Evaluation, 76:101233, 2023

    2023

  63. [71]

    Topping, E

    K. Topping, E. Gehringer, H. Khosravi, S. Gudipati, K. Jadhav, and S. Susarla. En- hancing peer assessment with artificial intelligence.International Journal of Educational Technology in Higher Education, 22:3, Jan. 2025

    2025

  64. [72]

    K. J. Topping. Peer assessment between students in colleges and universities.Review of Educational Research, 68(3):249–276, 1998

    1998

  65. [73]

    Policy on generative artificial intelligence in learning, teaching and assessment.https://le.ac.uk/-/media/uol/docs/policies/ quality/ai-policy.pdf, May 2024

    University of Leicester AI Policy Working Group. Policy on generative artificial intelligence in learning, teaching and assessment.https://le.ac.uk/-/media/uol/docs/policies/ quality/ai-policy.pdf, May 2024. Issued May 2024; reviewed November 2024; accessed 2025-07-06. 18

    2024

  66. [74]

    AI tools for education.https://warwick.ac.uk/fac/cross_ fac/academic-development/ai/tools/, Feb

    University of Warwick. AI tools for education.https://warwick.ac.uk/fac/cross_ fac/academic-development/ai/tools/, Feb. 2025. Last revised on February 14, 2025; accessed 2025-07-06

    2025

  67. [75]

    Learning and teaching with AI.https://warwick.ac.uk/fac/ cross_fac/academic-development/ai/learning-and-teaching/, Feb

    University of Warwick. Learning and teaching with AI.https://warwick.ac.uk/fac/ cross_fac/academic-development/ai/learning-and-teaching/, Feb. 2025. Last re- vised on February 14, 2025; accessed 2025-07-06

    2025

  68. [76]

    Ethical approval – research integrity services, n.d

    University of Warwick. Ethical approval – research integrity services, n.d. Accessed: 2025- 07-06

    2025

  69. [77]

    AI tools for education.https:// warwick.ac.uk/fac/cross_fac/academic-development/ai/tools/, 2025

    University of Warwick, Academic Development Centre. AI tools for education.https:// warwick.ac.uk/fac/cross_fac/academic-development/ai/tools/, 2025. Last revised: 14 February 2025

    2025

  70. [78]

    Ig02: Data protection policy

    University of Warwick, Information & Digital Group. Ig02: Data protection policy. https://warwick.ac.uk/services/idg/im-policy-framework/policies/ig02/, June

  71. [79]

    Artificial intelligence information compliance policy (imp02).https://warwick.ac.uk/services/idg/ im-policy-framework/policies/imp02/, June 2025

    University of Warwick, Information and Digital Group. Artificial intelligence information compliance policy (imp02).https://warwick.ac.uk/services/idg/ im-policy-framework/policies/imp02/, June 2025. Last revised 17 June 2025; accessed 3 July 2025

    2025

  72. [80]

    M. Usher. Generative AI vs. instructor vs. peer assessments: A comparison of grading and feedback in higher education.Assessment & Evaluation in Higher Education, Apr. 2025

    2025

  73. [81]

    Watari, S

    T. Watari, S. Koyama, Y. Kato, Y. Paku, Y. Kanada, and H. Sakurai. Effect of moderation on rubric criteria for inter-rater reliability in an objective structured clinical examination with real patients.Fujita Medical Journal, 8(3):83–87, 2022

    2022

  74. [82]

    Privacy notice for students.https://www.wmgacademy.org.uk/ wp-content/uploads/2024/09/Privacy-Notice-for-Students.pdf, Sept

    WMG Academy Trust. Privacy notice for students.https://www.wmgacademy.org.uk/ wp-content/uploads/2024/09/Privacy-Notice-for-Students.pdf, Sept. 2024. Pub- lished: September 2024

    2024

  75. [83]

    Policies.https://www.wmgacademy.org.uk/about-us/ policies/, 2025

    WMG Academy Trust. Policies.https://www.wmgacademy.org.uk/about-us/ policies/, 2025. Accessed: 2025-07-03

    2025

  76. [84]

    M. Yang, Z. Yan, L. Yang, and Y. Zhan. Peer assessment: A powerful way to cultivate stu- dent assessment literacy. InUnderstanding and Developing Student Assessment Literacy, SpringerBriefs in Education, pages 51–67. Springer, Singapore, 2025

    2025

  77. [85]

    Zhao et al

    C. Zhao et al. Ai-assisted assessment in higher education: A systematic review.JETI, 6(4), 2024

    2024

  78. [86]

    C. Zhao, M. Silva, and S. Poulsen. Language models are few-shot graders, 2025

    2025

  79. [87]

    J. Zhao, E. Chapman, and P. G. P. Sabet. Generative ai and educational assessments: A systematic review.Educational Research and Practice, 51, Dec. 2024. The University of Western Australia; Curtin University

    2024

  80. [88]

    L. Zhao, V. Echeverría, Z. Swiecki, L. Yan, R. Alfredo, X. Li, D. Gašević, and R. Martinez- Maldonado. Epistemic network analysis for end-users: Closing the loop in the context of multimodal analytics for collaborative team learning. InProceedings of the 14th Learning 19 Analy...

    2024

  81. [2023]

    Guidance under review due to the Data (Use and Access) Act; accessed 2025-07-06

    2025

  82. [2024]

    Published via ASEE PEER

  83. [2025]

    Last revised: 17 June 2025

    2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.