pith. sign in

arxiv: 2604.15020 · v1 · submitted 2026-04-16 · 💻 cs.SE · cs.HC

Applying SHAPR in AI-Assisted Research Software Development: Lessons Learnt from Building a Share Trading System

Ka Ching Chan This is my paper

Pith reviewed 2026-05-10 11:03 UTC · model grok-4.3

classification 💻 cs.SE cs.HC
keywords SHAPRAI-assisted developmentresearch software engineeringdocumentationknowledge continuitylessons learnedshare trading systemgenerative AI
0
0 comments X

The pith

Applying the SHAPR framework to the development of a share trading system demonstrates that continuous documentation updates supported by quick capture and AI-assisted refinement maintain organised and usable project knowledge.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a case of applying the SHAPR framework to build a modular share trading system using an organized approach from the start. It generated an evidence base of notes, contracts, and documents that allowed knowledge to stay usable across cycles. Five lessons were identified regarding how contracts, source-of-truth layers, snapshots, co-evolution, and setup contribute to better continuity in AI-assisted work. A practical tool setup is described that supports the framework without tying to specific applications. This case provides guidance for researchers to address continuity and traceability issues in rapid generative AI development.

Core claim

In this case of SHAPR application to share trading system development, continuous documentation updates supported by quick capture and AI-assisted refinement helped maintain organised and usable project knowledge throughout development, from which five recurring lessons were identified on the stabilizing role of contracts, the coherence from a source-of-truth layer, strengthened continuity from cycle-boundary snapshots, the co-evolution of code and documentation, and knowledge generation from environment setup.

What carries the argument

SHAPR framework as applied through a working configuration of contracts, maintained source-of-truth documents, cycle-boundary snapshots, quick captures, and iterative refinement to structure human-AI collaboration and documentation.

Load-bearing premise

That the experiences and lessons from this single documented case of building a share trading system are broadly applicable or transferable to other AI-assisted research software development projects in different domains or team settings.

What would settle it

Conducting a similar AI-assisted development project in another domain without observing maintained organized knowledge or the five lessons would falsify the transferability of the findings.

Figures

Figures reproduced from arXiv: 2604.15020 by Ka Ching Chan.

Figure 1
Figure 1. Figure 1: Practical SHAPR operating configuration used in the case. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: General SHAPR process used within each development cycle. The [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
read the original abstract

Generative AI is changing how research software is developed, but rapid AI-assisted development can weaken continuity, traceability, and methodological clarity. SHAPR (Solo, Human-centred, AI-assisted PRactice) was proposed as a framework for structuring AI-assisted research software development. This paper presents a documented case of applying SHAPR to the development of a modular share trading system. From the outset, the project adopted a SHAPR-informed working configuration that shaped how interaction, implementation, and documentation were organised. Across iterative development cycles, the project generated a structured evidence base including reflection notes, development cycle review notes, source-of-truth documents, contracts, quick captures, workflow notes, and evolving code artefacts. The case showed that continuous documentation updates, supported by quick capture and AI-assisted refinement, helped maintain organised and usable project knowledge throughout development. Five recurring lessons were identified: contracts stabilised AI-assisted coding, a maintained source-of-truth layer improved coherence, cycle-boundary snapshots strengthened continuity, code and documentation co-evolved through quick capture and iterative refinement, and environment setup itself contributed to knowledge generation. The case also illustrates a practical SHAPR operating configuration in which a ChatGPT Project and cycle-specific chats supported interaction, reasoning, summarisation, and coding collaboration, PyCharm supported artefact implementation, and Obsidian supported external working memory, structured documentation, reflection, continuity, and repository-oriented note organisation, while remaining consistent with SHAPR's tool-agnostic principle. The paper contributes practical guidance and good practices for researchers conducting AI-assisted research software development.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper reports a single-author case study applying the SHAPR framework to develop a modular share trading system with generative AI assistance. It describes generation of a structured evidence base (reflection notes, source-of-truth documents, contracts, cycle reviews, quick captures) across iterative cycles and claims that continuous documentation updates supported by quick capture and AI-assisted refinement maintained organised and usable project knowledge. Five lessons are identified: contracts stabilise AI-assisted coding, a maintained source-of-truth layer improves coherence, cycle-boundary snapshots strengthen continuity, code and documentation co-evolve through quick capture and refinement, and environment setup contributes to knowledge generation. The paper also details a practical SHAPR operating configuration using ChatGPT Projects, PyCharm, and Obsidian while remaining tool-agnostic, and positions the work as contributing practical guidance for AI-assisted research software development.

Significance. If the lessons rest on systematic extraction from the documented artefacts, the paper supplies timely, concrete practitioner guidance on preserving continuity, traceability, and methodological clarity when using generative AI for research software. The emphasis on an external working memory layer (Obsidian) and cycle-boundary artefacts addresses a recognised pain point in rapid AI-assisted development. The tool-agnostic stance and explicit operating configuration are strengths that could seed follow-on empirical work or framework refinements in the software engineering community.

major comments (2)
  1. [Case study results and lessons identification] The central claim that the SHAPR-informed configuration 'helped maintain organised and usable project knowledge' and that the five lessons are 'recurring' rests entirely on the author's internal artefacts from one solo project. No quantitative metrics of knowledge usability, traceability, or coherence are reported, nor is there a non-SHAPR baseline or second case. This makes it impossible to separate framework effects from project-specific factors (domain, solo setting, tool choices). See the sections describing the evidence base and the five lessons.
  2. [Lessons Learned] The paper does not describe the analytic procedure used to derive the five lessons from the reflection notes, cycle reviews, and other artefacts. Without an explicit method (e.g., thematic coding, member checking, or inter-rater reliability), the lessons risk appearing post-hoc rather than systematically grounded in the evidence base.
minor comments (2)
  1. The abstract and introduction should more explicitly delimit the scope of the contribution, stating that the lessons are observations from one documented case rather than validated general principles.
  2. A short threats-to-validity subsection would help readers assess the single-author, self-documented nature of the study.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive and detailed feedback on our case study. We address each major comment below, providing clarifications on the nature of the evidence and the derivation of lessons, while indicating where revisions will be made to improve transparency and rigor.

read point-by-point responses

  1. Referee: [Case study results and lessons identification] The central claim that the SHAPR-informed configuration 'helped maintain organised and usable project knowledge' and that the five lessons are 'recurring' rests entirely on the author's internal artefacts from one solo project. No quantitative metrics of knowledge usability, traceability, or coherence are reported, nor is there a non-SHAPR baseline or second case. This makes it impossible to separate framework effects from project-specific factors (domain, solo setting, tool choices). See the sections describing the evidence base and the five lessons.

    Authors: We acknowledge that this is a single-author case study without quantitative metrics or a comparative baseline, which inherently limits causal attribution and generalizability. The paper positions the work as an illustrative account of applying the SHAPR framework, with claims grounded in the documented artefacts (reflection notes, cycle reviews, source-of-truth documents, contracts, and quick captures) rather than experimental controls. The term 'recurring' refers to patterns observed consistently across the project's iterative cycles. We will revise the manuscript to add an explicit limitations subsection discussing the single-case design, the absence of metrics or baselines, and the exploratory nature of the practitioner insights. revision: partial

  2. Referee: [Lessons Learned] The paper does not describe the analytic procedure used to derive the five lessons from the reflection notes, cycle reviews, and other artefacts. Without an explicit method (e.g., thematic coding, member checking, or inter-rater reliability), the lessons risk appearing post-hoc rather than systematically grounded in the evidence base.

    Authors: We agree that an explicit description of the analytic procedure is required to demonstrate how the lessons are systematically derived. The five lessons emerged from an inductive review process in which the author repeatedly examined the full evidence base, identified recurring themes related to documentation practices, AI interaction stability, and knowledge continuity, and cross-checked these against specific artefacts such as cycle-boundary snapshots and contracts. We will add a dedicated subsection (e.g., under 'Lessons Learned') detailing this thematic analysis approach, including the steps taken to ensure grounding in the artefacts. revision: yes

standing simulated objections not resolved
  • The absence of quantitative metrics of knowledge usability or a non-SHAPR baseline comparison, as these cannot be retroactively generated from the existing single-case artefacts without new data collection or additional studies.

Circularity Check

0 steps flagged

Reflective case study reports observations without circular derivations or self-referential reductions

full rationale

The paper is an empirical reflective case study documenting the application of the SHAPR framework to one share-trading-system project. The five lessons are extracted directly from the project's own artefacts (reflection notes, contracts, source-of-truth documents, quick captures, and cycle reviews). No equations, fitted parameters, predictions, or first-principles derivations appear that could reduce to the inputs by construction. Any reference to the prior proposal of SHAPR functions as background context rather than a load-bearing justification that loops back; the evidence base and claims remain the documented single-case experience itself. This is a standard self-contained empirical report with no circularity patterns matching the enumerated kinds.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

This is a qualitative case study paper with no mathematical models, quantitative fits, or postulated entities; the central claims rest on the domain assumption that reflective documentation from one project yields transferable lessons.

axioms (1)
  • domain assumption Qualitative lessons extracted from a single documented case study of AI-assisted software development are generalizable enough to provide practical guidance for other researchers.
    The paper bases its contribution on five recurring lessons from one project without multiple cases, statistical support, or external validation.

pith-pipeline@v0.9.0 · 5575 in / 1664 out tokens · 57427 ms · 2026-05-10T11:03:47.549288+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

8 extracted references · 8 canonical work pages

  1. [1]

    V. C. Stodden, D. Donoho, et al., ``Reproducible research: Addressing the need for data and code sharing in computational science,'' Comput. Sci. Eng., vol. 12, no. 5, pp. 8--12, Sep./Oct. 2010

    work page 2010

  2. [2]

    Guidelines for Human-AI Interaction,

    S. Amershi, D. Weld, M. Vorvoreanu, et al., ``Guidelines for human--AI interaction,'' in Proc. 2019 CHI Conf. Human Factors Comput. Syst., 2019, pp. 1--13, doi: 10.1145/3290605.3300233

    work page doi:10.1145/3290605.3300233 2019

  3. [3]

    Dynamical

    B. Shneiderman, Human-Centered AI. Oxford, U.K.: Oxford Univ. Press, 2022, doi: 10.1093/oso/9780192845290.001.0001

    work page doi:10.1093/oso/9780192845290.001.0001 2022

  4. [4]

    K. C. Chan, ``SHAPR: A Solo Human-Centred and AI-Assisted Practice Framework for Research Software Development,'' arXiv preprint arXiv:2602.12443, 2026

    work page arXiv 2026

  5. [5]

    K. C. Chan, ``SHAPR: Operationalising Human--AI Collaborative Research Through Structured Knowledge Generation,'' arXiv preprint arXiv:2603.25660, 2026

    work page arXiv 2026

  6. [6]

    M. K. Sein, O. Henfridsson, S. Purao, M. Rossi, and R. Lindgren, ``Action design research,'' MIS Quart., vol. 35, no. 1, pp. 37--56, 2011

    work page 2011

  7. [7]

    K. C. Chan, ``Code and documentation for applying SHAPR in AI-assisted software development: Building a share trading system,'' Zenodo, 2026, doi: 10.5281/zenodo.19592651

    work page doi:10.5281/zenodo.19592651 2026

  8. [8]

    + '&m ؕ:w+2Yg u /v뤰 S9gv]/ȝ V SRPtE閞n< ZWN2ٍ@xв - UYzlG` ؘneśk|

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

    work page