pith. sign in

arxiv: 2606.12424 · v1 · pith:G34L2Q6Bnew · submitted 2026-05-10 · 💻 cs.CY · cs.AI· cs.HC

AI-Automation Tooling in Computer Engineering Education: Mixed-Methods TAM/UTAUT Evidence for a General Acceptance Attitude

Aung Pyae This is my paper

Pith reviewed 2026-06-30 22:36 UTC · model grok-4.3

classification 💻 cs.CY cs.AIcs.HC
keywords AI automation toolingtechnology acceptanceTAM/UTAUTcomputer engineering educationmixed-methods studyperformance expectancycurricular adoptionn8n platform
0
0 comments X

The pith

Undergraduate computer engineering students accept AI automation tools favorably across all measured constructs, with performance expectancy as the strongest factor.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether students will accept generative AI and low-code workflow tools as useful and worth using in their training. It ran three workshops with 103 Thai undergraduates using the n8n platform and collected both survey scores and open feedback. Acceptance proved positive on every construct with large effects. The data also show that the usual separate dimensions collapse into one overall acceptance attitude in this setting. These results point toward adding the tools to computing curricula and using three specific teaching adjustments to support adoption.

Core claim

Acceptance of AI automation tooling proved favorable across all six TAM/UTAUT constructs with large effect sizes. Performance expectancy emerged as the strongest construct and hedonic motivation as the weakest. Dimensionality checks showed the constructs collapsing into a single general acceptance factor. Qualitative feedback aligned on usefulness but revealed a minority skeptical about output reliability. The pattern supports curricular adoption and identifies three instructional levers: instruction-sequencing scaffolds, self-efficacy supports, and trust-calibration interventions.

What carries the argument

A 12-item Likert survey instrument that maps student responses to the six technology-acceptance constructs of performance expectancy, effort expectancy, behavioral intention, self-efficacy, hedonic motivation, and output quality.

If this is right

  • Curricular adoption of AI automation tooling receives empirical support for undergraduate computing education.
  • Instruction-sequencing scaffolds, self-efficacy supports, and trust-calibration interventions function as usable levers for improving acceptance.
  • Short post-workshop instruments can capture a unified acceptance attitude rather than distinct sub-facets.
  • Qualitative reliability concerns must be addressed even when quantitative scores remain favorable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Acceptance patterns observed here may extend to students in related technical fields such as data science or information systems.
  • Demonstrating tool reliability through concrete examples could shrink the skeptical minority on output quality.
  • Follow-up measurements of actual tool use weeks after the workshops would test whether immediate acceptance predicts longer-term behavior.
  • The single-factor result raises the possibility that brief surveys after short exposures do not separate the classic TAM dimensions.

Load-bearing premise

The 12-item instrument validly measures the six constructs in this setting and the sample of 103 students from three Thai workshops is representative of broader computer engineering education.

What would settle it

A larger study in other countries or institutions that finds low or negative scores on performance expectancy would contradict the acceptance claim.

Figures

Figures reproduced from arXiv: 2606.12424 by Aung Pyae.

Figure 1
Figure 1. Figure 1: Workshop protocol used in all three identically scripted sessions; the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustrative n8n canvas from the workshop showing a simple Trigger [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
read the original abstract

As generative AI and low-code workflow platforms become routine in software practice, a key educational question is whether the next generation of computer engineers will accept these tools as useful, usable, and worthy of sustained engagement. This paper reports a mixed-methods, cross-sectional study of undergraduate computer engineering students' acceptance of AI automation tooling, instantiated through the open-source platform n8n across three identically scripted workshops in Thailand (n = 103). A 12-item, five-point Likert instrument mapped to six TAM/UTAUT constructs - Performance Expectancy (PE), Effort Expectancy (EE), Behavioral Intention (BI), Self-Efficacy (SE), Hedonic Motivation (HM), and Output Quality (OQ) - was complemented by inductive thematic analysis of open-ended feedback. Analyses combined ordinal reliability estimation, bootstrap confidence intervals, non-parametric tests, multiple-comparison-controlled correlations, polychoric dimensionality diagnostics, a common-method-bias check, and between-session comparisons. Acceptance was favorable across all six constructs with large effect sizes, with PE emerging as the strongest construct and HM as the weakest. Dimensionality diagnostics further revealed that canonical TAM/UTAUT sub-facets collapsed into a single general acceptance factor in this short-form post-workshop context, a finding with important methodological and theoretical implications. Qualitative themes converged with the quantitative profile regarding usefulness and enthusiasm but diverged on output quality, revealing a small yet articulate reliability-skeptical minority. The findings support the curricular adoption of AI automation tooling in undergraduate computing education and identify three theory-grounded instructional levers: instruction-sequencing scaffolds, self-efficacy supports, and trust-calibration interventions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript reports a mixed-methods cross-sectional study of n=103 undergraduate computer engineering students in three Thailand workshops on acceptance of AI automation tooling (n8n). A 12-item Likert instrument mapped to six TAM/UTAUT constructs (PE, EE, BI, SE, HM, OQ) is analyzed with ordinal reliability, bootstrap CIs, non-parametric tests, polychoric dimensionality diagnostics, and common-method bias checks, complemented by thematic analysis of open feedback. It claims favorable acceptance across all six constructs with large effect sizes (PE strongest, HM weakest), notes collapse of constructs into a single general acceptance factor, reports qualitative convergence on usefulness but divergence on output quality, and concludes support for curricular adoption with three theory-grounded instructional levers.

Significance. If the measurement model and construct interpretations hold, the work supplies empirical TAM/UTAUT evidence on AI tooling acceptance in computing education, identifies PE as a key driver, and offers practical levers (instruction-sequencing, self-efficacy supports, trust-calibration). The mixed-methods design, bootstrap/non-parametric methods, and explicit dimensionality diagnostic add methodological transparency; the collapse finding itself is a useful cautionary result for short-form post-workshop instruments.

major comments (2)
  1. [Abstract] Abstract: the claim of favorable acceptance 'across all six constructs with large effect sizes' and the identification of PE as strongest/HM as weakest treats the constructs as separable and comparable, yet the same paragraph reports that 'polychoric dimensionality diagnostics showed the six TAM/UTAUT constructs collapsed into a single general acceptance factor.' This internal tension makes the six-construct profile and derived instructional levers rest on an unsupported measurement model.
  2. [Methods/Results] Methods/Results (instrument validation section): the 12-item instrument is asserted to map to the six constructs, but no factor loadings, item-total correlations, or CFA fit indices are referenced to justify retaining distinct construct scores after the polychoric analysis indicated unidimensionality. Without this, mean-score comparisons and effect-size claims for individual constructs cannot be interpreted as evidence for separable TAM/UTAUT facets.
minor comments (2)
  1. [Methods] The Thailand workshop sample (n=103) is described but no power analysis or discussion of selection effects is provided; this is a presentation gap rather than a load-bearing flaw.
  2. [Results] Table or figure captions for the polychoric diagnostics and construct means should explicitly state the number of factors retained and the proportion of variance explained.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for identifying the measurement-model tension in our presentation. We agree that the abstract and results sections require revision to avoid implying separability after the unidimensionality finding. We address each major comment below and will make the corresponding changes.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim of favorable acceptance 'across all six constructs with large effect sizes' and the identification of PE as strongest/HM as weakest treats the constructs as separable and comparable, yet the same paragraph reports that 'polychoric dimensionality diagnostics showed the six TAM/UTAUT constructs collapsed into a single general acceptance factor.' This internal tension makes the six-construct profile and derived instructional levers rest on an unsupported measurement model.

    Authors: We agree that the current abstract wording creates an internal inconsistency by reporting both construct-level comparisons and the collapse into a single factor. The strongest honest defense is that the per-construct means were computed from the a priori item mapping for descriptive transparency, but we accept that this does not constitute evidence of separable facets once unidimensionality is established. We will revise the abstract to state the unidimensionality result first, present the construct-level statistics only as supplementary item-group summaries with explicit qualification, and reframe the instructional levers around the general acceptance factor plus qualitative themes rather than differential construct strengths. revision: yes

  2. Referee: [Methods/Results] Methods/Results (instrument validation section): the 12-item instrument is asserted to map to the six constructs, but no factor loadings, item-total correlations, or CFA fit indices are referenced to justify retaining distinct construct scores after the polychoric analysis indicated unidimensionality. Without this, mean-score comparisons and effect-size claims for individual constructs cannot be interpreted as evidence for separable TAM/UTAUT facets.

    Authors: The referee is correct: the manuscript reports polychoric diagnostics showing unidimensionality yet still presents and compares six construct scores without loadings or multi-factor fit statistics. We cannot defend the current presentation as evidence for distinct facets. We will add explicit text in the results section stating that, given the unidimensional solution, all primary analyses treat the scale as a single general acceptance factor; the original construct groupings will be retained only for supplementary descriptive tables with the caveat that they do not represent separable dimensions. Effect-size claims will be limited to the overall factor and to qualitative convergence. revision: yes

Circularity Check

0 steps flagged

Empirical survey reports no circular derivation chain

full rationale

The paper collects primary survey data (n=103) and applies established TAM/UTAUT constructs plus standard statistical diagnostics (polychoric, reliability, non-parametric tests). All quantitative claims are direct empirical summaries of the collected responses rather than predictions derived from fitted parameters or self-citations. The dimensionality collapse is reported as an observed diagnostic, not used to generate the main acceptance-profile claims. No step reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper relies on the established TAM/UTAUT framework as axioms and applies them to new data; no new entities postulated. The main addition is empirical evidence from the study.

axioms (2)
  • domain assumption The TAM/UTAUT constructs (PE, EE, BI, SE, HM, OQ) are valid and applicable to measuring acceptance of AI automation tools in educational settings.
    The paper maps the 12-item instrument to these six constructs without re-deriving them.
  • domain assumption The workshops were identically scripted and the sample is sufficient for the statistical analyses performed.
    Cross-sectional study with n=103 across three sessions; assumes standardization and adequate power.

pith-pipeline@v0.9.1-grok · 5832 in / 1407 out tokens · 36038 ms · 2026-06-30T22:36:13.144267+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 1 canonical work pages

  1. [1]

    n8n: Workflow automation tool,

    n8n GmbH, “n8n: Workflow automation tool,” n8n.io. [Online]. Available: https://n8n.io. [Accessed: 24-Apr-2026]

    2026

  2. [2]

    Perceived usefulness, perceived ease of use, and user acceptance of information technology,

    F. D. Davis, “Perceived usefulness, perceived ease of use, and user acceptance of information technology,” MIS Quart., vol. 13, no. 3, pp. 319–340, Sep. 1989

    1989

  3. [3]

    User acceptance of information technology: Toward a unified view,

    V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User acceptance of information technology: Toward a unified view,” MIS Quart., vol. 27, no. 3, pp. 425–478, Sep. 2003

    2003

  4. [4]

    Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology,

    V. Venkatesh, J. Y. L. Thong, and X. Xu, “Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology,” MIS Quart., vol. 36, no. 1, pp. 157– 178, Mar. 2012

    2012

  5. [5]

    A theoretical extension of the technology acceptance model: Four longitudinal field studies,

    V. Venkatesh and F. D. Davis, “A theoretical extension of the technology acceptance model: Four longitudinal field studies,” Manage. Sci., vol. 46, no. 2, pp. 186–204, Feb. 2000

    2000

  6. [6]

    Self -efficacy: Toward a unifying theory of behavioral change,

    A. Bandura, “Self -efficacy: Toward a unifying theory of behavioral change,” Psychol. Rev., vol. 84, no. 2, pp. 191–215, Mar. 1977

    1977

  7. [7]

    Computer self -efficacy: Development of a measure and initial test,

    D. R. Compeau and C. A. Higgins, “Computer self -efficacy: Development of a measure and initial test,” MIS Quart., vol. 19, no. 2, pp. 189–211, Jun. 1995

    1995

  8. [8]

    Trust in automation: Designing for appropriate reliance,

    J. D. Lee and K. A. See, “Trust in automation: Designing for appropriate reliance,” Human Factors, vol. 46, no. 1, pp. 50–80, Mar. 2004

    2004

  9. [9]

    Trust in automation: Integrating empirical evidence on factors that influence trust,

    K. A. Hoff and M. Bashir, “Trust in automation: Integrating empirical evidence on factors that influence trust,” Human Factors, vol. 57, no. 3, pp. 407–434, May 2015

    2015

  10. [10]

    Humans and automation: Use, misuse, disuse, abuse,

    R. Parasuraman and V. Riley, “Humans and automation: Use, misuse, disuse, abuse,” Human Factors, vol. 39, no. 2, pp. 230–253, Jun. 1997

    1997

  11. [11]

    A model for types and levels of human interaction with automation,

    R. Parasuraman, T. B. Sheridan, and C. D. Wickens, “A model for types and levels of human interaction with automation,” IEEE Trans. Syst., Man, Cybern. A, Syst. Humans, vol. 30, no. 3, pp. 286–297, May 2000

    2000

  12. [12]

    Does the whole exceed its parts? The effect of AI explanations on complementary team performance,

    G. Bansal, T. Wu, J. Zhou, R. Fok, B. Nushi, E. Kamar, M. T. Ribeiro, and D. S. Weld, “Does the whole exceed its parts? The effect of AI explanations on complementary team performance,” in Proc. 2021 CHI Conf. Human Factors Comput. Syst. (CHI '21) , Yokohama, Japan, 2021, Art. 81, pp. 1–16

    2021

  13. [13]

    To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI - assisted decision-making,

    Z. Buçinca, M. B. Malaya, and K. Z. Gajos, “To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI - assisted decision-making,” Proc. ACM Hum.-Comput. Interact., vol. 5, no. CSCW1, Art. 188, pp. 1–21, Apr. 2021

    2021

  14. [14]

    Extrinsic and intrinsic motivation to use computers in the workplace,

    F. D. Davis, R. P. Bagozzi, and P. R. Warshaw, “Extrinsic and intrinsic motivation to use computers in the workplace,” J. Appl. Soc. Psychol., vol. 22, no. 14, pp. 1111–1132, Jul. 1992

    1992

  15. [15]

    Supporting the understanding and comparison of low -code development platforms,

    A. Sahay, A. Indamutsa, D. Di Ruscio, and A. Pierantonio, “Supporting the understanding and comparison of low -code development platforms,” in Proc. 46th Euromicro Conf. Softw. Eng. Adv. Appl. (SEAA), Portorož, Slovenia, 2020, pp. 171–178

    2020

  16. [16]

    Common method biases in behavioral research: A critical review of the literature and recommended remedies,

    P. M. Podsakoff, S. B. MacKenzie, J. -Y. Lee, and N. P. Podsakoff, “Common method biases in behavioral research: A critical review of the literature and recommended remedies,” J. Appl. Psychol., vol. 88, no. 5, pp. 879–903, Oct. 2003

    2003

  17. [17]

    The reliability of a two - item scale: Pearson, Cronbach, or Spearman –Brown?

    R. Eisinga, M. te Grotenhuis, and B. Pelzer, “The reliability of a two - item scale: Pearson, Cronbach, or Spearman –Brown?” Int. J. Public Health, vol. 58, no. 4, pp. 637–642, Aug. 2013

    2013

  18. [18]

    Coefficient alpha and the internal structure of tests,

    L. J. Cronbach, “Coefficient alpha and the internal structure of tests,” Psychometrika, vol. 16, no. 3, pp. 297–334, Sep. 1951

    1951

  19. [19]

    R. P. McDonald, Test Theory: A Unified Treatment . Mahwah, NJ, USA: Lawrence Erlbaum, 1999

    1999

  20. [20]

    A rationale and test for the number of factors in factor analysis,

    J. L. Horn, “A rationale and test for the number of factors in factor analysis,” Psychometrika, vol. 30, no. 2, pp. 179–185, Jun. 1965

    1965

  21. [21]

    Little Jiffy, Mark IV,

    H. F. Kaiser and J. Rice, “Little Jiffy, Mark IV,” Educ. Psychol. Meas., vol. 34, no. 1, pp. 111–117, Apr. 1974

    1974

  22. [22]

    A simple sequentially rejective multiple test procedure,

    S. Holm, “A simple sequentially rejective multiple test procedure,” Scand. J. Stat., vol. 6, no. 2, pp. 65–70, 1979

    1979

  23. [23]

    A power primer,

    J. Cohen, “A power primer,” Psychol. Bull., vol. 112, no. 1, pp. 155 – 159, Jul. 1992

    1992

  24. [24]

    Analyzing Likert Data

    H. N. Boone and D. A. Boone, “Analyzing Likert data,” J. Extension, vol. 50, no. 2, Art. 48, Apr. 2012, doi: 10.34068/joe.50.02.48

    work page doi:10.34068/joe.50.02.48 2012

  25. [25]

    Using thematic analysis in psychology,

    V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qual. Res. Psychol., vol. 3, no. 2, pp. 77–101, 2006

    2006