AI as Equalizer or Amplifier? Task Complexity as the Moderating Factor for Human Expertise in Hybrid Intelligence Systems

Tao An

REVIEW 4 major objections 4 minor 15 references

The paper argues that generative AI is a cognitive amplifier: output quality depends on the expertise of the human directing it, not on prompt-engineering skill.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 07:12 UTC pith:7JLXYREV

load-bearing objection A plausible and clearly written position paper whose central claim is under-supported: it never compares the expert–novice gap with AI to the gap without AI, and its own field observations are anecdotal and over-labeled as validation. the 4 major comments →

arxiv 2512.10961 v2 pith:7JLXYREV submitted 2025-10-30 cs.HC cs.AI

AI as Equalizer or Amplifier? Task Complexity as the Moderating Factor for Human Expertise in Hybrid Intelligence Systems

Tao An This is my paper

classification cs.HC cs.AI

keywords cognitive amplificationhuman-AI collaborationexpert-novice differencestask complexitysycophancyintelligence amplificationAI literacyworkforce development

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that generative AI tools are best understood as cognitive amplifiers, not equalizers: the quality of AI-assisted work depends primarily on the domain expertise and judgment of the human directing the tool, not on prompt-engineering skill. It reconciles conflicting findings by proposing that AI equalizes performance on well-structured routine tasks while amplifying pre-existing expert-novice differences on complex tasks that require deep judgment. The proposed mechanism is AI's sycophancy bias—the tendency to defer to whatever the user says—which makes expert corrections converge toward better outputs and novice misconceptions converge toward worse ones. If true, this reframes workforce training and system design: invest in judgment and domain knowledge rather than tool mechanics, and build AI that rewards, rather than replaces, expertise.

Core claim

On the paper's terms, the central claim is that 'AI functions primarily as a cognitive amplifier: a system whose output quality depends fundamentally on the expertise of the human who directs it.' Human contribution operates at three layers—problem definition, quality evaluation, and iterative refinement—and users engage at three levels: passive acceptance, iterative collaboration, and cognitive direction. Domain expertise determines effectiveness at each layer. The paper argues that the same sycophancy mechanism that makes AI agree with flawed premises also lets experts steer outputs rapidly; a novice who cannot evaluate outputs uses each AI interaction to entrench errors, creating a multip

What carries the argument

The central explanatory mechanism is sycophancy bias: large language models trained with reinforcement learning from human feedback tend to defer to user statements, updating roughly 2.5 times more strongly than Bayesian norms when challenged. The paper treats this as the property that makes AI an amplifier rather than a corrector. The supporting framework is the three-layer model of human contribution (problem definition, quality evaluation, iterative refinement) and the three-level engagement model (passive acceptance, iterative collaboration, cognitive direction), with the claim that movement between levels requires domain expertise and metacognition, not technical training.

Load-bearing premise

The central claim rests on the author's informal field observations of 580 training participants being representative and unbiased; if those observations are confounded by self-selection, task differences, or evaluation bias, the empirical backing reduces to two external studies that support only a weaker version.

What would settle it

A pre-registered experiment in which experts and novices use identical AI tools on tasks spanning low to high complexity, with outputs scored blind by independent raters; the amplification claim is falsified if the expert-novice gap does not grow with task complexity or if novices close the gap on complex tasks.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

The equalizer and amplifier findings in the literature are not contradictory; they describe two regimes selected by task complexity.
Workforce development should invest in domain expertise and evaluative judgment, not just prompt-engineering techniques.
AI systems should adapt to user expertise: scaffold novices, give experts direct control, and build friction into critical decisions.
Because of sycophancy, repeated AI interaction can narrow expert-novice gaps on routine tasks while widening them on complex judgment tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If sycophancy is the mechanism, then measuring a model's deference profile could predict where expert-novice gaps will appear in different tasks.
A direct test would randomize experts and novices to AI-assisted and unassisted conditions across low- and high-complexity tasks, with outputs scored blind; the paper's claim predicts an interaction between expertise and task complexity.
The argument implies that 'AI literacy' curricula should be re-centered on domain judgment and evaluation rather than tool mechanics.
The amplifier logic suggests organizations should track capability development longitudinally, not just productivity, to see whether AI use erodes or builds expertise.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A plausible and clearly written position paper whose central claim is under-supported: it never compares the expert–novice gap with AI to the gap without AI, and its own field observations are anecdotal and over-labeled as validation.

read the letter

You should know the basics: this is a position paper arguing that generative AI amplifies existing expertise rather than replacing it, and that task complexity decides whether AI equalizes or widens gaps. That framing is useful and probably right in spirit, but the paper's evidence is much thinner than its figures suggest.

What it does well: the synthesis of Sun et al. (2022) and the METR developer study is honest, the sycophancy mechanism is imported from credible published work, and the practical recommendations—train judgment, not prompt templates; keep humans in the loop—are sensible. The author also states plainly in §5.2 that the field observations lack controlled experimental design and cannot support causal claims. That candor should count for something.

The soft spots are real and load-bearing. The biggest is the baseline problem: to claim AI amplifies expertise, you need to show the expert–novice gap grows when AI is used, relative to human-only work. The paper never does that. Sun et al. compares experts and novices only in an AI-assisted setting; METR only studies experienced developers with and without AI but doesn't interact expertise with the treatment. Figure 1 draws a "Human Only (baseline)" curve, but it's explicitly conceptual. So the evidence is equally consistent with AI being a neutral mirror that preserves an existing gap. The author may believe amplification happens—I suspect it often does—but this manuscript doesn't demonstrate it.

Second, the field observations, presented with exact percentages and capability levels (Figures 3 and 5), have no measurement protocol, no baselines, no blind coding, and no inter-rater reliability. They're essentially recollections from the author's own training sessions, and Figure 7 claims the "three-layer framework" was "validated" at n=580—a few pages before the limitations section concedes no causal claims. That's an internal tension you can point to precisely. Third, the abstract and full text disagree on scope (small software team vs. 580 professionals; HHAI reconciliation vs. workforce-development amplifier position), and the novelty is limited because the three-layer framework restates the expert advantages in Sun et al.

For all that, the core direction is plausible, and the paper doesn't try to hide its limits entirely. It's a reading-group piece for people thinking about expertise and AI adoption, not a citable empirical result.

Would I send it to review? Yes, with the expectation of major revision. The baseline confound needs to be addressed or the claim scoped down, and the field observations should be described as illustrative, not validating. A serious referee could help the author turn a reasonable position piece into something defensible.

Referee Report

4 major / 4 minor

Summary. This position paper argues, from the author's training observations and two external studies, that generative AI acts as a 'cognitive amplifier' rather than an equalizer: output quality depends primarily on the user's domain expertise, not prompt-engineering skill. The author proposes three layers of human contribution (problem definition, quality evaluation, iterative refinement) and three levels of engagement (passive acceptance, iterative collaboration, cognitive direction), and reconciles the equalizer/amplifier debate by proposing that AI equalizes on routine tasks and amplifies on complex tasks, with sycophancy as the mechanism. The paper includes recommendations for workforce development and AI system design and closes with future research questions and limitations.

Significance. If the central claim were established, the paper would make a useful contribution: it offers a clear reconciliation of seemingly contradictory findings in the human-AI collaboration literature, and its practical recommendations (invest in domain expertise and judgment, not just prompt engineering) are actionable and sensible. The paper also usefully connects the amplifier framing to the sycophancy literature, and its emphasis on expertise-sensitive design is timely for the HHAI community. However, as presented, the empirical support is far weaker than the abstract's language of 'demonstrating' and Figure 7's 'validated' suggests. The external studies cited support the weaker claim that experts and novices differ when both use AI, not that AI amplifies pre-existing differences relative to a human-only baseline. The author's own field observations lack the methodological detail needed to support the quantitative claims (e.g., +45% vs. +20% gains, 30/60/95 capability levels), and the framework itself is derived from and then 'validated' on the same observations. The paper's value is therefore as a hypothesis-generating position piece, not as an empirical demonstration. This distinctio

major comments (4)

[Abstract and §1] The abstract states the paper 'demonstrates' that domain expertise determines amplification effectiveness, and §1 says this insight is 'validated through both research evidence and field observation.' But the evidence cited does not establish amplification: Sun et al. (2022) compares experts and novices only in an AI-assisted setting, with no human-only condition; METR (2025) studies only experienced developers; and the field observations (Figures 3, 5, 7) report no baseline or no-AI control. An expert–novice gap in AI-assisted performance is consistent with AI being a neutral mirror that preserves a pre-existing gap. To support the amplifier claim, the paper needs a crossed design (expertise × AI vs. no-AI) or a before/after measurement. As written, the central claim is underdetermined by the cited evidence.
[§3.2, Figures 3, 5, 7] The quantitative core of the paper rests on the author's own field observations, but no methodology is disclosed: how 'performance improvement' was measured, what the baseline was, how outputs were scored, whether evaluation was blind, or whether any inter-rater reliability was computed. The author's dual role as trainer and evaluator is a potential bias. Figures 3 and 5 report specific numbers (+45%, +35%, +20%, 30/60/95, etc.) as if they were measurements, but the note only says 'based on systematic field observations.' The 'Validated three-layer framework (n=580)' milestone in Figure 7 is circular: §3.1 states the framework was developed 'based on ... systematic observation across professional training contexts,' and the same observations are then used to validate it. §5.2 concedes the observations 'lack the controlled experimental design needed for causal claims,' which directly cont
[§3.4] The sycophancy mechanism is central to the paper's explanation of why AI amplifies rather than corrects: expert feedback converges rapidly, novice feedback entrenches errors. The cited studies (§3.4, refs [12]–[14]) support the existence of sycophancy and overconfidence/underconfidence biases, but none measures the differential convergence or entrenchment rates of expert versus novice iterative refinement. The anecdotal examples ('AI keeps agreeing with me but the results keep getting worse'; the junior analyst's 'not being bullish enough' critique) are illustrative but cannot carry the quantitative 'multiplicative effect on the expert-novice gap' claim. This is a potential mechanism, not an established one; the paper should label it as a hypothesis and propose a concrete test.
[§5.2] The limitations section appropriately concedes the lack of controlled design, but it does not resolve the mismatch with the paper's strong claims. More importantly, it says field observations provide 'valuable real-world validation of research findings,' yet no specific validation protocol is described. Given that the observations are the sole source for the framework's empirical percentages and the 'validated' milestone, the paper should either (a) substantially weaken the epistemic claims throughout (e.g., replace 'demonstrating' with 'proposing,' 'validated' with 'consistent with'), or (b) provide the missing methodological detail, including instruments, scoring rubrics, and any available reliability checks. Without one of these, the current gap between rhetoric and evidence is too large for a journal article.

minor comments (4)

[References] Several references are non-archival or informal: [2] is a Wikipedia page, [6] is a CircleCI blog, [7] and [8] are blog/arXiv preprints without author or venue information, and [10] lacks a venue. This is acceptable for a position paper but should be cleaned up, and the Nature reference [13] should give the exact journal title (npj Digital Medicine) and authors.
[Terminology] The paper uses 'three layers' (human contribution) and 'three levels' (engagement) with similar names; Figures 4 and 5 help, but the text would benefit from a summary table distinguishing Layer 1–3 from Level 1–3, since the proximity of 'Layer 3: Iterative Refinement' and 'Level 3: Cognitive Direction' invites confusion.
[Figure 1] Figure 1 is labeled 'Conceptual diagram,' which is good, but the curve labeled 'Human Only (baseline)' is not referenced anywhere in the text and no data or citation supports its shape. Either remove it or explicitly state that it is a placeholder illustrating the hypothesis, not an empirical result.
[§3.2, 'From training observation'] The first-person anecdotes are engaging but often unverifiable. Since the paper already concedes limited generalizability, consider moving these to a clearly marked 'illustrative examples' subsection or table so they are not read as evidence.

Circularity Check

1 steps flagged

Three-layer framework is 'validated' by the same field observations from which it was constructed.

specific steps

self definitional [Introduction (§1), §3.1–3.2, Figure 7]
"The critical insight, validated through both research evidence and field observation, is that domain expertise, judgment capability, and iterative refinement skills—not technical facility with AI—determine amplification effectiveness. ... Validated three-layer framework (n=580). Findings consistently confirmed that domain expertise, not technical skill, predicts AI effectiveness across all industries."

The framework's content (domain expertise, judgment, iterative refinement as the three layers/levels) is introduced in §3.1 as proposed 'based on empirical evidence and systematic observation' and in §3.2 via 'training observation.' Figure 7 then lists the same observations as the milestone 'Validated three-layer framework (n=580)' and says the findings 'consistently confirmed' the framework. The validation is therefore the construction restated: no independent dataset, no pre-registered prediction, no held-out comparison. §5.2 concedes the observations 'lack the controlled experimental design needed for causal claims.' This is a self-confirmation loop rather than an external test.

full rationale

The core expertise-amplification hypothesis is not wholly circular: it is independently supported in weaker form by Sun et al. (2022) and METR (2025), and the sycophancy mechanism is grounded in external literature. However, the paper's claim to have 'validated' its three-layer framework rests on the author's own 580-participant field observations, which were also the source from which the framework was induced. Figure 7's 'Validated three-layer framework (n=580)' is thus a restatement of the input rather than a prediction. The abstract's 'demonstrating' and Figure 7's 'validated' language overstate what this self-referential evidence can support, especially since §5.2 admits the observations lack controlled experimental design. The absence of a human-only baseline and the undisclosed scoring criteria are additional evidentiary weaknesses, but they are inferential gaps rather than definitional circularity, and they do not increase the score beyond the framework-validation loop. Overall score 5 reflects partial circularity: the framework validation reduces to its own inputs, while the central differential-effectiveness claim retains independent external content.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The paper's central model is built from two constructs (three contribution layers, three engagement levels) plus a moderation thesis (routine tasks equalize, complex tasks amplify). The moderation thesis borrows from the equalizer/jagged-frontier literature that the paper does not cite; the three layers borrow from the paper's own cited reference [4]; the three levels are a re-branded compression of the classic levels-of-automation taxonomy. What is genuinely the paper's own is the observational dataset (n=580), which is also exactly what is not verifiable: no protocol, no raw data, no scoring rules, no inferential statistics. The free parameters are the unmeasured percentages that the figures present as measured. The axioms are the domain assumptions needed to generalize from two cited studies and from the author's own training sessions to a universal amplifier law. No new physical entities are invented; the framework constructs are the only 'invented' items and neither has an independent falsifiable handle in this paper.

free parameters (3)

Per-role performance gains from field training (technical experts +45%, middle management +35%, executives +30%, student = Figure 3: +45%/+35%/+30%/+25%/+20%
Plotted as measured outcomes of the author's training observations, but no measurement instrument, baseline, scoring rubric, or statistical test is described anywhere in the text; the numbers function as asserted values that the framework is claimed to 'validate.'
Capability-level percentages (prompt quality / output quality / user judgment across the three engagement levels) = Figure 5 caption: 30/60/90, 30/65/95, 20/60/95 (approximate)
Ad hoc illustrative values for the three engagement levels, labeled 'relative capabilities based on systematic field observations' with no measurement basis disclosed.
Training cohort counts (n=50 → 580 across 2023–2025) = n = 50 → 150 → 300 → 450 → 580 (Figure 7)
Self-reported enrollment numbers across five role categories; unverifiable without training records, and the abstract's description ('a small software product team') does not match the full text's 580 professionals across multiple corporations.

axioms (4)

domain assumption LLMs trained on broad internet data necessarily produce statistically generic ('mediocre') outputs without expert direction
Stated §3.3 as 'a fundamental characteristic of systems trained on averaged data' and used as the premise for why human direction is required; asserted without proof and contested for reasoning-capable models.
domain assumption The author's corporate-training observations are a valid measure of real-world expert–novice performance differences
The entire empirical load of the framework rests on this; §5.2 concedes the observations 'lack the controlled experimental design needed for causal claims' and 'may not generalize.'
domain assumption Sycophancy findings from the cited medical and general-domain studies transfer unchanged to the author's training contexts
The near-100% compliance and ~2.5x Bayesian-overweighting results [12,13,14] are borrowed wholesale and used as the mechanism for the amplification claim; the domain transfer is assumed, not demonstrated.
domain assumption Expertise effects from Sun et al. (conversational-agent design) and METR (open-source software) generalize to writing, analytics, and market research
The universal amplifier law is inferred from two cited task domains; generalization across knowledge-work domains is asserted rather than derived.

invented entities (2)

Three-level engagement model (passive acceptance / iterative collaboration / cognitive direction) no independent evidence
purpose: Classifies modes of human–AI interaction claimed to explain amplification effectiveness; movement between levels is claimed to require domain expertise and metacognition, not technical training.
No test is attached to the levels (Figure 5's percentages are illustrative); the construct re-describes the long-standing human-automation levels-of-control idea under a new name.
Three layers of human contribution (problem definition / quality evaluation / iterative refinement) no independent evidence
purpose: Decomposes the human input that determines AI output quality; 'quality evaluation' is called 'the single most critical bottleneck.'
Directly reproduces the expert advantages already documented in the cited Sun et al. [4]; presented as a new framework without independent measurement.

pith-pipeline@v1.3.0-alltime-deepseek · 7615 in / 22504 out tokens · 201968 ms · 2026-08-04T07:12:02.882840+00:00 · methodology

0 comments

read the original abstract

A growing body of empirical research suggests that generative AI narrows performance gaps between novice and expert workers on routine tasks--the so-called "equalizer" effect. This paper challenges the generality of that conclusion. Drawing on cognitive augmentation theory, expert-novice research, and structured observations of in-house generative-AI use across a small software product team, we argue that AI functions primarily as a cognitive amplifier: a system whose output quality depends fundamentally on the expertise of the human who directs it. We present a framework comprising three layers of human contribution (problem definition, quality evaluation, iterative refinement) and three levels of engagement (passive acceptance, iterative collaboration, cognitive direction), demonstrating that domain expertise--not prompt engineering skill--determines amplification effectiveness. We reconcile the equalizer and amplifier perspectives by proposing that AI equalizes performance on well-structured, routine tasks while amplifying pre-existing differences on complex tasks requiring deep judgment. This reconciliation carries direct implications for hybrid human-AI system design: rather than building AI that replaces expertise, we should build AI that rewards and develops it. We outline a research agenda for the HHAI community centered on expertise-sensitive AI design, adaptive collaboration interfaces, and longitudinal studies of human capability development in AI-augmented work.

Figures

Figures reproduced from arXiv: 2512.10961 by Tao An.

**Figure 2.** Figure 2: Expert vs. Novice Workflow Comparison. Experts engage in a systematic cycle of thoughtful [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: Field Training Observations (2023-2025). Data from systematic observations of 580 professionals [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: Three Layers of Human Contribution to AI-Assisted Work. Domain expertise determines effec [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

**Figure 5.** Figure 5: Three Levels of AI Engagement. Movement between levels requires development of domain exper [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

**Figure 6.** Figure 6: The Sycophancy Problem: A Dangerous Cycle. When novices hold misconceptions (Step 1), AI [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

**Figure 7.** Figure 7: Field Training Timeline (2023-2025). Systematic observations progressed from an initial co [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

15 extracted references · 3 linked inside Pith

[1]

Augmenting human in- tellect: A conceptual framework

Douglas C Engelbart. Augmenting human in- tellect: A conceptual framework. Technical re- port, Stanford Research Institute, 1962

1962
[2]

Augmented cognition, 2025

Wikipedia. Augmented cognition, 2025. URLhttps://en.wikipedia.org/wiki/ Augmented_cognition

2025
[3]

Brain augmentation and neuroscience tech- nologies.Frontiers in Human Neuroscience,
[4]

Comparing ex- perts and novices for ai data work

Lining Sun, Yiren Liu, Geena Joseph, Zeyu Yu, Haiyi Zhu, and Steven P Dow. Comparing ex- perts and novices for ai data work. InProceed- ings of the AAAI Conference on Human Com- putation and Crowdsourcing, volume 10, pages 195–206, 2022

2022
[5]

Measuring the impact of early-2025 ai on experienced open-source developer pro- ductivity

METR. Measuring the impact of early-2025 ai on experienced open-source developer pro- ductivity. Technical report, 2025. URLhttps: //metr.org/blog/2025-07-10-early-2025- ai-experienced-os-dev-study/

2025
[6]

Prompt engineering: A guide to im- proving llm performance, 2024

CircleCI. Prompt engineering: A guide to im- proving llm performance, 2024. URLhttps:// circleci.com/blog/prompt-engineering/

2024
[7]

URLhttps://arxiv

Does prompt formatting have any impact on llm performance?, 2024. URLhttps://arxiv. org/html/2411.10541v1

Pith/arXiv arXiv 2024
[8]

The impact of prompt bloat on llm output quality, 2025

MLOps Community. The impact of prompt bloat on llm output quality, 2025. URL https://mlops.community/the-impact-of- prompt-bloat-on-llm-output-quality/

2025
[9]

Ai liter- acy and its implications for prompt engineering strategies.International Journal of Educational Technology in Higher Education, 21(1), 2024

Olaf Zawacki-Richter, Victoria I Marín, Melissa Bond, and Franziska Gouverneur. Ai liter- acy and its implications for prompt engineering strategies.International Journal of Educational Technology in Higher Education, 21(1), 2024

2024
[10]

Chatgpt in higher educa- tion: Considerations for academic integrity and student learning

Debby RE Cotton, Peter A Cotton, and J Reuben Shipway. Chatgpt in higher educa- tion: Considerations for academic integrity and student learning. 2023

2023
[11]

Chatgptunveiled: Un- derstanding perceptions of academic integrity in higher education.Journal of Academic Ethics, 2024

Reyhaneh Hashemi Mogavi, Chong Deng, Jae- woo John Kim, Pan Zhou, Kyung Sun Kwon, andBradMehlenbacher. Chatgptunveiled: Un- derstanding perceptions of academic integrity in higher education.Journal of Academic Ethics, 2024

2024
[12]

URLhttps://arxiv

Sycophancy in large language models: Causes and mitigations, 2024. URLhttps://arxiv. org/abs/2411.15287. 10

Pith/arXiv arXiv 2024
[13]

URL https://www.nature.com/articles/s41746- 025-02008-z

When helpfulness backfires: Llms and the risk of false medical information due to sycophantic behavior.Nature Digital Medicine, 2025. URL https://www.nature.com/articles/s41746- 025-02008-z

2025
[14]

URL https://arxiv.org/abs/2507.03120

How overconfidence in initial choices and un- derconfidence under criticism modulate change of mind in large language models, 2025. URL https://arxiv.org/abs/2507.03120. 11

Pith/arXiv arXiv 2025
[2022]

URLhttps://pmc.ncbi.nlm.nih.gov/ articles/PMC9538357/

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.

[1] [1]

Augmenting human in- tellect: A conceptual framework

Douglas C Engelbart. Augmenting human in- tellect: A conceptual framework. Technical re- port, Stanford Research Institute, 1962

1962

[2] [2]

Augmented cognition, 2025

Wikipedia. Augmented cognition, 2025. URLhttps://en.wikipedia.org/wiki/ Augmented_cognition

2025

[3] [3]

Brain augmentation and neuroscience tech- nologies.Frontiers in Human Neuroscience,

[4] [4]

Comparing ex- perts and novices for ai data work

Lining Sun, Yiren Liu, Geena Joseph, Zeyu Yu, Haiyi Zhu, and Steven P Dow. Comparing ex- perts and novices for ai data work. InProceed- ings of the AAAI Conference on Human Com- putation and Crowdsourcing, volume 10, pages 195–206, 2022

2022

[5] [5]

Measuring the impact of early-2025 ai on experienced open-source developer pro- ductivity

METR. Measuring the impact of early-2025 ai on experienced open-source developer pro- ductivity. Technical report, 2025. URLhttps: //metr.org/blog/2025-07-10-early-2025- ai-experienced-os-dev-study/

2025

[6] [6]

Prompt engineering: A guide to im- proving llm performance, 2024

CircleCI. Prompt engineering: A guide to im- proving llm performance, 2024. URLhttps:// circleci.com/blog/prompt-engineering/

2024

[7] [7]

URLhttps://arxiv

Does prompt formatting have any impact on llm performance?, 2024. URLhttps://arxiv. org/html/2411.10541v1

Pith/arXiv arXiv 2024

[8] [8]

The impact of prompt bloat on llm output quality, 2025

MLOps Community. The impact of prompt bloat on llm output quality, 2025. URL https://mlops.community/the-impact-of- prompt-bloat-on-llm-output-quality/

2025

[9] [9]

Ai liter- acy and its implications for prompt engineering strategies.International Journal of Educational Technology in Higher Education, 21(1), 2024

Olaf Zawacki-Richter, Victoria I Marín, Melissa Bond, and Franziska Gouverneur. Ai liter- acy and its implications for prompt engineering strategies.International Journal of Educational Technology in Higher Education, 21(1), 2024

2024

[10] [10]

Chatgpt in higher educa- tion: Considerations for academic integrity and student learning

Debby RE Cotton, Peter A Cotton, and J Reuben Shipway. Chatgpt in higher educa- tion: Considerations for academic integrity and student learning. 2023

2023

[11] [11]

Chatgptunveiled: Un- derstanding perceptions of academic integrity in higher education.Journal of Academic Ethics, 2024

Reyhaneh Hashemi Mogavi, Chong Deng, Jae- woo John Kim, Pan Zhou, Kyung Sun Kwon, andBradMehlenbacher. Chatgptunveiled: Un- derstanding perceptions of academic integrity in higher education.Journal of Academic Ethics, 2024

2024

[12] [12]

URLhttps://arxiv

Sycophancy in large language models: Causes and mitigations, 2024. URLhttps://arxiv. org/abs/2411.15287. 10

Pith/arXiv arXiv 2024

[13] [13]

URL https://www.nature.com/articles/s41746- 025-02008-z

When helpfulness backfires: Llms and the risk of false medical information due to sycophantic behavior.Nature Digital Medicine, 2025. URL https://www.nature.com/articles/s41746- 025-02008-z

2025

[14] [14]

URL https://arxiv.org/abs/2507.03120

How overconfidence in initial choices and un- derconfidence under criticism modulate change of mind in large language models, 2025. URL https://arxiv.org/abs/2507.03120. 11

Pith/arXiv arXiv 2025

[15] [2022]

URLhttps://pmc.ncbi.nlm.nih.gov/ articles/PMC9538357/