Epistemic reflections on AI answering our questions: overwatch, erudite, logician, interlocutor

Ella-Jenna Oosterglorenwoud; Johan F. Hoorn

arxiv: 2304.14352 · v2 · submitted 2023-04-23 · 💻 cs.CY · cs.AI· cs.LO

Epistemic reflections on AI answering our questions: overwatch, erudite, logician, interlocutor

Johan F. Hoorn , Ella-Jenna Oosterglorenwoud This is my paper

Pith reviewed 2026-05-24 09:14 UTC · model grok-4.3

classification 💻 cs.CY cs.AIcs.LO

keywords AI relianceGrice maximplagiarism detectionType II erroraffirming the consequentobserver effectLLM trustworthinessepistemic violation

0 comments

The pith

Careless reliance on AI to answer questions or judge output violates Grice's Maxim of Quality and Lemoine's Maxim of Innocence.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper contends that people increasingly turn to large language models for advice in finance, law, and medicine, often accepting responses without logical checks or empirical validation. This practice is framed as breaching Grice's standard for truthful communication and Lemoine's legal protection against unfounded guilt. The authors focus on how plagiarism scanners can be set with AI output as the default null, so that a failure to find differences gets misinterpreted as proof of AI authorship through the affirming-the-consequent fallacy. They argue this setup produces false accusations and that LLMs need specific inference systems integrated before they can serve as reliable partners. The discussion closes by noting that uncertainty and classification are already shaped by the observer's beliefs rather than by the AI output itself.

Core claim

Careless reliance on AI to answer our questions and to judge our output is a violation of Grice's Maxim of Quality as well as a violation of Lemoine's legal Maxim of Innocence. A low-sensitivity plagiarism scanner may produce a Type II error by failing to detect difference (the null hypothesis wrongly maintained). The fallacy of affirming the consequent occurs when the failure to detect difference is then interpreted as evidence of equivalence or demonstration of AI authorship. If the test is specified so that 'AI-generated' is effectively treated as the default H0, then a finding of 'no difference from AI' is taken as support for that null. Such a mis-specified test results in studentsbeing

What carries the argument

The mis-specified statistical test for AI authorship that sets 'AI-generated' as the default null hypothesis, turning Type II errors into misinterpreted evidence of AI authorship via affirming the consequent.

If this is right

Students can be treated as guilty of AI use or plagiarism unless they produce detectable differences from AI output.
Unverified acceptance of AI advice in medical, legal, and financial domains constitutes an epistemic violation.
LLMs require integrated inference systems to avoid becoming an uncontrolled 'sorcerer's apprentice'.
Classification and interpretation of any output already depend on the observer's belief system and tolerance for ambiguity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Detection protocols in education could be revised to place the burden on proving AI use rather than on proving human authorship.
The same null-hypothesis reversal risk may appear in automated systems for code review or image authenticity checks.
Explicit guidelines on when to accept or verify AI output could reduce the identified maxim violations.
The observer-effect observation points to similar belief-driven filtering in any human-AI judgment loop.

Load-bearing premise

The statistical test for AI authorship is routinely mis-specified by setting 'AI-generated' as the default null hypothesis, so that failure to detect difference is interpreted as positive evidence of AI authorship.

What would settle it

An audit of standard plagiarism-detection software showing whether 'AI-generated' is in fact set as the default null or whether 'no difference' findings are treated as affirmative evidence of AI use.

Figures

Figures reproduced from arXiv: 2304.14352 by Ella-Jenna Oosterglorenwoud, Johan F. Hoorn.

read the original abstract

Currently, there is a trend for the wider public to rely on LLMs for financial or legal consultation, medical and mental support (Chatterji et al., 2025), often accepting the advice provided without necessarily seeking logical verification or empirical validation. While one might be fortunate enough to encounter a model with a particularly solid 'ground truth' or with auxiliary logic-symbolic reasoning capabilities, it remains a somewhat uncertain endeavour. Output is simply taken at face value, without further question. Yet, careless reliance on AI to answer our questions and to judge our output is a violation of Grice's Maxim of Quality as well as a violation of Lemoine's legal Maxim of Innocence. A low-sensitivity plagiarism scanner may produce a Type II error by failing to detect difference (the null hypothesis wrongly maintained). The fallacy of affirming the consequent occurs when the failure to detect difference is then interpreted as evidence of equivalence or demonstration of AI authorship. If the test is specified so that 'AI-generated' is effectively treated as the default H0, then a finding of 'no difference from AI' is taken as support for that null. Such a mis-specified test results in students being treated as guilty (AI/plagiarism) unless suspects can generate sufficient detectable difference from AI output, which yields false accusations under a fair null hypothesis (that the student wrote the work). To avoid LLMs becoming a sorcerer's apprentice, knowledge is required about which inference systems are or should become integrated for an LLM to become a trustworthy sparring partner. We end on a wider perspective where the formalisation of the observer effect shows that uncertainty, classification, and interpretation are already shaped by the human or artificial agency's belief system, affective state, and tolerance for ambiguity, rather than at the stage of LLM output.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This is a short commentary applying Grice and basic stats to LLM reliance, but the key detector example stays hypothetical.

read the letter

The main takeaway is that the paper flags risks in taking LLM answers at face value and in using detectors to judge student work, framing both as violations of Grice's quality maxim and Lemoine's innocence maxim. It adds the observer effect to note that beliefs shape how output gets read. That is the extent of it. No new framework or data appears. The text is clear on the affirming-the-consequent point and on why a low-sensitivity test can produce misleading results when the null is set a certain way. Those are standard reminders applied to current tools. The paper does not claim to derive anything from first principles or run any checks. The soft spot sits in the central illustration. The argument treats the mis-specification of 'AI-generated' as default null as a live problem that leads to false accusations, yet the text gives no citations, no tool documentation, and no examples showing that common scanners actually run this way. Without that grounding the scenario stays conditional rather than demonstrated. The closing call for better inference systems in LLMs is reasonable but also general. Readers already following AI-ethics or education-policy discussions might pick up a quick logical angle here. The piece does not organize new phenomena or test claims, so it does not look like something that needs referee time. I would not route it to peer review.

Referee Report

1 major / 1 minor

Summary. The paper claims that careless public reliance on LLMs for financial, legal, medical, or mental-health advice, and for judging human output, violates Grice's Maxim of Quality and Lemoine's Maxim of Innocence. The central illustration is a statistical scenario in which AI detectors or plagiarism scanners are mis-specified by treating 'AI-generated' as the default null hypothesis H0; failure to detect difference is then misinterpreted (via affirming the consequent) as positive evidence of AI authorship, producing false accusations. The manuscript introduces four epistemic roles (overwatch, erudite, logician, interlocutor) as a framework for trustworthy AI interaction and closes with a reflection on the observer effect shaping classification and interpretation.

Significance. If the central claims hold, the work would usefully connect standard logical and statistical fallacies to concrete risks in AI-mediated epistemic practices. The four-role taxonomy offers a potentially generative framing for AI as a 'sparring partner,' though it is introduced without formal definitions or derivations. The paper supplies no empirical data, formal derivations, error analyses, or citations documenting the prevalence of the claimed H0 mis-specification, which limits its contribution to the evidentiary base of the field.

major comments (1)

[Abstract] Abstract (paragraph beginning 'A low-sensitivity plagiarism scanner...'): The concrete scenario used to illustrate the maxim violations rests on the premise that 'if the test is specified so that AI-generated is effectively treated as the default H0' then failure to detect difference is taken as support for AI authorship. No citations, tool documentation, empirical examples, or references to common detectors are supplied to establish that this null specification is routine. This premise is load-bearing for the central claim that such practices constitute violations of Grice's and Lemoine's maxims.

minor comments (1)

[Abstract] The four roles (overwatch, erudite, logician, interlocutor) are named in the title and abstract but receive no operational definitions or explicit linkage to the maxim-violation argument; a brief clarifying subsection would improve readability.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive comments. We respond to the single major comment below.

read point-by-point responses

Referee: [Abstract] Abstract (paragraph beginning 'A low-sensitivity plagiarism scanner...'): The concrete scenario used to illustrate the maxim violations rests on the premise that 'if the test is specified so that AI-generated is effectively treated as the default H0' then failure to detect difference is taken as support for AI authorship. No citations, tool documentation, empirical examples, or references to common detectors are supplied to establish that this null specification is routine. This premise is load-bearing for the central claim that such practices constitute violations of Grice's and Lemoine's maxims.

Authors: The scenario is presented as a constructed logical illustration of affirming the consequent under a mis-specified null, not as an empirical assertion that this H0 choice is routine across detectors. The manuscript is an epistemic reflection rather than an empirical study and therefore supplies no prevalence data or tool-specific documentation. We will revise the abstract to state explicitly that the example is illustrative of the logical structure and its epistemic consequences, thereby removing any implication of documented routine practice while preserving the connection to the maxim violations. revision: yes

Circularity Check

0 steps flagged

No significant circularity; arguments rely on external citations without self-referential reduction

full rationale

The paper advances philosophical claims about maxim violations and statistical mis-specification of AI detectors. These rest on citations to Grice and Lemoine plus an illustrative scenario about H0 specification, but contain no equations, fitted parameters, or derivations that reduce to the paper's own inputs by construction. No self-citation chains, ansatzes smuggled via prior work, or renamings of known results appear. The central illustration is unsupported by evidence in the text, but that is a correctness/empirical issue rather than circularity per the enumerated patterns. The derivation chain is self-contained against external benchmarks and does not exhibit any of the six flagged reduction types.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 1 invented entities

The paper relies on standard philosophical maxims and the observer effect as background assumptions without introducing fitted parameters or new entities with independent evidence.

axioms (3)

domain assumption Grice's Maxim of Quality applies directly to LLM outputs in human-AI consultation
Invoked to classify uncritical acceptance as a violation.
domain assumption Lemoine's legal Maxim of Innocence applies to AI authorship detection
Used to frame false accusations as legal/ethical problems.
domain assumption The formalisation of the observer effect already shapes classification at the level of human or artificial belief systems
Stated in the final sentence as the wider perspective.

invented entities (1)

overwatch, erudite, logician, interlocutor no independent evidence
purpose: Proposed categories or roles for AI systems to become trustworthy sparring partners
Listed in the title as potential inference systems to integrate; no independent evidence supplied in abstract.

pith-pipeline@v0.9.0 · 5873 in / 1598 out tokens · 31536 ms · 2026-05-24T09:14:15.373975+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

5 extracted references · 5 canonical work pages

[1]

Donath, J. (2007). Signals in social supernets. Journal of Computer‐Mediated Communication, 13(1), 231-251. doi: 10.1111/j.1083-6101.2007.00394.x Feigenbaum, E. A. (2003). Some challenges and grand challenges for computational intelligence. Journal of the Association for Computing Machinery, 50, 32-40. Fellmeth, A. X., & Horwitz, M. (2009). Guide to Latin...

work page doi:10.1111/j.1083-6101.2007.00394.x 2007
[2]

(1961/1939)

Jeffreys, H. (1961/1939). Theory of probability. Oxford, UK: Clarendon. Krügel, S., Ostermaier, A., & Uhl, M. (2023). ChatGPT’s inconsistent moral advice influences users’ judgment. Nature: Scientific Reports, 13(1),

work page 1961
[3]

Lindquist, E. F. (1940). Statistical analysis in educational research. Boston, MA: Houghton Mifflin. 8 Loftus, E. F. (1993). The reality of repressed memories. The American Psychologist, 48(5), 518-

work page 1940
[4]

Neyman, J., & Pearson, E. S. (1933). IX. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231(694-706), 289-337. Pennington, K. (2003). Innocent until proven guilty: The origins of a legal maxim. Jurist, 63,

work page 1933
[5]

__main__

Searle, J. R. (1980) Minds, brains, and programs. Behavioral Brain Science, 3(3), 417-457. Turing, A. M. (1950). I.—Computing machinery and intelligence. Mind, LIX(236), 433-460. doi: 10.1093/mind/LIX.236.433 Ullmann, W. (1950). The defense of the accused in the medieval inquisition. The Irish Ecclesiastical Record, 73, 481-489. Walther, J. B., Van der He...

work page doi:10.1093/mind/lix.236.433 1980

[1] [1]

Donath, J. (2007). Signals in social supernets. Journal of Computer‐Mediated Communication, 13(1), 231-251. doi: 10.1111/j.1083-6101.2007.00394.x Feigenbaum, E. A. (2003). Some challenges and grand challenges for computational intelligence. Journal of the Association for Computing Machinery, 50, 32-40. Fellmeth, A. X., & Horwitz, M. (2009). Guide to Latin...

work page doi:10.1111/j.1083-6101.2007.00394.x 2007

[2] [2]

(1961/1939)

Jeffreys, H. (1961/1939). Theory of probability. Oxford, UK: Clarendon. Krügel, S., Ostermaier, A., & Uhl, M. (2023). ChatGPT’s inconsistent moral advice influences users’ judgment. Nature: Scientific Reports, 13(1),

work page 1961

[3] [3]

Lindquist, E. F. (1940). Statistical analysis in educational research. Boston, MA: Houghton Mifflin. 8 Loftus, E. F. (1993). The reality of repressed memories. The American Psychologist, 48(5), 518-

work page 1940

[4] [4]

Neyman, J., & Pearson, E. S. (1933). IX. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231(694-706), 289-337. Pennington, K. (2003). Innocent until proven guilty: The origins of a legal maxim. Jurist, 63,

work page 1933

[5] [5]

__main__

Searle, J. R. (1980) Minds, brains, and programs. Behavioral Brain Science, 3(3), 417-457. Turing, A. M. (1950). I.—Computing machinery and intelligence. Mind, LIX(236), 433-460. doi: 10.1093/mind/LIX.236.433 Ullmann, W. (1950). The defense of the accused in the medieval inquisition. The Irish Ecclesiastical Record, 73, 481-489. Walther, J. B., Van der He...

work page doi:10.1093/mind/lix.236.433 1980