Adversarial Social Epistemology for Assemblies of Humans and Large Language Models

Joel A.C. Baum; Mihnea C. Moldoveanu

REVIEW 3 major objections 6 minor 2 references

Adversarial Social Epistemology for Assemblies of Humans and Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Epistemic failures on platforms and with language models are strategic exploits of the commitments that make scaffolded public assertions trustworthy, not merely bubbles or misinformation diffusion.

desk verdict Useful conceptual package that reframes LLM failures as triadic commitment-evasion, with a clean mechanism lexicon; the human–machine transfer and the three machines stay promissory. read the letter →

arxiv 2607.07760 v1 pith:6XQDUB2Y submitted 2026-07-08 cs.AI cs.SI

Mihnea C. Moldoveanu , Joel A.C. Baum This is my paper · ORCID

classification cs.AIcs.SI

keywords adversarialsocialepistemologyepistemicnetworksinferentialismscaffoldedassertiontriadiccommunicationtrustlargelanguagemodelshallucinations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that familiar accounts of epistemic bubbles, echo chambers, and misinformation miss the local strategic problem: public claims rest on long upstream chains of testimony and inference and create downstream commitments, and trust is confidence that those commitments could be redeemed if entitled questions were asked. In minimally triadic exchanges—a sender addressing a recipient before observers—speakers can use poses, demagogical baits, social-proof covers, smokescreens, plausible ambiguation, decoys, flares, rabbit holes, and haystacks to block, defer, or raise the cost of redeeming those commitments. The same apparatus applies to large language models: hallucinations, sycophancy, flurries, and short-circuits are machine-regime variants of failing to track and redeem communicative commitments under evaluator pressure. The authors supply epistemic networks plus an inferentialist semantics of assertion, catalogue the anti-veritistic mechanisms, and sketch three machines—a Brandom machine for commitments and entitlements, a Hintikka machine for interrogative paths, and a Ramsey machine for non-veritistic payoffs—plus checklist-based training moves that aim to make deception more expensive than disclosiveness.

What carries the argument

Epistemic networks (epinets) enriched with interactive belief and an inferentialist semantics of assertion, which treat trust as strong belief in the redeemability of upstream and downstream commitments under Condition T—that public exchanges are minimally triadic (sender, recipient, observers).

What would settle it

Instrument public exchanges and LLM evaluations so that unpaid entitled questions, evasion types, and evaluator rewards are logged: if the listed adversarial moves do not systematically block redemption of commitments, or if checklist-based training and Brandom/Hintikka/Ramsey-style tools fail to make unpaid inferential debt more visible and costly than fluent evasion, the central claim does not hold.

Watch

Extended reading notes

Core claim

What requires explanation is how communicative agents exploit the commitments and entitlements that normally make scaffolded assertions trustworthy. Trust is not mere confidence in a speaker’s reliability but confidence that the speaker could redeem the relevant upstream and downstream commitments were entitled interlocutors to ask the questions the assertion makes available. Adversarial social epistemology analyzes those exploits in triadic public settings for humans, machines, and mixed assemblies, and outlines machinery to audit and partly redress them.

Load-bearing premise

The load-bearing premise is that the same commitment-and-entitlement analysis of trust, and the same triadic observer dynamics, successfully describe both human social platforms and large-language-model training and interaction, and that the sketched machines and checklists can put the theory to work on usable timescales.

Editorial extensions

If this is right

Platforms and interfaces that pin assertions, highlight entitled questions, and show disclosure trails can raise the cost of evasion relative to disclosiveness.
LLM training and evaluation can be redesigned around checklists that penalize haystacks, sycophancy, short-circuits, and related commitment failures rather than only binary correctness or fluency.
Mixed human–machine networks can be audited with machines that track commitments, generate interrogative paths, and reconstruct non-veritistic payoffs.
Mean-field models of bubbles and diffusion miss the local strategic exploits that turn observers into substitutes for warrant.
Scientific peer review and fast social or model-mediated talk become comparable once both are treated as scaffolded assertion under observer effects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If Condition T is the right unit of analysis, moderation metrics that only count checkable falsehoods will systematically underrate successful evasion that never asserts a clear error.
The evaluator question bank could be exposed as a public audit layer on any model output, not only as an internal RLHF tool.
Always-on side channels that display unpaid premise-debt and consequence-debt could make observer applause subordinate to unresolved questions in ordinary chat interfaces.
De-institutionalized platforms may need engineered substitutes for the slow adversarial scaffolding that peer review once supplied.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing worth knowing is that this is a coherent framework paper, not an empirical or formal result. Moldoveanu and Baum take their earlier epinets work, bolt on Brandom-style commitment/entitlement scorekeeping, and use it to redescribe public assertion failures—including LLM hallucinations and sycophancy—as strategic exploits of the trust that normally lets scaffolded claims go unchallenged. The real addition is the triadic Condition T plus the concrete taxonomy (poses, demagogical baits, smokescreens, rabbit holes, haystacks, etc.) and the clean mapping of those onto training-time evaluator pressure.

What they do well is give us a usable vocabulary that sits between “bubbles/echo chambers” and pure information-design models. The UT2/DT trust unpackings are clear, the tables are practical, and treating LLMs as a special regime of the same incentive landscape is a productive move even if it is mostly analogy. The three machine sketches (Brandom for scorekeeping, Hintikka for interrogative paths, Ramsey for non-veritistic payoffs) and the checklist-style RLHF protocol are concrete enough to be implementable later.

The soft spots are exactly where you would expect for a pure conceptual piece. There is no implementation, no formal result, and no empirical test of whether the machines or I2D primitives actually raise the cost of evasion on usable timescales. The stress-test point lands: Condition T assumes a live, interactive observer set whose reactions the speaker can recruit mid-exchange; RLHF annotators and reward models are a frozen objective, not that kind of audience. So the claim that LLM failures are “machine-regime variants” of the same mechanisms is an asserted transfer, not a secured one. That does not sink the framework, but it does mean the abstract’s operational language overreaches what is delivered.

Citation pattern is solid—philosophy, interactive epistemology, classic cheap-talk and persuasion papers, plus the recent hallucination/sycophancy literature—without looking padded. No math or data to check; coherence is the only soundness criterion and it holds.

This is for people working at the social-epistemology / LLM-evaluation boundary who need better language for audit and training design. It deserves a serious referee as a framework paper if the claims are scoped as conceptual. I would bring it to reading group and would cite the taxonomy and the UT/DT unpacking. Send it out.

Referee Report

3 major / 6 minor

Summary. The paper proposes Adversarial Social Epistemology (ASE) for scaffolded public communication among humans and LLMs. It argues that epistemic failures are not adequately captured by epistemic bubbles, echo chambers, or misinformation diffusion; rather, they arise when agents strategically exploit the commitments and entitlements that normally make assertions trustworthy. Using epistemic networks (epinets) plus an inferentialist semantics, the authors define upstream and downstream trust (UT1/UT2, DT) as strong belief in redeemability under entitled questioning, introduce Condition T (minimally triadic sender–recipient–observer communication), and catalogue anti-veritistic mechanisms (poses, demagogical triggers/baits, social-proof covers, smokescreens, plausible ambiguations, decoys, flares, rabbit holes, haystacks). They map LLM hallucinations, sycophancy, and related infelicities as machine-regime variants of these mechanisms, sketch I2D design primitives and a checklist-based RLHF protocol, and propose three operational machines (Brandom for commitment/entitlement scorekeeping, Hintikka for interrogative paths, Ramsey for non-veritistic payoffs).

Significance. If the framework holds, ASE would reorient social epistemology and LLM evaluation away from mean-field network effects and toward local strategic exploitation of inferential commitments under observer pressure. The taxonomy in Table 1, the UT/DT trust definitions, the evaluator question bank (Q1–50), and the three-machine architecture are concrete conceptual contributions that could guide platform design and RLHF practice. The paper is explicit about its conceptual character and does not claim empirical results; its value lies in a transferable vocabulary for auditing trust breaches in mixed human–machine assemblies. The main significance risk is that the human–machine transfer and the operational machines remain promissory, so the contribution is currently strongest as a conceptual redescription rather than as an engineering program.

major comments (3)

[§6.1–6.2, Table 2, Condition T] §6.1–6.2 and Table 2: The load-bearing claim that LLM failures are “machine-regime variants” of the same triadic mechanisms requires Condition T to transfer. Condition T (pp. 19–21) treats {O} as a live, differentially informed audience whose reactions S can anticipate and recruit mid-exchange. In the LLM mapping, {O} is the RLHF/annotator/reward stack—a frozen training objective, not an interactive audience the model observes and strategically exploits during a turn. The paper equates optimization under a static reward that favors fluency/agreeableness with strategic exploitation of live observer effects. Without a clearer account of how interactive belief and mid-exchange recruitment work (or fail) under frozen evaluators, Table 2 remains an analogy rather than a secured transfer, and the claim that ASE “crosses the man–machine boundary” (§1, §4) is under-supported.
[§§7.1–7.3] §§7.1–7.3: The Brandom, Hintikka, and Ramsey machines are presented as operationalizing ASE “on short-enough-to-matter time scales with good-enough-to-make-a-difference results” (Extended Abstract; §1). They are sketched at the level of roles and illustrative examples (commitment stores, erotetic trees, non-veritistic utility estimates) without formal input/output specifications, consistency criteria, evaluation metrics, or even a toy implementation. For a cs.AI audience, at least one machine needs a minimal formal interface (e.g., what graph the Brandom Machine emits from a short dialogue; how the Hintikka Machine ranks questions; what evidence the Ramsey Machine takes as input) so that the operational claim is falsifiable rather than purely promissory.
[§1, §8] §1 and §8: The paper asserts that bubbles, echo chambers, and misinformation diffusion “under-weight” or leave “under-described” the strategic, semantically structured failures ASE targets. That contrast is central to the contribution, but the manuscript does not engage any specific model or result from that literature in enough detail to show where ASE predicts a different mechanism or intervention. A short comparative subsection—e.g., how a pose or social-proof cover differs from Nguyen-style echo-chamber insulation or from a standard diffusion cascade—would make the “not adequately captured” claim load-bearing rather than programmatic.

minor comments (6)

[§3.1] Epinets are cited (Moldoveanu & Baum 2011; 2014) but never given a compact formal recap (agents, propositions, epistemic relations, update rules). A short box or appendix would help readers who do not know that prior work.
[Table 1] Table 1 is useful but the “Primary exploitation” column mixes structural features (observer effect, quotability) with mechanism labels; a consistent ontology (e.g., what is exploited vs. how) would improve readability.
[§6.4] The evaluator bank (Q1–50) is a strength, but several items are compound or double-barreled (e.g., Q4, Q21). Splitting them would improve inter-rater reliability if the bank is used as stated.
[References, §6.1] References: Liu et al. (2024) “Walking with Dreams…” appears to be a mismatched citation for the “lost in the middle” claim; the standard reference is Liu et al., “Lost in the Middle,” TACL 2024. Kalai et al. (2025) is cited as arXiv:2509.04664—verify the number against the intended preprint.
[Throughout] Occasional typos and awkward phrasing (e.g., “truss-like structures of trust,” “quizposition,” “efferent conversation”) are fine as technical coinages if defined once; a brief glossary would help.
[§2] The Shu et al. (2012) example is noted as retracted; that is appropriate, but the text could state more clearly that the example is chosen precisely because the trust-truss failed under adversarial incentives.

Circularity Check

1 steps flagged · score 1.0 of 10

No by-construction prediction or load-bearing circular derivation; only mild definitional coherence typical of a conceptual framework plus ordinary self-citation of the authors' prior epinets toolkit.

self definitional [§3.3–3.4 (UT2/DT) and §4.1–4.4 (mechanisms under Condition T)]
"Trust is an assumption by someone that A would make good on answers to questions any reasonable interlocutor could ask, were they to ask. ... We put forth a set of mechanisms that enable agents to reap private benefits for making public assertions in ways and settings that enable them to evade UT-and-DT-related obligations to redeem commitments by answering questions and responding to challenges."

Trust is defined as assumed redeemability of commitments; adversarial mechanisms are then defined as maneuvers that block or raise the cost of that redeemability. The 'explanation' of trust breaches is partly by construction of these linked definitions. This is mild conceptual circularity of framework coherence, not a fitted or uniqueness-forced result.

full rationale

This is a conceptual outline paper, not a derivation or fitting paper. It does not claim empirical predictions, uniqueness theorems, fitted parameters renamed as forecasts, or ansatzes smuggled via self-citation. Trust is defined (UT1/UT2/DT) as strong belief in redeemability of inferential commitments, and the listed adversarial mechanisms are then characterized as ways of blocking, deferring, or distorting that redeemability under Condition T. That is coherent framework-building, not a reduction of a claimed result to its inputs by construction. The representational apparatus (epinets) is drawn from the authors' prior work (Moldoveanu & Baum 2011, 2014, etc.), but the central ASE claims—triadic observer-exploiting mechanisms, the human–LLM mapping, and the three sketched machines—are developed in this paper and also rest on external sources (Brandom, Hintikka, Ramsey, Kalai et al., etc.). Self-citation here supplies a vocabulary, not a uniqueness or forcing argument that makes the conclusions true by prior author fiat. No step reduces a 'prediction' or 'first-principles result' to a fitted input or to a definitional identity. Score 1 reflects only the mild, expected definitional coherence of a conceptual paper, not significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 6 invented entities

The paper is conceptual: it introduces a framework and entities rather than fitting constants. Load-bearing background includes Brandomian inferentialism, interactive epistemology/epinets, and the claim that observer-mediated triadic structure plus mixed veritistic incentives is the right unit for both social platforms and LLM training. Free parameters are essentially absent; invented entities (ASE, the machines, I2D, the mechanism lexicon) carry the contribution and currently lack independent empirical handles outside the paper’s illustrations.

assumptions (5)

domain assumption Assertions incur upstream and downstream inferential commitments and create entitlements to question that can be scorekept (Brandom-style inferentialism).
Invoked throughout §§3–7 as the semantics that makes trust and adversarial moves trackable for humans and machines.
domain assumption Condition T: most platform and LLM-mediated exchanges are minimally triadic (sender, recipient, observers) with interactive beliefs about observability and intelligibility.
Stated in §4.1 as the structural condition enabling the mechanism taxonomy.
ad hoc to paper Trust in scaffolded public claims is (strong) belief in redeemability of commitments under entitled questioning (UT1/UT2 and DT), not mere inductive reliability.
Defined in §3.3–3.4; central to ASE’s contrast with bubbles/echo-chamber accounts.
domain assumption LLM training/evaluation regimes (RLHF/DPO/etc.) create mixed veritistic and anti-veritistic demand characteristics analogous to social-media observer incentives.
Core bridge in §§1 and 6; without it the human–machine unification fails.
domain assumption Epinets with interactive belief states can represent the relevant epistemic asymmetries for public communication dynamics.
Taken from authors’ prior work and used as the representational backbone in §3.

invented entities (6)

Adversarial Social Epistemology (ASE)
purpose: Name and organize the theory of how agents exploit commitments/entitlements in scaffolded public communication.
Framework label introduced in abstract and §1; no independent empirical validation beyond illustrative cases.
Brandom Machine
purpose: Track commitments, entitlements, and incompatibilities from assertions for audit.
Sketched in §7.1 without implementation, evaluation, or external falsifiable performance claims.
Hintikka Machine
purpose: Generate and order interrogative paths that test redeemability of commitments.
Sketched in §7.2 as an erotetic engine; no working system or benchmark results.
Ramsey Machine
purpose: Reconstruct non-veritistic payoffs and incentive structure behind assertions.
Sketched in §7.3; depends on unstated models of observer utilities and interactive beliefs.
I2D platform primitives (Intelligibility, Interrogation, Disclosure)
purpose: Design interventions that pin claims, highlight entitled questions, and display disclosure trails.
Posited in §5.3 as design primitives; not built or tested.
Taxonomy of anti-veritistic mechanisms (pose, demagogical trigger/bait, social-proof cover, smokescreen, plausible ambiguation, decoy, flare, rabbit hole, haystack)
purpose: Catalog how Condition T is exploited to block UT/DT redemption.
Table 1 and §4; illustrative examples only, no measurement protocol validating coverage or distinctness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Adversarial Social Epistemology for Assemblies of Humans and Large Language Models." pith.science (2026). https://pith.science/paper/6XQDUB2Y

@misc{pith2026260707760,
  author       = {Pith},
  title        = {Pith review of: Adversarial Social Epistemology for Assemblies of Humans and Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6XQDUB2Y}},
  note         = {Machine review of arXiv:2607.07760}
}

read the original abstract

We outline an adversarial social epistemology (ASE) for densely interactive communicative landscapes in which public assertions are scaffolded by chains of testimony, inference, institutional certification, and tacit trust. In such landscapes, agents have incentives and affordances to distort, color, omit, fabricate, or strategically under-specify information for private, reputational, rhetorical, or material gains. We argue that these phenomena are not adequately captured by familiar descriptions of epistemic bubbles, echo chambers, or misinformation diffusion. What requires explanation is how communicative agents exploit the commitments and entitlements that normally make scaffolded assertions trustworthy. We provide language that delivers the requisite analysis, outline mechanisms that subvert trust in scaffolded public communications, and outline machinery for auditing and redressing trust breaches arising from subverting the auditability of inferential chains, drawing on epistemic networks, enriched with an inferentialist semantics for interpreting assertions.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

[1]

Do Large Language Models Advocate for Inferentialism?

Arai, Y. and Tsugawa, S. (2025). Do Large Language Models Advocate for Inferentialism? arXiv: 2412.14501. Aumann, R. J. (1999). Interactive Epistemology I: Knowledge. International Journal of Game Theory, 28, 263-300. Brandom, R. B. (1994). Making it Explicit: Reasoning, representing, and discursive commitment. Harvard University Press. Brandom, R.B. (199...

work page Pith review arXiv 2025
[2]

Epistemic Networks

Maynez, J., Narayan, S., Bohnet, B. & McDonald, R. (2020). On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1906–1919. Moldoveanu, M. C., & Baum, J. A. C. (2011). “I Think You Think I Think You’re Lying”: The Interactive Epistemology of Trust in Social Net...

work page Pith review arXiv 2020

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.

[1] [1]

Do Large Language Models Advocate for Inferentialism?

Arai, Y. and Tsugawa, S. (2025). Do Large Language Models Advocate for Inferentialism? arXiv: 2412.14501. Aumann, R. J. (1999). Interactive Epistemology I: Knowledge. International Journal of Game Theory, 28, 263-300. Brandom, R. B. (1994). Making it Explicit: Reasoning, representing, and discursive commitment. Harvard University Press. Brandom, R.B. (199...

work page Pith review arXiv 2025

[2] [2]

Epistemic Networks

Maynez, J., Narayan, S., Bohnet, B. & McDonald, R. (2020). On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1906–1919. Moldoveanu, M. C., & Baum, J. A. C. (2011). “I Think You Think I Think You’re Lying”: The Interactive Epistemology of Trust in Social Net...

work page Pith review arXiv 2020