{"id":"c12ecc58-cf31-468d-9724-5546de86c514","arxiv_id":"2507.19548","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Neither purely democratic nor purely epistocratic AI alignment may be justified on its own, so hybrid frameworks combining expert input, participation, and anti-monopoly safeguards appear more promising.","lead":"This paper asks who should decide what an AI considers right and wrong: the people it affects, or a small group of experts. It argues that neither approach works alone and that combining both, with safeguards, is more promising.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's route to hybrid frameworks depends on an exit-based account of coercion: if structural coercion is recognized, the 'multiple AI' safeguard cannot neutralize the democratic justification, and the central conclusion loses its main support.","rationale":"The paper is a fair and well-structured mapping of justificatory strategies for democratic AI alignment, and the reader's weakest-assumption diagnosis is correct: the central argument hinges on a contested, exit-based account of coercion. The reader's conditional verdict remains appropriate because the paper could be repaired by engaging with structural-coercion literature and specifying when exit costs matter. I do not see a fatal flaw; the concern is a demand for defense, not a demonstration of falsehood. The proposed test — a concrete high-switching-cost case — would show whether the plural-AI safeguard generalizes, and if it does not, the paper must weaken its conclusion or add a substantive democratic component. Thus the correct verdict stays conditional, with the coercion account as the required revision.","tokens_in":10289,"tokens_out":6412,"duration_ms":85140,"concrete_test":"Take the exact 'vegan supermarket' case and alter one parameter: the alternative exists but requires 40 minutes of travel, a 20% price increase, or migration of one's calendar, contacts, and medical records to a different AI assistant. Apply the paper's own 'great opportunity cost' caveat: if crossing any plausible cost threshold makes the AI's noncompliance coercive, then the general claim 'you are no more coerced than by a vegan-only supermarket' is false. Write a one-page analysis applying the Section 4 coercion definition to this altered case; if the analysis shows coercion, the Section 5 anti-monopoly safeguard cannot carry the argument alone, and the hybrid conclusion must be revised to specify when exit costs are low enough for the safeguard to work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 4, the authors argue that a normatively noncompliant AI does not coerce a user whenever the user can 'freely go elsewhere' — the vegan-supermarket analogy. This is load-bearing because the Section 5 conclusion that neither pure approach suffices is reached partly by letting epistocratic approaches 'prevent the threat of illegitimate coercion by preventing the threat of coercion' through plural AIs and anti-monopoly safeguards. That move assumes a negative-liberty, exit-based conception of coercion. On broader accounts — structural coercion, domination as subjection to another's will, or coercion via high switching costs — the mere availability of another AI does not remove the coercion; the user is still required to bear exit costs or to live under constraints set by another party. The paper even concedes the 'great opportunity cost' caveat, but never specifies when opportunity costs become coercion. If the broader account is right, then the anti-monopoly safeguard is insufficient, and either democratic justification must do real work in every alignment context, or the hybrid conclusion must be restricted to cases where exit is genuinely costless — which is not the general case. Since the non-instrumental argument for democracy is allowed to fail only on the strength of this account, the account needs explicit defense against alternative coercion theories.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the normative problem of AI alignment, defined as the question of what normative constraints an AI system should satisfy, as opposed to the technical problem of implementing them. It contrasts democratic approaches, in which affected stakeholders determine the constraints, with epistocratic approaches, which defer to normative experts. The authors distinguish instrumental justifications (better outcomes, wisdom of crowds) from non-instrumental justifications (preventing illegitimate authority or coercion). They argue that normative and metanormative uncertainty create a justificatory gap that democratic approaches try to fill with political justification, but they identify unresolved burdens for the coercion-based justification, especially the claim that coercion only occurs when an AI is the de facto or de jure standard. They conclude that neither purely democratic nor purely epistocratic approaches may suffice on their own, pointing toward hybrid frameworks with anti-monopoly safeguards.","tokens_in":10533,"tokens_out":5128,"duration_ms":61264,"significance":"If the analysis holds, the paper makes a useful contribution by clearing away a common conflation of technical and normative alignment and by laying out the exact argumentative burdens that a coercion-based democratic justification must meet. Its taxonomy of instrumental versus non-instrumental justifications, and its list of four necessary propositions for the coercion argument, are genuinely helpful for future work. The paper engages fairly with opposing views, including epistemic decision rules and anti-monopoly safeguards, and it explicitly acknowledges tensions it does not resolve. The main value is as a map of the dialectical terrain rather than as a positive theory; its conclusions are conditional and forward-looking. It would be more significant if the coercion account were defended and if the transition from identified burdens to the hybrid conclusion were made explicit.","major_comments":[{"comment":"The background-conditions response to coercion is load-bearing for proposition (iv) and for the Section 5 conclusion, but it relies on an exit-based conception of coercion that is asserted rather than defended. The claim that a user is not coerced when she can 'freely go elsewhere' works only on a narrow, negative-liberty account of coercion; on broader accounts, such as structural coercion or domination as subjection to another's will, the availability of a differently aligned AI does not remove the coercive character of the noncompliant AI. The paper itself concedes the 'great opportunity cost' caveat but never specifies when opportunity costs become coercion. Since the conclusion that epistocratic approaches can 'take the sting out of' the non-instrumental justification depends on this account, the authors need to defend it against alternative theories of coercion or explicitly restrict the conclusion to cases of genuinely costless exit.","section":"4 (coercion, vegan-supermarket example)"},{"comment":"The suggestion that epistocratic approaches can justify coercion through decision rules such as maximise expected choiceworthiness assumes that a practical justification for choosing certain normative constraints transfers directly to the coercion of users who do not endorse those constraints. That transfer is not trivial: a decision rule may give designers a reason to implement a constraint without giving the affected user a justification for being subjected to it, particularly under the normative and metanormative uncertainty the paper itself emphasizes. Because this undercutting move contributes to the conclusion that epistocratic approaches can handle the coercion objection, the transfer claim needs explicit defense.","section":"4 (epistocratic response, decision rules)"},{"comment":"The central conclusion that 'neither purely epistocratic nor purely democratic approaches ... may be sufficient on their own' is not established by the preceding analysis. Sections 3 and 4 show at most that instrumental justifications require further empirical work and that the coercion-based non-instrumental justification faces unanswered objections. That is a directed challenge to democratic proponents; it does not by itself show that pure epistocratic or pure democratic solutions are insufficient, nor does it show that hybrid frameworks are superior. As it stands, the hybrid conclusion is a research prediction rather than an implication of the argument. Please either mark it explicitly as a forward-looking conjecture or supply an argument that the identified burdens cannot be met by either pure approach.","section":"5 (Conclusion)"}],"minor_comments":[{"comment":"There is a typographical artifact in the abstract: 'aimtofillthroughpoliticalratherthantheoreticaljustification' should be 'aim to fill through political rather than theoretical justification'. The author affiliation also contains a typo: 'Saarbücken' should be 'Saarbrücken'.","section":"Abstract"},{"comment":"The discussion of Schuster and Kilov would benefit from a brief statement of the disagreement: the authors claim that Schuster and Kilov conflate the technical and normative problems, but the paragraph asserts this rather than showing where exactly the conflation occurs.","section":"1"},{"comment":"The examples 'Brexit, Trump, the climate crisis' are too compressed to carry the argument. If these are meant as cases where people vote against their own interests, at least one example should be spelled out, or they should be replaced with less contested cases.","section":"3"},{"comment":"The clarification that 'the primary coercer is not the AI itself but the person or organisation that defines the normative constraints' introduces an agency question that is never revisited: in the later examples (the AI assistant refusing to buy meat), the corporate and technical chain of responsibility is not identified. A sentence about who, in the authors' view, defines constraints in current deployment contexts would help.","section":"4"},{"comment":"The paper moves from observed 'reasonable disagreement' to 'normative uncertainty' as the rational response. This inference would benefit from a footnote distinguishing epistemic uncertainty from the possibility that disagreement merely reflects different evaluative perspectives; not all readers will accept that persistent disagreement entails uncertainty about the normative facts.","section":"2"}],"recommendation":"major_revision","confidential_remarks":"This is a solid conceptual paper that is suitable for the journal's scope. The revision should focus on defending the coercion account and making the modal status of the hybrid conclusion precise. I saw no issues with attribution or self-citation: the self-citations (e.g., Baum 2025) serve as background taxonomy and are not load-bearing premises. The paper is best read as a critical survey of existing justifications rather than a positive theory; making that status explicit would improve it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a read. The paper's main contribution is a clean analytical separation: it distinguishes the technical from the normative alignment problem, sorts justifications into instrumental vs. non-instrumental, and then funnels the non-instrumental justification through a four-condition test for coercion. That structure is genuinely useful, not just repackaged. It also does honest work on the democratic side: the bootstrapping problem for democratic procedures and the 'lowest common denominator' worry are real challenges that proponents tend to gloss over. The authors do not strawman epistocratic views, and their conclusion is appropriately modest — hybrid frameworks, not a knockout.\n\nThe soft spot is exactly where the stress-test note lands. In Section 4 they argue that a normatively noncompliant AI does not coerce you if you can 'freely go elsewhere,' via the vegan-supermarket and bowling-club analogies. That is load-bearing: it lets epistocratic approaches neutralize the coercion-based justification by ensuring plural AI options and anti-monopoly safeguards. But the account is asserted, not defended. On a broader account — structural coercion, domination as subjection to another's will, or coercion through high switching costs — the availability of another AI does not automatically dissolve the coercion. The paper concedes 'great opportunity cost' but never specifies when opportunity cost becomes coercion. If coercion is read broadly, the sting is not taken out, and either democratic justification must do work everywhere or the hybrid conclusion must be restricted to low-exit-cost contexts. This is fixable: they should engage with at least one alternative coercion theory and say where they draw the line.\n\nMinor but worth noting: the instrumental section is thin (Condorcet conditions, bias risks) and the discussion of decision rules like maximize expected choiceworthiness is sketched. Those are limitations the authors flag themselves, so not unfair.\n\nBottom line: a serious, honest conceptual contribution for the AI governance and ethics audience. It deserves referee time and likely publication after the coercion account is strengthened. I'd bring it to reading group, and I'd cite the taxonomy in my own work. Recommend engaging with it substantively.","headline":"A clean conceptual map of democratic vs. epistocratic justifications that earns its place, but its hybrid conclusion leans on an exit-based coercion account that needs explicit defense.","tokens_in":11018,"tokens_out":1597,"would_cite":true,"duration_ms":18525,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Neither expert rule nor public rule can legitimate AI alignment by itself; the paper argues for hybrid institutions combining expert judgment, participatory input, and safeguards against AI monopolies.","keywords":["AI alignment","legitimacy","democratic justification","public reason","value imposition","epistocracy","normative uncertainty","coercion"],"falsifier":"Examine a real deployment in which one AI is the only reasonable gateway to an essential function — for example, a government service whose sole interface is an aligned chatbot — and check whether users can realistically switch to an alternative or act on their own. Under the paper's own criteria, users without a free alternative are genuinely coerced, so finding that such coercion occurs routinely even in markets with nominally competing AI products would falsify the claim that keeping alternatives available neutralizes the coercion-based justification for democratic input. Conversely, deployments where alternatives genuinely exist and users report no restraint would support the paper's conclusion.","tokens_in":10082,"feed_emoji":"⚖️","tokens_out":17694,"duration_ms":169106,"temperature":0.7,"pith_summary":"This paper asks who should decide what an AI system may and may not do — the normative half of the alignment problem, as opposed to the technical half of implementing constraints. The authors weigh two answers: deferring to normative experts (epistocratic approaches) or letting everyone affected decide (democratic approaches), and they clarify that techniques like crowdsourcing, RLHF, and constitutional AI are implementation methods, compatible with either answer. Their central argument is that deep, reasonable disagreement about what is right leaves a justificatory gap: no one can prove their values are the correct ones, so expert-imposed alignment stands on weak ground, and democratic participation is meant to supply political legitimacy instead. Working through the strongest non-instrumental justification for democracy — that expert-driven alignment illegitimately coerces users — they conclude it is not decisive, because coercion depends on background conditions such as whether users can freely switch to a differently aligned AI, and because experts have their own resources, including decision rules for normative uncertainty and the ability to prevent any AI from becoming a de facto standard. The paper concludes that neither pure approach suffices on its own, pointing toward hybrid frameworks that combine expert judgment with targeted participatory input and institutional safeguards against uniform alignment and AI monopolies.","feed_headline":"Neither experts nor the people alone can set AI's values","feed_subtitle":"Deep disagreement means no expert can justify imposing values on everyone—so democracy must enter, but not alone.","key_machinery":"The argument is carried by the 'justificatory gap': normative and metanormative uncertainty — reasonable disagreement about which reasons matter, how strongly, and whether any unique normative truth exists — eliminates the theoretical justification that would legitimize expert-chosen alignment constraints, leaving a space that democratic approaches try to fill with political justification. Within that frame, the load-bearing mechanism is the coercion analysis with its background conditions (the account the paper draws on calls them 'tempering factors'): an aligned AI only actually coerces a user when the user cannot freely act on their own desires, for instance because the AI is the de facto or de jure standard for some purpose and switching is unavailable or too costly. This mechanism is what lets the paper defuse the democratic justification, since background conditions can be controlled — multiple differently aligned AIs can be kept available — and it is also what lets epistocratic approaches answer the coercion worry. A secondary piece of machinery is the three-step decomposition of producing normative constraints (identify the relevant reasons, measure their relative strength, aggregate them into overall verdicts), which shows how hybrid approaches can partition the labor between experts and the affected public scenario by scenario, step by step, or within a single step.","core_discovery":"The paper's central claim is that neither purely epistocratic nor purely democratic approaches to the normative problem of AI alignment are sufficient on their own, and that the coercion-based justification for democratic approaches is weaker than its proponents assume. First, because people reasonably disagree about what is right and even about whether a unique normative truth exists, normative and metanormative uncertainty removes the sure theoretical justification that would legitimize any imposed set of constraints; this justificatory gap is what democratic participation aims to fill with political justification. Second, the most promising non-instrumental justification for democracy — that expert-driven alignment illegitimately coerces users — must establish four propositions: that users can be coerced through an AI's alignment, that such coercion would be unjustified, that democratic procedures can produce a justification legitimizing it, and that epistocratic approaches cannot prevent it. The authors argue none of these comes for free: coercion only occurs under background conditions (a user who can freely use another AI or act on their own is not coerced), democratic procedures face a bootstrapping problem and risk settling on a minimal normative denominator, and epistocratic approaches can invoke decision rules such as maximise expected choiceworthiness to justify their choices under uncertainty while also preventing coercion by keeping any single AI from becoming the de facto or de jure standard. The conclusion is not that democratic participation is worthless but that context-sensitive hybrid frameworks, combining expert judgment with participatory input and institutional safeguards against AI monopolization, are the most suitable path.","pith_inferences":["The analysis implies an empirical prediction: the force of the democratic, anti-coercion case should track market structure, being strong where AI use is effectively mandatory (government services, default preinstalled assistants, employer-mandated tools) and weak where genuine exit options exist; measuring user exit options across deployment contexts and correlating them with reported value impos","The 'lowest common normative denominator' objection suggests a testable failure mode: democratic processes among diverse stakeholders may systematically produce alignment constraints too thin to regulate AI behavior, forcing a return to expert content; a deliberation experiment on concrete alignment scenarios could measure how much common ground actually emerges.","The paper analyzes coercion of users, but its own definition of affected stakeholders includes people only indirectly touched by an AI's behavior; extending the coercion analysis to bystanders and third parties would clarify when democratic input matters beyond the immediate user-AI interaction.","Because the paper notes that normative uncertainty is not evenly distributed, a design principle for the hybrid frameworks it endorses suggests itself: use democratic input where disagreement runs deep and expert deference where near-consensus exists, and test whether such a division of labor is stable in concrete alignment case studies."],"forward_implications":["Democratic proposals for AI alignment must pass a four-part test — users can be coerced through alignment, such coercion would be illegitimate, democratic procedures can legitimize it, and epistocratic approaches cannot prevent it — and none of the four parts comes for free.","Crowdsourcing, RLHF, and constitutional AI should be treated as technical implementation techniques: they are silent on who determines the normative constraints, so debates that treat them as inherently democratic settle nothing about the normative problem.","Ensuring that no single AI becomes the de facto or de jure standard is not merely a market-structure concern; it is the concrete mechanism that prevents alignment from being coercive in the first place.","Decision rules designed for normative uncertainty, such as maximise expected choiceworthiness, give epistocratic approaches a practical justification for their chosen constraints, so democratic proponents must engage those rules rather than assume expertise cannot justify.","The question the paper leaves open is contextual: which aspects of the normative problem should be settled by expert knowledge, which by democratic input, and under what institutional conditions — the answer is expected to vary by application context."],"supporting_citations":[{"why":"Establishes the two-part framing of alignment as a technical plus a normative problem, which the whole analysis presupposes.","marker":"[9]"},{"why":"The view the paper corrects: crowdsourcing, RLHF, and constitutional AI are treated there as normative solutions rather than as implementation techniques.","marker":"[21]"},{"why":"Quoted proponent of democratic alignment; supplies the instrumental argument that expert-driven alignment exacerbates inequalities.","marker":"[12]"},{"why":"The Delphi experiment; quoted for the claim that top-down approaches force scientists to impose their values on others.","marker":"[13]"},{"why":"Quoted for the worry that expert-driven alignment causes value imposition or domination, the target of the coercion-based justification.","marker":"[10]"},{"why":"Supplies the taxonomy of instrumental versus non-instrumental justifications for democracy that organizes the paper's argument.","marker":"[8]"},{"why":"Provides the jury theorem analysis, including weakened assumptions, used to assess the wisdom-of-crowds justification.","marker":"[11]"},{"why":"Defends maximise expected choiceworthiness, the decision rule that gives epistocratic approaches a practical justification under normative uncertainty.","marker":"[17]"},{"why":"Supplies the background-conditions account of when coercion actually occurs, the mechanism that defuses the democratic justification.","marker":"[16]"}],"fun_headline_variants":["Democracy alone can't justify AI alignment, expert input needed too","Hybrid governance, not pure democracy, for AI alignment","Coercion case for democratic AI alignment is shaky","AI alignment needs experts plus public, not one alone","Neither pure democracy nor expert rule alone sets AI values"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire argument leans on a specific, contested account of coercion, namely that a user is not coerced by an AI's noncompliance if they can freely use a different AI or act on their own; if one instead accepts a broader account of coercion (structural coercion, or mere subjection to another's will), the background-conditions response fails and the democratic justification for alignment retains its full force.","fun_headline_variants_meta":{"raw":{"variants":["Democracy alone can't justify AI alignment, expert input needed too","Hybrid governance, not pure democracy, for AI alignment","Coercion case for democratic AI alignment is shaky","AI alignment needs experts plus public, not one alone","Neither pure democracy nor expert rule alone sets AI values"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2634,"prompt_tokens":990,"completion_tokens":1644,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":606,"completion_tokens_details":{"reasoning_tokens":1564}},"tokens_in":606,"tokens_out":1644,"duration_ms":12881,"temperature":1.0,"reasoning_tokens":1564,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:30:56.009694+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Examine a real deployment in which one AI is the only reasonable gateway to an essential function — for example, a government service whose sole interface is an aligned chatbot — and check whether users can realistically switch to an alternative or act on their own. Under the paper's own criteria, users without a free alternative are genuinely coerced, so finding that such coercion occurs routinely even in markets with nominally competing AI products would falsify the claim that keeping alternatives available neutralizes the coercion-based justification for democratic input. Conversely, deployments where alternatives genuinely exist and users report no restraint would support the paper's conclusion.","supporting_citations":[{"cited_title":"Minds and Machines, 2(5), 411–437","cited_arxiv_id":null,"evidence_quote":"Establishes the two-part framing of alignment as a technical plus a normative problem, which the whole analysis presupposes."},{"cited_title":"AI & Society","cited_arxiv_id":null,"evidence_quote":"The view the paper corrects: crowdsourcing, RLHF, and constitutional AI are treated there as normative solutions rather than as implementation techniques."},{"cited_title":"T., Papyshev, G., Wong, J","cited_arxiv_id":null,"evidence_quote":"Quoted proponent of democratic alignment; supplies the instrumental argument that expert-driven alignment exacerbates inequalities."},{"cited_title":"Philosophical Studies, (2025)","cited_arxiv_id":null,"evidence_quote":"Quoted for the worry that expert-driven alignment causes value imposition or domination, the target of the coercion-based justification."},{"cited_title":"The Stanford Encyclopedia of Philosophy","cited_arxiv_id":null,"evidence_quote":"Supplies the taxonomy of instrumental versus non-instrumental justifications for democracy that organizes the paper's argument."},{"cited_title":"E., Spiekermann, K.: An Epistemic Theory of Democracy, Oxford Uni- versity Press, Oxford","cited_arxiv_id":null,"evidence_quote":"Provides the jury theorem analysis, including weakened assumptions, used to assess the wisdom-of-crowds justification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defends maximise expected choiceworthiness, the decision rule that gives epistocratic approaches a practical justification under normative uncertainty."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the background-conditions account of when coercion actually occurs, the mechanism that defuses the democratic justification."}],"review_version":1}