{"id":"ee368ec7-8b96-4a10-8896-14a39b54e862","arxiv_id":"1908.09805","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Stylometry successfully distinguishes human from machine text but fails to distinguish truthful from false machine-generated text when the same language model produces both.","lead":"This paper shows that style-based detectors, which work well at telling human text from machine text, mostly fail at telling true machine text from false machine text. The authors build two test sets where a language model inserts or rewrites sentences, and a state-of-the-art detector can barely beat chance at spotting the false ones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Blanket 'stylometry cannot' claim rests on a single neural detector; feature-based stylometry is never tested.","rationale":"Read in good faith, the paper makes a valuable controlled observation: the same Grover-Mega detector that excels at provenance detection (94%) performs poorly at veracity detection (53-71%) on two carefully constructed benchmarks, and the authors are transparent about a length confound in the QA extension dataset. The data URL is a real asset for reproducibility. However, the paper's headline claim is much broader than what the experiments support. The single most load-bearing concern is the inference from one neural discriminator to 'stylometry' as a class. Grover-Mega is a Transformer-based neural model, not a feature-based stylometric system, and the paper does not test any of the traditional stylometric feature families it cites. A universal negative about a method class cannot be established by a single instance, especially when that instance is a black-box neural model whose failure might reflect its inductive bias rather than the absence of stylistic signal. This is precisely the reader's weakest assumption, so I agree with the CONDITIONAL verdict. The concrete check I propose would settle the concern: if classical feature-based stylometry also fails on these benchmarks, the paper's conclusion is much better supported; if it succeeds, the conclusion must be narrowed to neural provenance-style detectors. Until that check is run, the appropriate verdict remains CONDITIONAL, not ACCEPT or REJECT.","tokens_in":12013,"tokens_out":3856,"duration_ms":44325,"concrete_test":"On the released benchmark data, implement a suite of classical stylometric feature sets (function-word frequencies, POS tag distributions, sentence-length and readability metrics, LIWC-style lexicons, and TF-IDF n-grams) with standard classifiers (logistic regression, SVM, random forest) under repeated stratified cross-validation with bootstrapped confidence intervals. Pre-register a threshold: if any feature-based classifier exceeds, say, 80% balanced accuracy on either veracity benchmark, the paper's blanket claim is falsified; if all remain near chance (95% CI overlapping or close to 50-60%), the generalization is strengthened. Also compute bootstrap intervals for Grover-Mega's 53%, 65%, and 71% accuracies to check whether they are significantly above chance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a universal negative: Section 6 concludes that 'stylometry-based classifiers cannot identify auto-generated intentionally misleading content,' and the abstract generalizes to 'stylometry fails.' Yet Section 3 states, 'We used a Grover-Mega discriminator for all of the experiments.' Grover-Mega is one neural model, originally trained as a provenance discriminator and then fine-tuned; it is not a representative sample of the stylometry approach class. The paper itself surveys a rich set of feature-based stylometric methods in Section 2 (function words, POS tags, syntactic structure, deception cues), but none of these are evaluated on the new veracity benchmarks. The internal control is compelling for this one detector: it reaches 94% provenance accuracy but only 53-71% veracity accuracy. That shows Grover-Mega fails on these tasks, but it does not license the conclusion that stylometry as a whole fails. A feature-based detector could in principle exploit cues that the neural discriminator's inductive bias overlooks. Additionally, the reported veracity accuracies are point estimates without confidence intervals, so it is unclear whether 53% is even significantly above chance. The load-bearing assumption is that one model stands in for an entire class; without testing alternative stylometric representations, the broad conclusion exceeds the evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper asks whether stylometry can distinguish true from false machine-generated text when both are produced by the same language model. The authors construct two benchmarks: a QA-extension task in which Grover generates answers to news questions and humans label them as true or false, and an article-modification task in which negations are added/removed by GPT-2 Medium. Using a Grover-Mega discriminator fine-tuned adaptively on each attack, they report accuracies of 71%, 53%, 65%, and 65% on the veracity tasks, versus 90–95% on provenance detection tasks (full articles and vanilla extensions). They conclude that stylometry can detect machine-generated text (provenance) but cannot identify intentionally misleading content, and they recommend non-stylometric approaches such as fact checking. Human evaluations and an analysis of the length confound in the QA task are included.","tokens_in":12230,"tokens_out":6799,"duration_ms":67744,"significance":"The paper makes a valuable negative claim with practical implications. The internal control—showing that the same detector achieves high provenance accuracy while failing on veracity—is a strong and well-executed comparison. The benchmarks (QA extension and modification) are reusable resources, the authors release the data, and the human evaluation anchors the machine results. The length-confound analysis in Section 4.1 is honest and useful. If the generalization is properly scoped, the result will be influential for the fake-news detection community. The main weakness is that the universal negative about 'stylometry' is supported by only one neural detector; the near-chance results also lack confidence intervals.","major_comments":[{"comment":"The conclusion generalizes from one detector to the entire class of stylometry methods. Section 3 states that 'We used a Grover-Mega discriminator for all of the experiments,' yet Section 6 concludes that 'stylometry-based classifiers cannot identify auto-generated intentionally misleading content,' and the abstract claims 'stylometry is limited.' Grover-Mega is a single neural model (a Transformer), and the paper surveys but does not evaluate feature-based stylometry methods (function words, POS tags, syntactic structure) in Section 2. Without experiments or an argument showing that Grover-Mega is representative of the class, the blanket conclusion exceeds the evidence. Please either narrow the claims to the tested model (e.g., 'the Grover-Mega neural stylometry detector') or add evaluations of at least one or two feature-based stylometric baselines on the same benchmarks.","section":"Section 3 and Section 6"},{"comment":"The key veracity accuracies are reported as point estimates without confidence intervals or significance tests. In particular, the 53% accuracy for m=2 modification may be statistically indistinguishable from chance; the statement in Section 4.1 that 'the classifier fails completely' needs support from a confidence interval or a binomial test. Similarly, the 62% accuracy on the short-answer QA subset and the 65% accuracies for m=6 and m=10 lack error bars. Please add bootstrap confidence intervals (or exact binomial intervals) for the accuracy/F1 values in Table 1, and ideally for the provenance results in Table 2 for comparison.","section":"Table 1 and Section 4.1"},{"comment":"The length confound in the QA-extension task is acknowledged but not controlled for at the level required by the central claim. The authors show that a length-only linear classifier reaches 56% and that the detector's accuracy falls from 71% to 62% when restricted to short false answers, but this post-hoc subset analysis is not accompanied by a length-matched evaluation or a significance test. Since the near-chance performance on the veracity task is a key evidence for the paper's conclusion, please provide a direct comparison on a length-matched test set (or report accuracy conditioned on answer length) to demonstrate that the remaining signal is not an artifact of the dataset construction.","section":"Section 4.1 and Table 1"}],"minor_comments":[{"comment":"The manuscript describes Grover-Mega as a 'stylometry detector' without defining why a Transformer-based neural classifier counts as stylometry; please clarify the connection (e.g., it operates on surface form and is trained for discrimination) in one sentence.","section":"Section 3"},{"comment":"The human evaluation details are thin: exact participant counts, recruitment, and instructions are not given for the 'about 100 examples' per dataset, and the reported Cohen's kappa applies to the veracity labeling of generated answers rather than to the human detection task. Please add a sentence specifying the evaluation protocol.","section":"Section 4"},{"comment":"The text in Section 5 mentions the 'stronger Grover-Mega generator' for the QA setting, but the reader is not told whether this generator is the same checkpoint as the discriminator; please state model names and parameters clearly once in Section 3 and refer back.","section":"Section 5, Table 2"},{"comment":"Some references appear incomplete (e.g., 'Doc' and 'Hox' entries lack authors and years), and a few URLs are given without access dates; please complete the bibliography according to the journal style.","section":"References"},{"comment":"The term 'real' is used for both human-written text and LM-generated truthful text; since the paper's distinction is veracity rather than provenance, consider a terminology note to avoid confusion.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and important question, and the benchmarks are a genuine contribution. My main reservation is the gap between the evidence (one detector, no CIs) and the universal claim in the title/abstract. I believe a major revision asking for a scoped claim or additional baselines is appropriate; the central experiment itself is sound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper makes a real point: provenance detection and veracity detection are different tasks, and a detector that excels at the first can be near chance at the second. The authors build two veracity-labeled benchmarks for LM-generated misinformation (QA-based extension and article modification), and show that the same Grover-Mega discriminator hits 94% F1 on provenance but only 53–71% accuracy on veracity. That internal control is the paper's strongest contribution, and it's a good one.\n\nThe benchmarks themselves are thoughtfully built. The modification dataset keeps negation counts balanced to avoid an obvious confound; the QA extension includes a length-based baseline and shows performance drops when they control for it. Inter-annotator agreement on the fake/real labels is substantial (kappa 0.78). These are the kinds of details that make the empirical claim credible.\n\nThe soft spot is the leap from \"this detector fails\" to \"stylometry fails.\" The paper surveys classic feature-based stylometry in Section 2 but never tests any of it on the new benchmarks. Grover-Mega is one neural model with a particular inductive bias; a feature-based detector could in principle exploit cues that this model misses. So the abstract's claim that \"stylometry fails\" is broader than the evidence. The results are also point estimates without confidence intervals, so it's unclear whether the 53% accuracy on the m=2 modification is even above chance. The human evaluation is very small—two subjects for one task, and a few for the other—so those numbers are suggestive at best.\n\nNone of this sinks the core comparison. The same detector succeeding at provenance and failing at veracity is exactly the kind of controlled demonstration that should make people working on fake-news detection pause. The paper is honest about its limitations, and the recommendation to focus on fact-checking and non-textual signals is reasonable given the evidence.\n\nWho should read it: anyone working on LM-generated text detection or misinformation benchmarks. It deserves a serious referee, but the revision should test at least one feature-based stylometric baseline and report variance. I'd take it to our reading group and would likely cite it as the cautionary result about provenance-based detectors.","headline":"A genuine, useful negative result about veracity detection for machine-generated text, though the broad 'stylometry fails' claim outruns the evidence of a single neural detector.","tokens_in":12753,"tokens_out":984,"would_cite":true,"duration_ms":11994,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Stylometry cannot tell whether AI-generated news is true or false, because language models write true and false content in the same style.","keywords":["stylometry","machine-generated fake news","language models","provenance detection","veracity detection","fake news benchmarks","neural fake news","misinformation detection"],"falsifier":"A style-only classifier that, after fine-tuning on the paper's two veracity benchmarks (QA-extension and article-modification), exceeds roughly 85 percent balanced accuracy in distinguishing true from false machine text would falsify the claim that LM-generated true and false content are stylistically indistinguishable.","tokens_in":11820,"feed_emoji":"🤖","tokens_out":8338,"duration_ms":74147,"temperature":0.7,"pith_summary":"The paper argues that stylometry, the analysis of writing style, cannot reliably distinguish false machine-generated news from true machine-generated news, even though it can tell machine text from human text. Using a single state-of-the-art style-based detector on two new benchmarks—an auto-completion task and an article-editing task—the authors show that the detector identifies machine authorship with up to 95 percent accuracy on a single generated sentence, but its accuracy for telling true from false machine text peaks at 71 percent and falls to 53 percent when negations are inserted. The reason is that language models produce stylistically consistent text regardless of whether the content is truthful, unlike human writers who show style changes when deceiving. The paper therefore recommends that defenses against machine-generated misinformation move beyond style and incorporate fact-checking or other external evidence.","feed_headline":"Stylometry catches AI authors but not AI lies","feed_subtitle":"Detectors can flag machine-written text, but veracity needs fact-checking, not style.","key_machinery":"The load-bearing object is the Grover-Mega discriminator, a large Transformer-based neural classifier that is fine-tuned to decide whether a text was written by a language model or a human. The paper's controlled comparison is what isolates style from truth: in the QA-extension benchmark, the same Grover-Mega model generates both the true and the false answers, and in the article-modification benchmark a second model (GPT-2) decides where to add or remove negations while the underlying article's style is preserved. This pairing ensures that any observable stylistic difference between the classes would have to reflect the veracity of the content, and the benchmark results show that no usable difference exists. The two datasets are therefore the experimental instruments that carry the argument.","core_discovery":"The central discovery is the decoupling of provenance from veracity in machine-written text. On the provenance task, a fine-tuned Grover-Mega discriminator classifies full machine versus human articles with 94 percent accuracy and detects even a single machine-written sentence within a human article at 95 percent accuracy. On the veracity task, where both 'real' and 'fake' examples are generated by the same language model, the same detector reaches only 71 percent accuracy on the question-answering extension benchmark and 53 to 65 percent on the article-modification benchmark; a simple sentence-length baseline already accounts for part of the QA-extension score. These numbers support the paper's claim that LM-generated true and false content are stylistically indistinguishable, and that stylometry can prevent impersonation but not detect LM-generated misinformation.","pith_inferences":["The paper's argument implies that any detector restricted to the text itself will face the same barrier whenever true and false content are generated by the same model, so future style-based defenses would need to find signals the generation process does not control.","A natural testable extension is to reuse the two benchmarks to evaluate whether constraining generation with a fact-checking module (for example, decoding that prefers claims supported by an external corpus) makes the generated true and false texts diverge stylistically enough to be separated.","Extending the same experimental design to non-English news, tables, or medical text would show whether the style-veracity decoupling is a general property of current language models or specific to English prose."],"forward_implications":["Detectors can still flag text as machine-generated, which is useful against impersonation, but should not be used to infer that the text is false.","Fake-news evaluation benchmarks must include mixed human-machine and fully machine text with both true and false content, or they will overstate detection performance.","Fact-checking and methods that use external evidence become the central defense against machine-generated misinformation.","Human readers, when given external sources, detect machine-generated false claims much better than text-only classifiers, supporting the development of human-in-the-loop tools."],"supporting_citations":[{"why":"Supplies the Grover generator and the Grover-Mega discriminator used for all provenance and veracity experiments.","marker":"Zellers et al. 2019"},{"why":"Supplies GPT-2, used to create the auto-completion extension attacks and the WebText data for provenance evaluation.","marker":"Radford et al. 2019"},{"why":"Provides the newsQA dataset of CNN articles with questions and gold answers on which the QA-extension benchmark is built.","marker":"Trischler et al. 2017"},{"why":"Provides the New York Times article corpus used as real text in the modification and extension benchmarks.","marker":"Thompson 2017"},{"why":"Establishes that human deceptive writing differs stylistically from truthful writing, the assumption the paper tests and refutes for machine text.","marker":"Ott et al. 2011"},{"why":"Introduces the learning-based machine-versus-human provenance discrimination approach that this paper fine-tunes and compares against.","marker":"Bakhtin et al. 2019"}],"fun_headline_variants":["Stylometry IDs AI authors but can't catch AI falsehoods","Style detectors know who wrote it, not if it's true","AI text style is consistent, hiding lies from stylometry","Provenance easy, veracity hard: stylometry falls short","Machine style blurs truth and fiction, beating style checks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's sweeping conclusion assumes the Grover-Mega discriminator is representative of all stylometry detectors; if another style-based method could tell true from false machine text, the claim that stylometry fails would not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Stylometry IDs AI authors but can't catch AI falsehoods","Style detectors know who wrote it, not if it's true","AI text style is consistent, hiding lies from stylometry","Provenance easy, veracity hard: stylometry falls short","Machine style blurs truth and fiction, beating style checks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000258,"raw_usage":{"total_tokens":1545,"prompt_tokens":869,"completion_tokens":676,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":589}},"tokens_in":485,"tokens_out":676,"duration_ms":7045,"temperature":1.0,"reasoning_tokens":589,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:01:02.506790+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A style-only classifier that, after fine-tuning on the paper's two veracity benchmarks (QA-extension and article-modification), exceeds roughly 85 percent balanced accuracy in distinguishing true from false machine text would falsify the claim that LM-generated true and false content are stylistically indistinguishable.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Grover generator and the Grover-Mega discriminator used for all provenance and veracity experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies GPT-2, used to create the auto-completion extension attacks and the WebText data for provenance evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the newsQA dataset of CNN articles with questions and gold answers on which the QA-extension benchmark is built."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the New York Times article corpus used as real text in the modification and extension benchmarks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that human deceptive writing differs stylistically from truthful writing, the assumption the paper tests and refutes for machine text."}],"review_version":1}