REVIEW 7 cited by
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
With the rapid progress of large language models (LLMs) and the huge amount of text they generated, it becomes more and more impractical to manually distinguish whether a text is machine-generated. Given the growing use of LLMs in social media and education, it prompts us to develop methods to detect machine-generated text, preventing malicious usage such as plagiarism, misinformation, and propaganda. Previous work has studied several zero-shot methods, which require no training data. These methods achieve good performance, but there is still a lot of room for improvement. In this paper, we introduce two novel zero-shot methods for detecting machine-generated text by leveraging the log rank information. One is called DetectLLM-LRR, which is fast and efficient, and the other is called DetectLLM-NPR, which is more accurate, but slower due to the need for perturbations. Our experiments on three datasets and seven language models show that our proposed methods improve over the state of the art by 3.9 and 1.75 AUROC points absolute. Moreover, DetectLLM-NPR needs fewer perturbations than previous work to achieve the same level of performance, which makes it more practical for real-world use. We also investigate the efficiency--performance trade-off based on users preference on these two measures and we provide intuition for using them in practice effectively. We release the data and the code of both methods in https://github.com/mbzuai-nlp/DetectLLM
Forward citations
Cited by 7 Pith papers
-
Zero-Shot Detection of LLM-Generated Code via Approximated Task Conditioning
ATC detects AI-generated code by asking a language model to reconstruct the programming task, then scoring token entropy under that reconstructed task, outperforming prior zero-shot detectors on Python, C++, and Java ...
-
Black-Box Detection of LLM-Generated Text Using Generalized Jensen-Shannon Divergence
SurpMark detects machine-generated text by estimating state-transition matrices from discretized surprisals and scoring them with generalized Jensen-Shannon divergence to human versus machine references.
-
Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry
A new modern Chinese poetry detection benchmark shows most current AI-text detectors are unreliable, particularly when LLMs imitate a human style.
-
COMPKE: Complex Question Answering under Knowledge Editing
COMPKE is a new benchmark with 11,924 complex questions that tests knowledge editing through one-to-many relations and logical operations, where existing editing methods often fail.
-
The Compositional Architecture of Regret in Large Language Models
The paper claims that regret in LLMs is encoded by interacting neuron groups detectable in the final hidden layer, using new S-CDI, RDS, and GIC metrics.
-
DP-MGTD: Privacy-Preserving Machine-Generated Text Detection via Adaptive Differentially Private Entity Sanitization
DP-MGTD claims that applying differential-privacy entity sanitization amplifies human-vs-machine text separability, reaching F1 > 0.99 on MGTBench-2.0 while satisfying an epsilon-DP guarantee.
-
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
SHE lowers behavioral hallucination scores by about 10 percent by detecting low visual-textual similarity and projecting out the hallucinated direction in embedding space.
Discussion (0). Sign in to comment.