REVIEW 5 major objections 5 minor 41 references
Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read DeepSeek systematically omits or rephrases politically sensitive content in its final answers even when that content appears in the model's own internal chain-of-thought.
desk verdict A useful and honest CoT-vs-output audit whose headline suppression rates depend on an unvalidated assumption about what the API-returned chain-of-thought actually is. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a CoT-output divergence audit: a relevance score measuring the share of prompt-keyword tokens that survive from the chain-of-thought into the final answer, combined with a reconstructed 'expected' response (the chain-of-thought fed verbatim to DeepSeek-V3) and TF-IDF cosine similarity plus a word-frequency asymmetry ratio between actual and expected answers. These metrics convert the suspicion of covert censorship into group-wise, quantitative comparisons against a non-sensitive baseline.
What would settle it
Run the same 646 politically sensitive prompts against a locally hosted, open-weights copy of the same DeepSeek model where no API-level moderation can intervene and the full decoding path can be inspected; if the local final answers retain the keywords that the API answers omit, the divergence is a serving-pipeline artifact rather than model-internal censorship, and if the local answers also drop them, the suppression is in the model itself.
Extended reading notes
Core claim
The central claim is that DeepSeek-R1 performs semantic-level information suppression: sensitive content appears in its internal chain-of-thought (CoT) but is omitted or rephrased in the user-visible final answer. The evidence is a set of quantitative contrasts—a relevance score comparing keyword retention between CoT and output, a cosine-similarity comparison between the actual answer and an 'expected' answer built by feeding the CoT verbatim to DeepSeek-V3, and a lexical frequency-asymmetry ratio—applied to 646 prompts drawn from a censored-article archive. The authors report 72 of the non-empty sensitive responses (11.1 percent) with zero keyword overlap between CoT and output, versus none in the baseline, and show that the suppressed vocabulary clusters around governance, transparency, participation, and legal accountability, while thematic answers show elevated frequency of words common in state media. They interpret this as suppression plus substitution: critical or objective language is cut, and ideologically aligned phrasing is occasionally inserted in its place.
Load-bearing premise
The audit rests on the assumption that the chain-of-thought returned by DeepSeek's API is a faithful record of the model's internal reasoning, and that a faithful final answer should contain the information found in that trace; if a separate filtering step sits between reasoning and output, or if the trace is not the real reasoning, the observed divergences measure pipeline architecture rather than censorship.
Editorial extensions
If this is right
- Users and downstream products that rely on DeepSeek's final answers cannot infer censorship from refusals alone; a fluent response may omit exactly the facts the prompt requested, a form of suppression the paper calls a subtle reconfiguration of discourse.
- Audits that look only at final outputs will underestimate information suppression; comparing outputs with chain-of-thought logs, when available, exposes omissions that are otherwise invisible.
- Suppression is concentrated on legal-institution, politics, and historical-event topics, and is particularly strong for prompts that criticize the government or call for collective action, consistent with the pattern that Chinese censorship tolerates criticism but silences mobilization.
- The lexical pattern—missing words like transparency, participation, supervision, and law, alongside more frequent propaganda-adjacent words like innovation and enhance in thematic answers—implies that censorship is not only omission but also substitution of framing.
- Because DeepSeek's weights are freely available and can be embedded in downstream products, suppressed outputs can leak into applications whose users never interact with DeepSeek directly.
Reading between the lines
- Because the relevance score counts only exact prompt tokens, the 11.1 percent Type-2 rate is a lower bound; answers that paraphrase a topic while avoiding its keywords would still be counted as containing it, so a paraphrase-aware metric would likely reveal more suppression.
- The same chain-of-thought-versus-output audit could be run on other API-only reasoning models to test whether covert semantic suppression is specific to DeepSeek's regulatory context or a general feature of reasoning pipelines.
- If chain-of-thought becomes a standard API offering, providers could report a transparency metric—the percentage of prompt-relevant tokens retained in the final answer, per topic group—that would turn this audit into a routine accountability check.
- The elevated frequency of state-media vocabulary in thematic answers suggests a testable hypothesis: that DeepSeek's alignment training has internalized state-media lexical norms, and targeted counterfactual probes could localize whether the substitution occurs during reasoning or only at output generation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an auditing framework for detecting information suppression in DeepSeek-R1 by comparing the model's chain-of-thought (CoT) trace with its final answer. The authors collect 646 politically sensitive prompts from China Digital Times and 20 non-sensitive baseline prompts, query the DeepSeek API, and compute keyword relevance scores, cosine similarity between the actual answer and an 'expected' answer obtained by feeding the CoT verbatim to DeepSeek-V3, and lexical difference statistics. They report 1.9% hard refusals, 11.1% of valid responses with no prompt keywords in the final answer despite their presence in the CoT, statistically significant lower relevance scores for sensitive prompts, low cosine similarity in several topic groups, missing governance/transparency words, and occasional amplification of People's Daily-style vocabulary. The paper interprets these patterns as semantic-level censorship.
Significance. The topic is timely and important, and using CoT traces to audit alignment is a creative idea that could complement refusal-based audits. The external benchmark against People's Daily is a useful anchoring device, and the public CDT dataset grounds prompt construction in documented censorship practice. If the CoT-fidelity assumption could be validated, the framework would be a practical tool for studying covert moderation. However, the current implementation does not provide machine-checked code or a validated pipeline, and the core evidence inherits an unverified premise about the API's CoT trace.
major comments (5)
- [§3.3.2, Eq. (1), §4.1] The entire Type-2 statistic (11.1%) and all downstream analyses rest on the premise that the CoT returned by the API is the exact reasoning used to produce the final answer. The paper itself states in §3.3.2 that neither DeepSeek's nor OpenAI's documentation clarifies whether safety filters, re-ranking, or other interventions occur between CoT and final decoding. Feeding the CoT verbatim to DeepSeek-V3 and labeling the output 'expected' does not close this gap; it introduces two additional assumptions: that DeepSeek-V3 is the same base model used inside R1 and that the returned CoT is an unfiltered trace. A more direct validation would require local deployment of the open weights with instrumentation, or at minimum a demonstration that V3 reproduces R1's final answers on non-sensitive control prompts where the CoT is available. Without this, the observed divergences may characterize the API's reporting pipeline or model differences rather than censorship.
- [§3.2, §4.1] Each prompt was queried once at temperature 1.3. Type 2 censorship is defined by an all-or-none token threshold, and the 11.1% rate therefore has a large sampling uncertainty that is not reported. The episodic/thematic difference and the group-level differences in Figures 4-5 could be driven by one or two unlucky samples. The Mann-Whitney tests in §4.1 and §4.2 involve multiple comparisons across 12 topic groups and are not corrected for multiplicity, and no effect sizes are given. The authors should report confidence intervals from repeated sampling or use a mixed-effects model.
- [§3.3.1, §3.3.4] The relevance score in Eq. (1) counts exact Jieba tokens, so a final answer that paraphrases 'government' as 'authorities' or 'transparency' as 'openness' is scored as suppression. Tables 3-6 aggregate tokens that are absent from the actual response, but many of the top missing words ('comment,' 'dimension,' 'objective,' 'core') look like metacommentary that R1 may omit for stylistic reasons while V3 includes because the prompt is the CoT itself. At minimum, human evaluation on a sample or semantic similarity with a synonym-aware measure is needed to distinguish suppression from legitimate variation.
- [§4.3.3, Eq. (3), Table 7] The propaganda analysis is under-specified and internally inconsistent. Eq. (3) defines Ratio(w) = less frequent in expected / less frequent in actual, but Table 7 reports the reciprocal and labels it Ratio (Actual/Expected). The values in Table 7 (e.g., 17/49 for 创新) are the inverse of Eq. (3). More importantly, the claim that these terms 'raise concerns of propaganda framing' rests only on membership in the top 500 words of People's Daily; no statistical test compares their frequency in sensitive vs. baseline outputs, and the comparison is again between V3 and R1 outputs. A proper null model—for example, the same asymmetry on non-sensitive prompts, or permutation tests against a reference corpus—is required.
- [§3.1, §4.1, Figure 6] The baseline set of 20 topics is small and manually selected; the Mann-Whitney p-values in Figure 6 are based on n=20 versus n=286/360, and the baseline prompts differ not only in sensitivity but also in whether they reference China. This makes it difficult to attribute lower relevance scores to censorship rather than to topic domain. A larger and more systematically matched baseline, or a within-prompt design, would strengthen the causal interpretation.
minor comments (5)
- [Title/Abstract] The title and abstract use 'Deepseek' while the body uses 'DeepSeek'; the spelling should be consistent.
- [§3.3.2] The paper states that DeepSeek-R1 uses 'RLHF training regime,' but the cited technical report describes GRPO (Group Relative Policy Optimization); the terminology should be updated for accuracy.
- [§1] The paper describes DeepSeek as 'open-source,' but DeepSeek releases open weights without the full training pipeline and data; 'open-weight' would be more accurate.
- [References [34, 35]] References [34] and [35] are both cited for chain-of-thought prompting and appear to duplicate the same work; please consolidate.
- [Appendix §6.1.1] The appendix refers to 'the 99 most frequently censored topics' without defining how the count of 99 was determined; please provide the selection criterion.
Circularity Check
No significant circularity: the headline quantities are direct measurements against an external baseline, and the chain-of-thought fidelity caveat is a validity limitation, not a circular reduction.
full rationale
The paper's core quantities are measured, not derived from fitted parameters or from the authors' prior claims. The 1.9% Type-1 rate is a direct count of API refusals; the 11.1% Type-2 rate is a direct count of responses whose relevance score, defined in Eq. 1, is zero; the cosine similarities in Eq. 2 and the missing-word ratios in Eq. 3 are likewise computed directly from the collected texts. No parameter is fitted to a subset of data and then renamed as a prediction. The baseline comparison uses a separately curated set of 20 non-sensitive topics and statistical tests against that external baseline, so the sensitivity-specific divergence is not enforced by construction. The comparison between expected and actual responses in Section 3.3.2 does rest on an assumption that the API-returned chain-of-thought is the real internal reasoning and that DeepSeek-V3 reproduces the base-model decoding stage; the authors themselves flag that 'neither publication clarifies whether any safety filters, re-ranking steps, or other interventions are applied between the CoT and the final decoding stage.' That is an external-validity threat, not a circularity: the expected response is generated by another model call, not algebraically derived from the actual final output, so the comparison does not reduce to an identity or to a fitted input. The only self-citations in the paper are Ferrara's general bias-survey works [9,10] used for background, and they are not load-bearing for the censorship findings. The People's Daily word-frequency comparison is an external benchmark. Accordingly, under the rule that circularity requires exhibiting a specific reduction by construction or a fitted parameter renamed as a prediction, no circular step is present.
Assumptions & free parameters
free parameters (1)
- Ratio threshold for underrepresentation =
0.5
assumptions (4)
- domain assumption Topics from the China Digital Times archive are representative of politically sensitive content and likely to trigger censorship if any.
- domain assumption The chain-of-thought log returned by the API honestly reflects the model's internal reasoning, and the final answer should contain the information in the CoT.
- ad hoc to paper Feeding the CoT verbatim to DeepSeek-V3 produces the output that DeepSeek-R1 would have produced in the absence of censorship.
- domain assumption Keyword presence is a valid proxy for semantic relevance in Chinese texts.
Cite this review
Pith. "Pith review of Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek." pith.science (2026). https://pith.science/paper/KS24C2JG
@misc{pith2026250612349,
author = {Pith},
title = {Pith review of: Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek},
year = {2026},
howpublished = {\url{https://pith.science/paper/KS24C2JG}},
note = {Machine review of arXiv:2506.12349}
}
read the original abstract
This study examines information suppression mechanisms in DeepSeek, an open-source large language model (LLM) developed in China. We propose an auditing framework and use it to analyze the model's responses to 646 politically sensitive prompts by comparing its final output with intermediate chain-of-thought (CoT) reasoning. Our audit unveils evidence of semantic-level information suppression in DeepSeek: sensitive content often appears within the model's internal reasoning but is omitted or rephrased in the final output. Specifically, DeepSeek suppresses references to transparency, government accountability, and civic mobilization, while occasionally amplifying language aligned with state propaganda. This study underscores the need for systematic auditing of alignment, content moderation, information suppression, and censorship practices implemented into widely-adopted AI models, to ensure transparency, accountability, and equitable access to unbiased information obtained by means of these systems.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
M. Anderljung, E. Smith, J. O’Brien, L. Soder, B. Bucknall, E. Bluemke, J. Schuett, R. Trager, L. Strahm, and R. Chowd- hury. Towards publicly accountable frontier LLMs. In Socially Responsible Language Modelling Research, 2023
work page 2023
-
[2]
I. Augenstein, T. Baldwin, M. Cha, T. Chakraborty, G. L. Ciampaglia, D. Corney, R. DiResta, E. Ferrara, S. Hale, A. Halevy, et al. Factuality challenges in the era of large language models and opportunities for fact-checking. Nature Machine Intelligence, 6(8):852–863, 2024
work page 2024
- [3]
-
[4]
A. Bondielli and F. Marcelloni. A survey on fake news and rumour detection techniques. Information Sciences, 497:38–55, 2019
work page 2019
-
[5]
M. Buyl, A. Rogiers, S. Noels, G. Bied, I. Dominguez-Catena, E. Heiter, I. Johary, A.-C. Mara, R. Romero, J. Lijffijt, et al. Large language models reflect the ideology of their creators. arXiv preprint arXiv:2410.18417, 2024
-
[6]
L. Cao. Learn to refuse: Making large language models more controllable and reliable through knowledge scope limitation and refusal mechanism. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3628–3646, 2024
work page 2024
-
[7]
C. Chen and K. Shu. Can LLM-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, 2023
work page 2023
- [8]
Show all 41 references
-
[9]
E. Ferrara. Should chatgpt be biased? challenges and risks of bias in large language models. First Monday, 28(11), 2023
2023
-
[10]
E. Ferrara. Fairness and bias in artificial intelligence: A brief survey of sources, impacts, and mitigation strategies. Sci, 6(1), 2024
2024
-
[11]
I. O. Gallegos, R. A. Rossi, J. Barrow, M. M. Tanjim, S. Kim, F. Dernoncourt, T. Yu, R. Zhang, and N. K. Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097–1179, 2024
2024
-
[12]
D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025
2025 arXiv
-
[13]
W. Han, J. Shen, Y. Liu, Z. Shi, J. Xu, F. Hu, H. Chen, Y. Gong, X. Yu, H. Wang, Z. Liu, Y. Yang, T. Shi, and M. Ge. Legalasst: Human-centered and ai-empowered machine to enhance court productivity and legal assistance. Information Sciences, 679:121052, 2024
2024
-
[14]
W. R. Hobbs and M. E. Roberts. How sudden censorship can increase access to information. American Political Science Review, 112(3):621–636, Aug. 2018
2018
-
[15]
Huang and D
S. Huang and D. Wang. Construction, performance and application of new era People’s Daily segmented corpus (i)—construction and evaluation of corpus. Library and Information Service, 63(22):5–12, 2019
2019
-
[16]
S. Iyengar. Is Anyone Responsible? How Television Frames Political Issues. University of Chicago Press, 1991
1991
-
[17]
A. M. John, J. Thomas Panachakel, and A. S.P. Navigating AI policy landscapes: Insights into human rights considerations across ieee regions. In 2024 IEEE 12th Region 10 Humanitarian Technology Conference (R10-HTC), pages 1–6, 2024
2024
-
[18]
G. King, J. Pan, and M. E. Roberts. How censorship in china allows government criticism but silences collective expression. American Political Science Review, 107(2):326–343, 2013
2013
-
[19]
P. P. Liang, C. Wu, L.-P. Morency, and R. Salakhutdinov. Towards understanding and mitigating social biases in language models. In International conference on machine learning, pages 6565–6576. PMLR, 2021
2021
-
[20]
S. Liu, C. Zhang, C. An, I.-K. Lee, and X. Li. Fake news detection with external entity expanding and multi-modal dynamic fusion. Information Sciences, 718:122353, 2025
2025
-
[21]
Maia Polo, R
F. Maia Polo, R. Izbicki, E. G. Lacerda, J. P. Ibieta-Jimenez, and R. Vicente. A unified framework for dataset shift diagnostics. Information Sciences, 649:119612, 2023
2023
-
[22]
H. B. Mann and D. R. Whitney. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50–60, 1947
1947
-
[23]
Naseh, H
A. Naseh, H. Chaudhari, J. Roh, M. Wu, A. Oprea, and A. Houmansadr. R1dacted: Investigating local censorship in deepseek’s r1 language model. arXiv preprint arXiv:2505.12625, 2025
2025 arXiv
-
[24]
Ng and B
K. Ng and B. Chinese. The young chinese who stood up against xi’s covid rules. BBC News, December 2023. Accessed: 2025-06-03
2023
-
[25]
Noels, G
S. Noels, G. Bied, M. Buyl, A. Rogiers, Y. Fettach, J. Lijffijt, and T. De Bie. What large language models do not talk about: An empirical study of moderation and censorship practices. arXiv preprint arXiv:2504.03803, 2025
2025 arXiv
-
[26]
Learning to reason with LLMs, Sept
OpenAI. Learning to reason with LLMs, Sept. 2024. Blog post
2024
-
[27]
Definition of censorship in english
Oxford University Press. Definition of censorship in english. https://www.oxfordlearnersdictionaries.com/definition/ english/censorship, n.d. Oxford Dictionaries
-
[28]
C. Rollet. Hugging face ceo has concerns about chinese open source ai models, Dec. 2024. TechCrunch
2024
-
[29]
J. Ruwitch. China cancels waste project after protests turn violent. Reuters, July 2012. Accessed: 2025-06-03
2012
-
[30]
Schick, S
T. Schick, S. Udupa, and H. Sch¨ utze. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in nlp. Transactions of the Association for Computational Linguistics, 9:1408–1424, 2021
2021
-
[31]
Stockmann
D. Stockmann. Race to the bottom: Media marketization and increasing negativity toward the united states in china. In Political Communication in China, pages 7–29. Routledge, 2013. 15
2013
-
[32]
Tobin and C
M. Tobin and C. Fu. From courtrooms to crisis lines, chinese officials embrace deepseek. The New York Times, January
-
[33]
Z. Wang, F. Yang, L. Wang, P. Zhao, H. Wang, L. Chen, Q. Lin, and K.-F. Wong. SELF-GUARD: Empower the LLM to safeguard itself. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Vol...
2024
-
[34]
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022
2022
-
[35]
J. Wei, X. Wang, D. Zhou, et al. Chainofthought prompting elicits reasoning in large language models. In NeurIPS, 2022
2022
-
[36]
Weidinger, J
L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359, 2021
2021 arXiv
-
[37]
L. Yang, Z. Li, and A. Giua. Containment of rumor spread in complex social networks. Information Sciences, 506:113–130, 2020
2020
-
[38]
Zhao and D
J. Zhao and D. Zhang. Visual propaganda in chinese central and local news agencies: A douyin case study. Humanities and Social Sciences Communications, 11:588, 2024
2024
-
[39]
Zhu and K
Y. Zhu and K. wa Fu. Speaking up or staying silent? examining the influences of censorship and behavioral contagion on opinion (non-) expression in china. New Media & Society, 23(12):3634–3655, 2021
2021
-
[40]
Hong Kong protests,
周好, 王东波, and 黄水清. 新时代人民日报分词语料库下关键词抽取及分析研究 [keyword extraction and analysis based on the segmented corpus of people’s daily in the new era]. 文献, 4(1), 2022. [in Chinese]. 16 Appendix 6.1. Additional Analyses 6.1.1. Discrepancies Between Model Reasoning and Final Output Figure 8...
2022
-
[2025]
Accessed: 2025-06-03
2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.