What's in the News? Towards Identification of Bias by Commission, Omission, and Source Selection (COSS)

Anastasia Zhukova; Bela Gipp; Felix Hamborg; Karsten Donnay; Terry Ruas

REVIEW 3 major objections 6 minor 12 references

What's in the News? Towards Identification of Bias by Commission, Omission, and Source Selection (COSS)

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a five-step pipeline that identifies bias by commission, omission, and source selection together by tracing paraphrased paragraphs across news articles.

desk verdict A clear, honest concept paper for joint COSS bias identification; no implementation yet, and the omission step needs a completeness baseline before it can deliver what it promises. read the letter →

arxiv 2508.02540 v1 pith:KMHCJNM5 submitted 2025-08-04 cs.CL

Anastasia Zhukova , Terry Ruas , Felix Hamborg , Karsten Donnay , Bela Gipp This is my paper · ORCID

classification cs.CL

keywords mediabiasbycommissionomissionsourceselectiontextreuseparaphraseidentificationsemanticsimilaritynewsanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a way to detect three kinds of news bias at once: bias by commission (adding information), bias by omission (leaving information out), and bias by source selection (choosing which outlets to rely on). Rather than treating these as separate problems, the authors design a single pipeline that starts from a seed article, retrieves other articles covering the same event, aligns paragraphs that convey the same message in different wording, and builds a time-ordered graph of text reuse. The graph lets an analyst see which paragraphs are original, which are copied or paraphrased from elsewhere, and how the political leaning of a paragraph changes as it travels across outlets. If the pipeline works, readers could inspect the provenance and slant of every paragraph in a news story instead of trusting an outlet's label.

What carries the argument

The central object is a directed graph of text reuse whose nodes are articles and paragraphs and whose edges encode the strength of semantic similarity between paraphrased paragraphs, with direction enforced by article timestamps. Paragraph-level paraphrase identification is the load-bearing mechanism: it lets the system treat 'same message, different words' as reuse, which is what distinguishes this approach from earlier copy-paste or TF-IDF-based methods. On top of this graph, the pipeline performs statistical and network analysis of reuse patterns and polarity changes, then visualizes the result on a timeline with per-paragraph polarity labels.

What would settle it

Take a set of event-related news articles with manually annotated paragraph-level paraphrases and known source facts; run the pipeline's text-alignment step and compare its matches to the human annotation. If two paragraphs a human agrees are equivalent are repeatedly left unmatched, or unrelated paragraphs are matched strongly enough to change polarity conclusions, the central claim would be contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the three traditionally separate bias types—commission, omission, and source selection—are manifestations of one underlying process: the way information is reused, paraphrased, and re-evaluated as it flows through the news ecosystem. The paper argues that identifying this process requires aligning text at the paragraph level, not just detecting direct copy-paste, and that semantic similarity methods can find paraphrases that use different and possibly loaded wording. By assigning polarity labels to articles and re-labelling individual paragraphs, the pipeline can reveal when a neutral statement becomes left- or right-leaning, when a source is dropped, and when a particular fact is never picked up by another outlet. The paper presents this as a concept pipeline with a visualization example, not a fully evaluated system.

Load-bearing premise

The whole approach assumes that paraphrase identification can reliably recognize paragraphs that say the same thing in different, possibly loaded wording; if that alignment step fails or is inconsistent, the graph and all bias conclusions built on it are unreliable.

Editorial extensions

If this is right

A seed article can be decomposed into original and reused paragraphs, with each reused paragraph traced to the outlet that first published it.
An article that draws most of its content from neutral wire services would be classified as less bias-prone than one built from unsourced slanted paragraphs.
The same paragraph can be tracked over time; if its polarity label changes from center to left when a left-leaning outlet reuses it, that is a concrete instance of bias by commission.
When a source paragraph never appears in a later article, that absence is visible as a missing edge in the graph and counts as evidence of bias by omission.
The visualization makes these patterns accessible to non-experts, so news readers could inspect provenance and slant rather than relying on an outlet's reputation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next step the authors do not spell out is to score an outlet by its reuse profile, such as the ratio of sourced to unsourced paragraphs, as a reproducible bias metric.
The same paragraph-alignment machinery could be applied outside journalism, for example to political speeches or corporate press releases, wherever the same message is reworded across documents.
A focused benchmark of human-annotated news paragraphs with intentionally loaded rewordings would test whether the alignment step is accurate enough for the concept to be operationalized at scale.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things up front. This is a concept paper, not a demonstrated system; it proposes a pipeline rather than showing one working. And the joint framing of commission, omission, and source selection (COSS) is genuinely new relative to the work it cites.

What it does well: it connects three previously separate bias-detection threads through a text-reuse graph with timestamps, and it pushes the alignment step beyond copy-paste to paraphrased paragraphs. That is the right level of granularity for tracking how a story changes as it propagates. The paragraph-level focus, the polarity relabeling along the timeline, and the example visualization in Figure 1 make the intended output concrete. The writing is clear and the authors are careful to say this is 'towards' identification, not a finished result.

The soft spots are real but proportionate for a short concept paper. The biggest is omission. The paper defines omission as identifying which parts of source articles were not picked up, but that is not enough to distinguish biased omission from ordinary summarization. Any condensed article drops paragraphs. Without a normative baseline for what should have been included, or a threshold that separates biased omission from editorial selection, non-reuse is just non-reuse. And because the omission set is relative to whatever the candidate-retrieval step found, a source missed by retrieval cannot contribute omitted material. So the joint COSS claim is under-specified on the omission axis. The stress-test note lands.

Second, there is no implementation or evaluation. That means the headline claim—that the pipeline identifies bias—is not supported by evidence. As an agenda it is coherent; as a result it is not there yet. The authors do not oversell, and the self-cites to NewsDeps, POLUSA, and word-choice bias are legitimate building blocks.

Who should read this? Researchers working on media-bias detection who want a unifying framework and a concrete set of design decisions to test. It would serve well as a workshop paper or as a framing section in a longer paper. Full archival publication would need at least one worked case or a small evaluation of the alignment step.

My recommendation: treat it as a serious research agenda, not as a validated method. If the venue accepts concept papers, send it to review; the omission-baseline issue and the missing evaluation should be the main referee questions. If the venue requires demonstrated results, desk reject on those grounds.

Referee Report

3 major / 6 minor

Summary. This manuscript proposes a concept pipeline for automatically identifying media bias by commission, omission, and source selection (COSS) as a joint three-fold objective. The pipeline consists of five stages: candidate retrieval, source retrieval and text alignment, construction of a graph of text reuse, statistical and network pattern analysis, and visualization. The authors argue that prior work addresses each bias type separately, usually with direct text-reuse matching, whereas the proposed approach uses paragraph-level paraphrase identification via semantic similarity, tracks polarity changes across outlets, and visualizes text-reuse patterns on a timeline. The paper is a position/concept statement: it describes the intended functionality and includes an illustrative figure but presents no implementation, experiments, dataset, or formal derivation. The abstract and conclusion frame the contribution as a methodology proposal for joint COSS identification.

Significance. If realized, the proposed pipeline would address a real gap: existing media-bias tools tend to handle one bias type or work at article level, while COSS identification requires jointly tracking what was added, what was omitted, and which sources are reused as a story propagates. The paper also connects to a credible research lineage (NewsDeps, POLUSA, word-choice bias), and the idea of using paragraph-level paraphrase alignment is well motivated. As a concept, it is useful for the community. However, the manuscript is only a proposal: no component is validated, and the decisive assumptions about alignment reliability and the operational definition of omission are left unsubstantiated. The significance is therefore conditional on future implementation and evaluation.

major comments (3)

[Section 2, Source retrieval and text alignment] The entire pipeline rests on the assertion that current paraphrase identification and semantic similarity methods can reliably detect paragraphs that convey the same message but use different and possibly loaded wording. No evidence, benchmark, or error analysis is provided to support this assumption. Since the graph of text reuse, the attribution of paragraphs to sources, the detection of polarity changes, and all downstream bias conclusions depend on the accuracy of this alignment step, the central claim of COSS identification cannot be assessed without at least a pilot evaluation or a concrete specification of the paraphrase-detection component and its expected performance.
[Section 2, Statistical and network analysis] The operationalization of omission bias as 'identifying which parts of the source articles were not picked up' is insufficient. Any condensed article will fail to reuse many paragraphs of its sources, and non-reuse only indicates biased omission if the missing content was expected, relevant, and available under some normative baseline. The manuscript supplies no such baseline, no threshold, and no comparison condition (e.g., against a neutral reference corpus) that would separate bias by omission from routine editorial summarization. Moreover, because candidate retrieval pre-defines the universe of event-related articles, any source missed by retrieval cannot contribute omitted paragraphs, so the omission set is relative to an incomplete and potentially biased retrieval. This under-specification blocks the joint COSS claim as stated.
[Section 2, Polarity classification] The proposed method assigns each article a polarity label induced from its outlet (e.g., Fox News as 'R') and then uses a paragraph-level polarity classifier to detect polarity changes in reused passages. This assumes that outlet-level slant reliably transfers to each paragraph, yet Figure 1 itself illustrates a counterexample: article A6 is left-oriented but contains a center-labeled original paragraph. Without a validated paragraph-level classifier or a discussion of label noise, the measured 'polarity changes' that drive the commission-bias analysis may reflect classifier error or outlet-label artifacts rather than actual bias in text reuse. The manuscript should acknowledge this risk and propose a validation strategy.

minor comments (6)

[Keywords] The keyword 'new analysis' appears to be a typo; it should likely read 'news analysis'.
[Section 1, Introduction and Related Work] The word 'adaption' should be 'adaptation' in the sentence 'However, the adaption and application...'.
[Figure 1] The caption says the figure orders articles 'by date-time,' but the figure itself lacks visible timestamp labels and axis legends; please make the temporal ordering and the polarity labels (L, C, R) self-explanatory.
[Section 2, Candidate retrieval] The listed data sources (LexisNexis, CommonCrawl, MediaCloud, GDELT) are heterogeneous in structure and coverage; the manuscript should clarify how event relevance is determined for each and whether the retrieval step assumes a particular corpus format.
[Section 2, Statistical and network analysis] The sentence '...identifying which parts of the source articles were not picked up by which articles were excluded from discussions' is grammatically tangled; please rewrite to distinguish between paragraphs omitted by a specific article and paragraphs excluded from the entire discussion.
[Section 2, Visualization] The reference to 'the strength of semantic similarities between paragraphs [9]' appears to point to NewsDeps, but the intended citation may instead be the POLUSA or another similarity-related work; please verify all citation numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the COSS pipeline is an unvalidated concept architecture whose stages are not defined by the target bias quantities.

full rationale

The manuscript is a short concept paper describing a pipeline for identifying bias by commission, omission, and source selection. It contains no equations, no fitted parameters, and no quantitative predictions. The central claim is architectural: candidate retrieval, semantic paragraph alignment, a timestamped reuse graph, statistical and network pattern analysis, and visualization can jointly support COSS analysis. None of these stages is defined by the target quantity. 'Bias by omission' is operationalized as the set of source paragraphs that are not reused, which is indeed a definition of the targeted pattern; however, the paper explicitly frames these as 'patterns of text reuse that may induce bias' rather than as derived first-principles results. The self-citations (POLUSA, NewsDeps, word-choice bias) are used as component technologies or datasets and are not invoked to exclude alternatives or to prove the joint COSS claim. The paper's own text repeatedly qualifies the outputs as 'may indicate' or 'might indicate,' so no prediction is being asserted as forced by the inputs. Under the stated criteria, I find no circular step.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters and no invented entities. Its claims rest on three unvalidated domain assumptions: outlet-derived polarity labels are adequate proxies for article slant; semantic similarity tools can reliably detect paraphrased paragraph reuse; and reuse patterns such as 'neutral sources imply reliability' are interpretable as bias indicators. These are stated in Section 2 but never tested.

assumptions (3)

domain assumption Outlet-inferred polarity labels are meaningful proxies for article slant.
Section 2, Candidate retrieval: 'to each article, we assign a polarity label induced from an outlet, e.g., each article from Fox News will be labeled as R'. The pipeline relies on these labels for polarity relabeling and pattern analysis.
domain assumption Paraphrase detection via semantic similarity can reliably identify conceptually reused paragraphs.
Section 2, Source retrieval and text alignment: 'we focus on identifying paraphrased sentences or paragraphs that convey the same message but use different and possibly loaded wording'; the pipeline assumes current paraphrase identification methods are accurate enough.
domain assumption Neutral-source reuse implies reliability, and slanted paragraphs with no sources indicate untrustworthiness.
Section 2, Statistical and network analysis: 'if an article reuses a significant amount of information from neutral sources... we could conclude that this article is reliable'; this is an interpretive assumption that is not empirically calibrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of What's in the News? Towards Identification of Bias by Commission, Omission, and Source Selection (COSS)." pith.science (2026). https://pith.science/paper/KMHCJNM5

@misc{pith2026250802540,
  author       = {Pith},
  title        = {Pith review of: What's in the News? Towards Identification of Bias by Commission, Omission, and Source Selection (COSS)},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KMHCJNM5}},
  note         = {Machine review of arXiv:2508.02540}
}

read the original abstract

In a world overwhelmed with news, determining which information comes from reliable sources or how neutral is the reported information in the news articles poses a challenge to news readers. In this paper, we propose a methodology for automatically identifying bias by commission, omission, and source selection (COSS) as a joint three-fold objective, as opposed to the previous work separately addressing these types of bias. In a pipeline concept, we describe the goals and tasks of its steps toward bias identification and provide an example of a visualization that leverages the extracted features and patterns of text reuse.

Figures

Figures reproduced from arXiv: 2508.02540 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 11 canonical work pages

[1]

Eytan Bakshy, Solomon Messing, and Lada A Adamic. 2015. Exposure to ide- ologically diverse news and opinion on Facebook. Science 348, 6239 (2015), 1130–1132

2015
[2]

Darrell Christian, Paula Froke, Sally Jacobsen, and David Minthorn. 2014. The Associated Press stylebook and briefing on media law . The Associated Press

work page 2014
[3]

Jonas Ehrhardt, Timo Spinde, Ali Vardasbi, and Felix Hamborg. 2021. Omis- sion of Information: Identifying Political Slant via an Analysis of Co-occurring Entities. In Information between Data and Knowledge . Schriften zur Information- swissenschaft, Vol. 74. Werner Hülsbusch, Glückstadt, 80–93. https://epub.uni- regensburg.de/44939/

work page 2021
[4]

Susanne Fengler and Stephan Ruß-Mohl. 2008. Journalists and the information- attention markets: Towards an economic theory of journalism. Journalism 9, 6 (2008), 667–690

work page 2008
[5]

Russell Frank. 2003. These crowded circumstances’ when pack journalists bash pack journalism. Journalism 4, 4 (2003), 441–458

work page 2003
[6]

Robert Gaizauskas, Jonathan Foster, Yorick Wilks, John Arundel, Paul Clough, and Scott Piao. 2001. The METER corpus: a corpus for analysing journalistic text reuse. In Proceedings of the corpus linguistics 2001 conference , Vol. 1. Citeseer

work page 2001
[7]

Lukas Gebhard and Felix Hamborg. 2020. The POLUSA dataset: 0.9 M politi- cal news articles balanced by time and outlet popularity. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 . 467–468

work page 2020
[8]

Felix Hamborg, Karsten Donnay, and Bela Gipp. 2019. Automated identification of media bias in news articles: an interdisciplinary literature review.International Journal on Digital Libraries 20, 4 (2019), 391–415

work page 2019

Show all 12 references

[9]

Felix Hamborg, Philipp Meschenmoser, Moritz Schubotz, Philipp Scharpf, and Bela Gipp. 2021. NewsDeps: Visualizing the Origin of Information in News Articles. Wahrheit und Fake im postfaktisch-digitalen Zeitalter: Distinktionen in den Geistes-und IT-Wissenschaften (2021), 151–166

2021
[10]

Felix Hamborg, Anastasia Zhukova, and Bela Gipp. 2019. Automated Identifi- cation of Media Bias by Word Choice and Labeling in News Articles. In 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL) . 196–205

2019
[11]

Jonathan Matusitz and Gerald-Mark Breen. 2012. An examination of pack jour- nalism as a form of groupthink: A theoretical and qualitative analysis. Journal of Human Behavior in the Social Environment 22, 7 (2012), 896–915

2012
[12]

Zizi Papacharissi and Maria de Fatima Oliveira. 2008. News frames terrorism: A comparative analysis of frames employed in terrorism coverage in US and UK newspapers. The international journal of press/politics 13, 1 (2008), 52–74

2008

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Eytan Bakshy, Solomon Messing, and Lada A Adamic. 2015. Exposure to ide- ologically diverse news and opinion on Facebook. Science 348, 6239 (2015), 1130–1132

2015

[2] [2]

Darrell Christian, Paula Froke, Sally Jacobsen, and David Minthorn. 2014. The Associated Press stylebook and briefing on media law . The Associated Press

work page 2014

[3] [3]

Jonas Ehrhardt, Timo Spinde, Ali Vardasbi, and Felix Hamborg. 2021. Omis- sion of Information: Identifying Political Slant via an Analysis of Co-occurring Entities. In Information between Data and Knowledge . Schriften zur Information- swissenschaft, Vol. 74. Werner Hülsbusch, Glückstadt, 80–93. https://epub.uni- regensburg.de/44939/

work page 2021

[4] [4]

Susanne Fengler and Stephan Ruß-Mohl. 2008. Journalists and the information- attention markets: Towards an economic theory of journalism. Journalism 9, 6 (2008), 667–690

work page 2008

[5] [5]

Russell Frank. 2003. These crowded circumstances’ when pack journalists bash pack journalism. Journalism 4, 4 (2003), 441–458

work page 2003

[6] [6]

Robert Gaizauskas, Jonathan Foster, Yorick Wilks, John Arundel, Paul Clough, and Scott Piao. 2001. The METER corpus: a corpus for analysing journalistic text reuse. In Proceedings of the corpus linguistics 2001 conference , Vol. 1. Citeseer

work page 2001

[7] [7]

Lukas Gebhard and Felix Hamborg. 2020. The POLUSA dataset: 0.9 M politi- cal news articles balanced by time and outlet popularity. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 . 467–468

work page 2020

[8] [8]

Felix Hamborg, Karsten Donnay, and Bela Gipp. 2019. Automated identification of media bias in news articles: an interdisciplinary literature review.International Journal on Digital Libraries 20, 4 (2019), 391–415

work page 2019