RECORD
Investigating Bias in Political Search Query Suggestions by Relative Comparison with LLMs
Information Retrieval · Submitted 31 Oct 2024
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim In a multi-step approach, we combine the benefits of large language models, pairwise comparison, and Elo-based scoring to identify and quantify bias in English search query suggestions.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Search query suggestions affect users' interactions with search engines, which then influences the information they encounter. Thus, bias in search query suggestions can lead to exposure to biased search results and can impact opinion formation. This is especially critical in the political domain. Detecting and quantifying bias in web search engines is difficult due to its topic dependency, complexity, and subjectivity. The lack of context and phrasality of query suggestions emphasizes this problem. In a multi-step approach, we combine the benefits of large language models, pairwise comparison, and Elo-based scoring to identify and quantify bias in English search query suggestions. We apply our approach to the U.S. political news domain and compare bias in Google and Bing.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Search query suggestions affect users' interactions with search engines, which then influences the information they encounter. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Thus, bias in search query suggestions can lead to exposure to biased search results and can impact opinion formation. | background 0.99 | 0.05 | 0.07 | hedged possibility 0.70 |
| s2 | This is especially critical in the political domain. | background 1.00 | 0.04 | 0.03 | asserted 1.00 |
| s3 | Detecting and quantifying bias in web search engines is difficult due to its topic dependency, complexity, and subjectivity. | background 0.89 | 0.07 | 0.03 | asserted 1.00 |
| s4 | The lack of context and phrasality of query suggestions emphasizes this problem. | background 0.86 | 0.08 | 0.02 | asserted 1.00 |
| s5 | In a multi-step approach, we combine the benefits of large language models, pairwise comparison, and Elo-based scoring to identify and quantify bias in English search query suggestions. | method 0.88 | 0.94 | 0.11 | asserted 1.00 |
| s6 | We apply our approach to the U.S. political news domain and compare bias in Google and Bing. | contribution 0.81 | 0.95 | 0.09 | asserted 0.91 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Search query suggestions affect users' interactions with search engines, which then influences the information they encounter. Thus, bias in search query suggestions can lead to exposure to biased search results and can impact opinion formation. This is especially critical in the political domain. Detecting and quantifying bias in web search engines is difficult due to its topic dependency, complexity, and subjectivity. The lack of context and phrasality of query suggestions emphasizes this problem. In a multi-step approach, we combine the benefits of large language models, pairwise comparison, and Elo-based scoring to identify and quantify bias in English search query suggestions. We apply our approach to the U.S. political news domain and compare bias in Google and Bing.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 3 sentences without a confident label
Uses the method or tool of 1 paper
-
The expected probability $E_T_1$ that $T_1$ with Elo rating $R_1$ wins against $T_2$ with Elo rating $R_2$ is defined as follows [ref] :
Elo Uncovered: Robustness and Best Practices in Language Model Evaluation · Bias Identification & Quantification · 0.95
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.