In Crowd Veritas: Leveraging Human Intelligence To Fight Misinformation

Michael Soprano

REVIEW 3 major objections 4 minor 295 references

In Crowd Veritas: Leveraging Human Intelligence To Fight Misinformation

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Non-expert crowd workers can reliably assess online truthfulness, this thesis shows.

desk verdict A solid, well-organized thesis compiling 13 peer-reviewed papers; the empirical work is careful, but the 'objectivity' claim overreaches relative to the expert-label gold standard. read the letter →

arxiv 2506.09221 v1 pith:H6QDUMYI submitted 2025-06-10 cs.IR cs.CYcs.SI

Michael Soprano This is my paper

classification cs.IRcs.CYcs.SI

keywords misinformationcrowdsourcingfact-checkingtruthfulnessassessmentcognitivebiaseslongitudinalstudiesexplainableAIE-BART

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This thesis sets out to show that non-expert crowd workers can assess the truthfulness of online statements well enough to support fact-checking at scale. It compares thousands of crowd judgments against expert labels from two public fact-checking archives, across three judgment scales, a longitudinal study on COVID-19 misinformation, and a seven-dimension truthfulness instrument. The thesis reports that aggregated non-expert judgments often align with expert assessments, that judgment quality improves with worker experience and with the recency of the statement, and that a multidimensional view of truthfulness captures distinct facets that help interpret judgments. It also identifies cognitive biases that distort fact-checking and proposes countermeasures, and it introduces E-BART, a neural model that jointly predicts truthfulness and generates human-readable explanations.

What carries the argument

The argument is carried by a pipeline of large-scale crowdsourcing experiments in which non-experts judge statements whose truthfulness has already been labeled by professional fact-checkers, so that agreement can be measured. Three judgment scales are used—a three-level scale, a six-level scale, and a 0-to-100 slider scale—and judgments are aggregated by mean or median before comparison. The longitudinal design repeats the task in multiple batches to isolate the effects of timing and experience. The multidimensional instrument asks workers to rate each statement on seven truthfulness dimensions. For bias, the machinery is a systematic literature review following standard reporting guidelines, plus controlled hypothesis-testing experiments with worker-trait questionnaires. For automation, the central object is the E-BART architecture, which places a Joint Prediction Head on the BART language model so that truthfulness classification and explanation generation share one training objective, with model calibration performed by temperature scaling.

What would settle it

Gather statements that two professional fact-checking organizations rate differently, or cases where later evidence shows the original expert rating was wrong, and run the same crowdsourcing protocol. If aggregated crowd judgments follow the expert labels rather than the independently verifiable facts, the core claim fails; crowd judgments that track documented facts where experts erred would strengthen it. A simpler check is temporal: the thesis predicts judgment quality falls as statements age, so a longitudinal dataset showing no such decline would count against the timing claim.

Watch

Extended reading notes

Core claim

On its own terms, the thesis establishes that the crowd can serve as a reliable source of truthfulness judgments. The central discovery is a converging body of evidence from controlled crowdsourcing experiments: agreement between aggregated non-expert judgments and expert fact-checker labels is high enough to make crowd-based fact-checking viable, especially when judgments are aggregated, when statements are assessed soon after publication, and when workers have prior experience. A second discovery is that truthfulness is not one-dimensional: seven dimensions—Correctness, Neutrality, Comprehensibility, Precision, Completeness, Speaker's Trustworthiness, and Informativeness—are largely non-redundant and jointly more informative than a single overall score. A third is that cognitive biases are systematic and measurable in crowd judgments, with 39 biases identified as relevant to fact-checking and specific worker traits associated with biased behavior. Finally, the E-BART model shows that a single architecture can jointly predict a truthfulness label and produce a coherent explanation, and that these machine-generated explanations help human judges detect misinformation.

Load-bearing premise

The load-bearing premise is that the labels produced by professional fact-checking organizations—the six-level U.S. archive and the three-level Australian archive used throughout—are the correct ground truth for truthfulness; if those labels are biased or wrong, then crowd agreement with them does not establish that non-experts can judge truthfulness.

Editorial extensions

If this is right

A crowd-based first-pass fact-checking layer becomes practical: non-expert judgments can flag suspicious claims for professional review, cutting the volume experts must inspect.
Longitudinal studies with repeated participation from the same workers are not only feasible but beneficial, since returning workers produce higher-quality judgments.
Truthfulness should be measured and modeled as multidimensional rather than as a single score, because the seven dimensions carry distinct information.
Automated fact-checking systems that generate explanations as part of the prediction, as E-BART does, can improve human skepticism and detection of misinformation.
Cognitive-bias countermeasures, selected from the 39 identified biases, can be embedded in task design to reduce systematic errors in crowd work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: if the agreement results generalize beyond political and COVID-19 claims, the same experimental pipeline could serve as real-time triage in which crowd labels prioritize claims for expert fact-checking; the thesis does not itself test that deployment.
Inference: because biases act differently on different truthfulness dimensions, debiasing interventions could be targeted per dimension and evaluated by tracking dimension-level error; this is a natural but untested extension.
Inference: the paradoxical link between reported belief in science and lower accuracy suggests domain confidence may be a moderating factor, so testing the protocol across scientific and health claims with varying controversy levels would clarify the scope.
Inference: the thesis's core claim is relative to expert-defined truthfulness; validating crowd judgment against independently verified facts would be a stronger test the thesis does not perform.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear [Colleague],

The thing to know: this is a PhD thesis, not a new research paper. It compiles 13 peer-reviewed articles by the candidate and collaborators into one coherent narrative. If you expect novel experimental results, you won't find them; if you want a thorough, well-organized synthesis of crowdsourced fact-checking work, this is it. The data release (OSF) and the detailed appendices (Crowd_Frame infrastructure) are genuinely useful assets.

What the thesis does well: it lays out three research lines — judgment scales, cognitive biases, automated fact-checking — and ties them together through a set of meta-questions. The experiments are large-scale (thousands of judgments, multiple platforms, longitudinal designs), and the statistical reporting is careful. The E-BART model for joint prediction and explanation is a sensible contribution, and the evaluation includes human-centered checks. The treatment of cognitive biases, with 39 biases and 11 countermeasures, is a reasonable synthesis even if the categories borrow from prior work.

The soft spots are real but not fatal. First, the central claim — that non-experts 'can objectively assess and categorize' truthfulness — is too strong. The thesis itself acknowledges in Section 1.2 that expert fact-checkers are human and susceptible to cognitive biases. If expert labels are the gold standard, then demonstrating crowd-expert agreement shows alignment with that standard, not objectivity in any absolute sense. The author could have hedged to 'can reliably approximate expert judgments' and the thesis would be on firmer ground. Second, because each chapter reproduces published findings, the novel contribution is the integration, not the evidence. That's fine for a thesis, but the arXiv version should be explicit that it's a compilation. Third, the stress-test's concern about expert bias is not addressed head-on; the limitations section discusses some issues, but the ground-truth assumption is left undefended.

Who should read it: people entering human-computation or fact-checking research, and practitioners who want an overview of the crowdsourcing landscape. It's a good reference for experimental design and known pitfalls.

Recommendation: I would send it to a serious referee, but conditional. The referee should push on the ground-truth assumption and on the 'objectivity' framing. The empirical core is solid and the data release is valuable. So: engage with it, but ask for a revised framing.

Referee Report

3 major / 4 minor

Summary. The thesis investigates whether non-expert crowd workers can reliably assess the truthfulness of online information. It is organized around three meta-research questions: crowdsourced truthfulness assessment (including judgment scales, longitudinal designs, and multidimensional judgments), cognitive biases in fact-checking, and automated joint prediction and explanation of truthfulness. The empirical core consists of large-scale crowdsourcing experiments on political and COVID-19 statements, with expert labels from PolitiFact and RMIT ABC Fact Check used as ground truth, plus a neural model called E-BART evaluated on FEVER, e-FEVER, and e-SNLI. The central claim is that non-expert judgments often align with expert assessments and that multidimensional crowd judgments are reliable across truthfulness dimensions.

Significance. If the central claim holds, the thesis offers several useful contributions: large released datasets of crowd truthfulness judgments, systematic evidence on judgment scales and longitudinal participation, a PRISMA-based review of cognitive biases relevant to fact-checking, an open-source crowdsourcing framework (Crowd_Frame), and an explainable model that jointly predicts truthfulness and generates explanations. Strengths include the scale of the data collection, the use of statistical testing, the grounding in peer-reviewed publications, and the public release of datasets and software. The main caveat is that the inferential target throughout is agreement with expert fact-checker labels, so the strength of the conclusions depends on what those labels are taken to represent.

major comments (3)

[§1.2, §1.10.1, Chapters 4–5, 7, 9]
[§1.10.1, Chapters 4–5, 7]
[§10.4.2 (RQ30), §1.10.2]

minor comments (4)

[§1.9, publication list]
[§4.2.2]
[§4.2.1, §4.4.1]
[§5.4.1, Figures 5.1–5.4]

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claims are empirical comparisons against external fact-checker labels, and the E-BART evaluation is held-out; the strongest conclusion overreaches a fallible gold standard, but no derivation reduces to its inputs by construction.

full rationale

This thesis is an empirical compilation: the central claims (RQ1, RQ5, RQ16) are direct measurements of agreement between crowd judgments and externally produced fact-checker labels from PolitiFact and RMIT ABC Fact Check. The main finding that non-expert judgments often align with expert assessments is a measured quantity that could have gone either way. The E-BART model (Chapter 10) is trained on FEVER, e-FEVER, and e-SNLI and evaluated on held-out test sets, including 3-fold cross-validation in Section 7.4.5, so its reported effectiveness is not a fitted-input prediction. The one load-bearing assumption is that expert labels are a valid gold standard for truthfulness; the thesis itself notes in Section 1.2 that fact-checking organizations rely on human judgment and are potentially susceptible to cognitive biases. This makes the Section 1.10.2 summary that non-expert crowds can objectively assess and categorize truthfulness an overreach in construct validity, but that is a correctness risk rather than circularity, because the thesis never defines reliable assessment as agreement with experts and then re-derives the same agreement as support. Self-citations are present (e.g., Section 1.1 cites prior work by the same group, and Chapter 5 chooses a single scale based on Chapter 4 findings), but those cited results are peer-reviewed, code- and data-released, and externally falsifiable, so they are real evidence and do not load-bear circularly. No step in the derivation chain reduces to its own inputs by construction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The thesis is empirical and relies on several domain assumptions rather than mathematical axioms. The central assumption is the validity of expert labels as ground truth. There are no invented physical entities; the E-BART model is a computational tool, not a postulated entity.

assumptions (3)

domain assumption Expert fact-checker labels (e.g., PolitiFact, RMIT ABC) are treated as ground truth for truthfulness.
All agreement metrics compare crowd judgments to these labels (Chapters 4, 5, 7). If the labels are wrong, the comparisons are meaningless.
domain assumption Ordinal truthfulness scales can be treated as interval scales for aggregation and analysis.
Section 4.2.2 justifies using the mean and treating categories as equidistant for practical purposes, though this is an approximation.
domain assumption Crowd workers on Amazon Mechanical Turk, Toloka, and Prolific are representative of the general population and provide non-expert judgments.
The thesis recruits workers from these platforms and generalizes findings beyond them.

how reviews work

0 comments

Cite this review

Pith. "Pith review of In Crowd Veritas: Leveraging Human Intelligence To Fight Misinformation." pith.science (2026). https://pith.science/paper/H6QDUMYI

@misc{pith2026250609221,
  author       = {Pith},
  title        = {Pith review of: In Crowd Veritas: Leveraging Human Intelligence To Fight Misinformation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H6QDUMYI}},
  note         = {Machine review of arXiv:2506.09221}
}

read the original abstract

The spread of online misinformation poses serious threats to democratic societies. Traditionally, expert fact-checkers verify the truthfulness of information through investigative processes. However, the volume and immediacy of online content present major scalability challenges. Crowdsourcing offers a promising alternative by leveraging non-expert judgments, but it introduces concerns about bias, accuracy, and interpretability. This thesis investigates how human intelligence can be harnessed to assess the truthfulness of online information, focusing on three areas: misinformation assessment, cognitive biases, and automated fact-checking systems. Through large-scale crowdsourcing experiments and statistical modeling, it identifies key factors influencing human judgments and introduces a model for the joint prediction and explanation of truthfulness. The findings show that non-expert judgments often align with expert assessments, particularly when factors such as timing and experience are considered. By deepening our understanding of human judgment and bias in truthfulness assessment, this thesis contributes to the development of more transparent, trustworthy, and interpretable systems for combating misinformation.

Figures

Figures reproduced from arXiv: 2506.09221 by the authors.

**Figure 4.1.** Distributions of judgment scores for S3: individual responses (top left), expert (gold) labels (top right), and aggregated crowd judgments (bottom). the judgments for S6 and S100 ( [PITH_FULL_IMAGE:figures/full_fig_p079_4_1.png] view at source ↗

**Figure 4.2.** Distributions of judgment scores for S6: individual responses (top left), expert (gold) labels (top right), and aggregated crowd judgments (bottom) [PITH_FULL_IMAGE:figures/full_fig_p080_4_2.png] view at source ↗

**Figure 4.3.** Distributions of judgment scores for S100: individual responses (top left), expert (gold) labels (top right), and aggregated crowd judgments (bottom) [PITH_FULL_IMAGE:figures/full_fig_p081_4_3.png] view at source ↗

Figures from the paper (67 more)

**Figure 4.4.** Figure 4.4: External agreement with PolitiFact and RMIT ABC Fact Check statements, [PITH_FULL_IMAGE:figures/full_fig_p083_4_4.png]

**Figure 4.5.** Figure 4.5: Pairwise agreement and relative frequency, computed for each HIT in the task. [PITH_FULL_IMAGE:figures/full_fig_p084_4_5.png]

**Figure 4.6.** Figure 4.6: Agreement between judgment scales for PolitiFact statements. [PITH_FULL_IMAGE:figures/full_fig_p085_4_6.png]

**Figure 4.7.** Figure 4.7: Agreement between judgment scales for RMIT ABC Fact Check statements. [PITH_FULL_IMAGE:figures/full_fig_p086_4_7.png]

**Figure 4.8.** Figure 4.8: α cuts sorted by decreasing values. mean, but it is again clear that the overall quality is rather low. Although the squares around the diagonal tend to be darker and contain higher values, there are many exceptions. These are mainly in the lower-left corners, indica…

**Figure 4.9.** Figure 4.9: Agreement with the ground truth using the median as the aggregation function [PITH_FULL_IMAGE:figures/full_fig_p088_4_9.png]

**Figure 4.10.** Figure 4.10: Agreement between S3 (first row) and S6 (second row), across RMIT ABC Fact Check (left column) and PolitiFact (right column). Aggregation function: majority vote [PITH_FULL_IMAGE:figures/full_fig_p089_4_10.png]

**Figure 4.11.** Figure 4.11: Agreement with ground truth for merged categories for PolitiFact. From top to [PITH_FULL_IMAGE:figures/full_fig_p091_4_11.png]

**Figure 5.1.** Figure 5.1: Agreement between the PolitiFact experts (x-axis) and the crowd judgments [PITH_FULL_IMAGE:figures/full_fig_p103_5_1.png]

**Figure 5.2.** Figure 5.2: Agreement between the PolitiFact experts (x-axis) and the crowd judgments [PITH_FULL_IMAGE:figures/full_fig_p105_5_2.png]

**Figure 5.3.** Figure 5.3: Crowd judgments transformed into coarser categories and then aggregated [PITH_FULL_IMAGE:figures/full_fig_p106_5_3.png]

**Figure 5.4.** Figure 5.4: Expert judgments merged into groups and then aggregated with the mean [PITH_FULL_IMAGE:figures/full_fig_p107_5_4.png]

**Figure 5.5.** Figure 5.5: Distribution of the ranks of the URLs selected by workers. [PITH_FULL_IMAGE:figures/full_fig_p110_5_5.png]

**Figure 5.6.** Figure 5.6: Effect of justification origin on worker accuracy. The left plot shows the absolute [PITH_FULL_IMAGE:figures/full_fig_p112_5_6.png]

**Figure 5.7.** Figure 5.7: Correlation between aggregated judgments (mean) across [PITH_FULL_IMAGE:figures/full_fig_p115_5_7.png]

**Figure 5.8.** Figure 5.8: Agreement between the PolitiFact expert labels (x-axis) and crowd judgments (y [PITH_FULL_IMAGE:figures/full_fig_p116_5_8.png]

**Figure 5.9.** Figure 5.9: Distribution of the ranks of the URLs selected by workers for all the batches. [PITH_FULL_IMAGE:figures/full_fig_p120_5_9.png]

**Figure 5.10.** Figure 5.10: Effect of justification origin on labeling error. Text copied or not copied from [PITH_FULL_IMAGE:figures/full_fig_p122_5_10.png]

**Figure 5.11.** Figure 5.11: Comparison of MAE and CEMORD for individual judgments from returning workers. Green indicates that the group on the y-axis has higher quality than the one on the x-axis; red indicates the opposite. The trend is consistent across both metrics. With the exception of …

**Figure 5.12.** Figure 5.12: Relative ordering of statements across batches according to MAE for each [PITH_FULL_IMAGE:figures/full_fig_p126_5_12.png]

**Figure 5.13.** Figure 5.13: MAE (aggregated by statement) plotted against the number of days elapsed between the date the statement was made and the date it was evaluated. Each point represents the MAE of a single statement within a batch. Dotted lines indicate the trend of MAE over time for e…

**Figure 6.1.** Figure 6.1: Number of workers who report 1, 2, or 3 previous experiences with longitudinal [PITH_FULL_IMAGE:figures/full_fig_p139_6_1.png]

**Figure 6.2.** Figure 6.2: Time elapsed in months since each previous experience with longitudinal studies [PITH_FULL_IMAGE:figures/full_fig_p140_6_2.png]

**Figure 6.3.** Figure 6.3: Number of sessions of the longitudinal study to which each reported experience [PITH_FULL_IMAGE:figures/full_fig_p140_6_3.png]

**Figure 6.4.** Figure 6.4: Time elapsed in days or months between the sessions of the longitudinal study [PITH_FULL_IMAGE:figures/full_fig_p141_6_4.png]

**Figure 6.5.** Figure 6.5: Duration in minutes or hours of the sessions of the longitudinal study to which [PITH_FULL_IMAGE:figures/full_fig_p142_6_5.png]

**Figure 6.6.** Figure 6.6: Crowdsourcing platforms where the longitudinal study to which each reported [PITH_FULL_IMAGE:figures/full_fig_p142_6_6.png]

**Figure 6.7.** Figure 6.7: Payment model of the longitudinal study to which each reported experience [PITH_FULL_IMAGE:figures/full_fig_p143_6_7.png]

**Figure 6.8.** Figure 6.8: Workers willingness to participate again in the longitudinal study to which each [PITH_FULL_IMAGE:figures/full_fig_p143_6_8.png]

**Figure 6.9.** Figure 6.9: Incentives that drive workers to participate in the longitudinal study to which [PITH_FULL_IMAGE:figures/full_fig_p146_6_9.png]

**Figure 6.10.** Figure 6.10: Completion claimed by workers of the longitudinal study to which each re [PITH_FULL_IMAGE:figures/full_fig_p146_6_10.png]

**Figure 6.11.** Figure 6.11: Incentives that drive workers to completing the longitudinal study to which [PITH_FULL_IMAGE:figures/full_fig_p147_6_11.png]

**Figure 6.12.** Figure 6.12: Reasons that limit the availability of longitudinal studies on crowdsourcing [PITH_FULL_IMAGE:figures/full_fig_p150_6_12.png]

**Figure 6.13.** Figure 6.13: Number of days workers would be happy to commit for a longitudinal study, [PITH_FULL_IMAGE:figures/full_fig_p151_6_13.png]

**Figure 6.14.** Figure 6.14: Reasons that drive workers to decline participation in longitudinal studies. [PITH_FULL_IMAGE:figures/full_fig_p152_6_14.png]

**Figure 6.15.** Figure 6.15: Preferred participation frequency in a longitudinal study according to workers. [PITH_FULL_IMAGE:figures/full_fig_p152_6_15.png]

**Figure 6.16.** Figure 6.16: Preferred session duration in hours for longitudinal studies according to work [PITH_FULL_IMAGE:figures/full_fig_p153_6_16.png]

**Figure 6.17.** Figure 6.17: Acceptable hourly payment in USD$ for participation in longitudinal studies [PITH_FULL_IMAGE:figures/full_fig_p153_6_17.png]

**Figure 6.18.** Figure 6.18: Preferred amount of time in hours to allocate on a daily basis for participating [PITH_FULL_IMAGE:figures/full_fig_p154_6_18.png]

**Figure 6.19.** Figure 6.19: Incentives that drive workers to participate in new longitudinal studies accord [PITH_FULL_IMAGE:figures/full_fig_p155_6_19.png]

**Figure 6.20.** Figure 6.20: Tasks type that workers would like to perform in a longitudinal studies accord [PITH_FULL_IMAGE:figures/full_fig_p155_6_20.png]

**Figure 6.21.** Figure 6.21: Benefits of being involved in longitudinal studies according to the workers. [PITH_FULL_IMAGE:figures/full_fig_p156_6_21.png]

**Figure 6.22.** Figure 6.22: Downsides of being involved in longitudinal studies according to the workers. [PITH_FULL_IMAGE:figures/full_fig_p157_6_22.png]

**Figure 6.23.** Figure 6.23: Summary of the barriers emerged from our analyses, along with 8 recommen [PITH_FULL_IMAGE:figures/full_fig_p170_6_23.png]

**Figure 7.1.** Figure 7.1: Abandonment rate shown as the number of workers reaching each task step. [PITH_FULL_IMAGE:figures/full_fig_p176_7_1.png]

**Figure 7.2.** Figure 7.2: Correlation between dimensions: individual judgments are shown in the lower [PITH_FULL_IMAGE:figures/full_fig_p178_7_2.png]

**Figure 7.3.** Figure 7.3: Crowd judgments aggregated by mean for three dimensions: [PITH_FULL_IMAGE:figures/full_fig_p180_7_3.png]

**Figure 7.4.** Figure 7.4: Correlation with the ground truth of Overall Truthfulness and a sample of the other dimensions. PolitiFact has been grouped into 3 bins. Mean used as aggregation function. Compare to [PITH_FULL_IMAGE:figures/full_fig_p181_7_4.png]

**Figure 7.5.** Figure 7.5: Average time (in seconds) spent by workers to judge the [PITH_FULL_IMAGE:figures/full_fig_p182_7_5.png]

**Figure 7.6.** Figure 7.6: Principal component analysis (PCA) of the statements [PITH_FULL_IMAGE:figures/full_fig_p185_7_6.png]

**Figure 7.7.** Figure 7.7: Truthfulness dimensions first aggregated using the mean function then com [PITH_FULL_IMAGE:figures/full_fig_p186_7_7.png]

**Figure 7.8.** Figure 7.8: Weighted mean aggregation of Overall Truthfulness judgments based on workers’ CRT scores, using the original PolitiFact and RMIT ABC Fact Check categories (left plot) and a the PolitiFact categories grouped into three bins. Compare with [PITH_FULL_IMAGE:figures/full…

**Figure 7.9.** Figure 7.9: Effectiveness metrics across the 3 folds for the RMIT ABC Fact Check 30-level [PITH_FULL_IMAGE:figures/full_fig_p192_7_9.png]

**Figure 7.10.** Figure 7.10: Visualization of the embedding space. Each dimension is compared to the [PITH_FULL_IMAGE:figures/full_fig_p196_7_10.png]

**Figure 8.1.** Figure 8.1: The PRISMA flow diagram for new systematic reviews which included searches of databases, registers and other sources. Adapted from Page et al. [386]. several extensions3 have been developed to facilitate the reporting of different types or aspects of systematic revie…

**Figure 8.2.** Figure 8.2: Data collection and selection process of the review. [PITH_FULL_IMAGE:figures/full_fig_p205_8_2.png]

**Figure 9.1.** Figure 9.1: Systematic differences in crowd worker bias metrics ( [PITH_FULL_IMAGE:figures/full_fig_p225_9_1.png]

**Figure 9.2.** Figure 9.2: Comparison of abandonment and failure rates. Orange: current task (Sec [PITH_FULL_IMAGE:figures/full_fig_p231_9_2.png]

**Figure 9.3.** Figure 9.3: Agreement between workers’ judgments and ground truth for three dimensions, [PITH_FULL_IMAGE:figures/full_fig_p232_9_3.png]

**Figure 9.4.** Figure 9.4: Scatter plots showing the relationships between workers’ [PITH_FULL_IMAGE:figures/full_fig_p235_9_4.png]

**Figure 10.1.** Figure 10.1: The Joint Prediction Head of the E-BART architecture. RQ29 Can such a model achieve both accurate classification decisions and high-quality natural language explanations? RQ30 Are machine-generated explanations useful for helping humans better judge information tru…

**Figure 10.2.** Figure 10.2: The inference process of the E-BART architecture. To facilitate classification, the hidden state embeddings corresponding to the final sequence separator token (</s> in BART) are extracted and passed to a small feed-forward network that shapes the output to match t…

**Figure 10.3.** Figure 10.3: The training configuration of the E-BART architecture. extracts the embeddings corresponding to the token immediately before the final sequence separator token. This step is performed to mirror the training process. The extracted embeddings are passed to the classif…

**Figure 10.4.** Figure 10.4: External agreement between ground truth and crowd for raw (first column) [PITH_FULL_IMAGE:figures/full_fig_p250_10_4.png]

**Figure 10.5.** Figure 10.5: External agreement between ground truth and crowd for raw (left chart) and [PITH_FULL_IMAGE:figures/full_fig_p251_10_5.png]

**Figure 10.6.** Figure 10.6: Reliability diagrams for the E-BARTFull model before and after calibration. Calibration is performed using Temperature Scaling [209] to align predicted confidence with observed accuracy. dotted 45-degree line in the reliability diagrams represents perfect calibratio…

**Figure 11.1.** Figure 11.1: Example of review labelling based on the adopted argumentation-theoretic [PITH_FULL_IMAGE:figures/full_fig_p263_11_1.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

295 extracted references · 84 canonical work pages

[1]

doi:10.1007/978-3-642-40802-1_31

doi:10.1007/978-3-642-40802-1_31
[2]

Goal-Setting Behavior of Workers on Crowd- sourcing Platforms: An Exploratory Study on MTurk and Prolific

Tahir Abbas and Ujwal Gadiraju. Goal-Setting Behavior of Workers on Crowd- sourcing Platforms: An Exploratory Study on MTurk and Prolific. In Jane Hsu and Ming Yin, editors,Proceedings of the Tenth AAAI Conference on Human Computa- tion and Crowdsourcing, volume 10, pages 2–13, Washington, DC, 2022. AAAI Press. doi:10.1609/hcomp.v10i1.21983

doi:10.1609/hcomp.v10i1.21983 2022
[3]

Tukey’s Honestly Significant Difference (HSD) Test.EncyclopediaofResearchDesign,3(1):1–5,2010

Hervé Abdi and Lynne J Williams. Tukey’s Honestly Significant Difference (HSD) Test.EncyclopediaofResearchDesign,3(1):1–5,2010. URLhttps://personal.utdalla s.edu/~Herve/abdi-HSD2010-pretty.pdf

2010
[4]

Cognitive Attraction and Online Misinformation.Palgrave Communi- cations, 5(1), 2 2019

Alberto Acerbi. Cognitive Attraction and Online Misinformation.Palgrave Communi- cations, 5(1), 2 2019. ISSN 2055-1045. doi:10.1057/s41599-019-0224-y

doi:10.1057/s41599-019-0224-y 2019
[5]

Wiley Series in Probability and Statistics.Wiley,2ndedition,2010

Alan Agresti.Analysis of Ordinal Categorical Data. Wiley Series in Probability and Statistics.Wiley,2ndedition,2010. ISBN9780470082898. doi:10.1002/9780470594001

doi:10.1002/9780470594001 2010
[6]

Explainable Fact Checking with Probabilistic Answer Set Programming

Naser Ahmadi, Joohyung Lee, Paolo Papotti, and Mohammed Saeed. Explainable Fact Checking with Probabilistic Answer Set Programming. In Maria Liakata and Andreas Vlachos, editors,Proceedings of the 2019 Truth and Trust Online Conference (TTO 2019), 2019. URLhttps://truthandtrustonline.com/wp-content/uploads /2019/09/paper_15.pdf

2019
[7]

Aiken.Rating Scales And Checklists: Evaluating Behavior, Personality, And Attitudes.Ratingscalesandchecklists: Evaluatingbehavior,personality,andattitudes

Lewis R. Aiken.Rating Scales And Checklists: Evaluating Behavior, Personality, And Attitudes.Ratingscalesandchecklists: Evaluatingbehavior,personality,andattitudes. John Wiley & Sons, Oxford, England, 1996. ISBN 0-471-12787-6

1996
[8]

FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP

Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP. In Waleed Ammar, Annie Louis, and Nasrin Mostafazadeh, editors,Proceedings of the 357 BIBLIOGRAPHY 358 of 455 2019 Conference of the North American Chapter of the Association for Computational Lin- guisti...

doi:10.18653/v1/n19-4010 2019

Show all 295 references

[9]

Fighting the COVID-19 Infodemic: Mod- eling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society

Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Dar- wish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gĳs Danoe, Friso Stolk, Britt Bruntink, and Preslav Nakov. Fighti...

2021
[10]

Evaluating Topic Representations for Exploring Document Collections.Journal of the Association for Information Science and Technology, 68(1):154–167, 2017

Nikolaos Aletras, Timothy Baldwin, Jey Han Lau, and Mark Stevenson. Evaluating Topic Representations for Exploring Document Collections.Journal of the Association for Information Science and Technology, 68(1):154–167, 2017. doi:10.1002/asi.23574

2017 doi
[11]

Where is Your Evidence: ImprovingFact-checkingbyJustificationModeling

Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. Where is Your Evidence: ImprovingFact-checkingbyJustificationModeling. InProceedingsoftheFirstWorkshop on Fact Extraction and VERification (FEVER), pages 85–90, Brussels, Belgium, 11 2018. Association for Computational Ling...

2018 doi
[12]

ConductingMaliciousCybersecurityExperiments on Crowdsourcing Platforms

AsmaaAljohaniandJamesJones. ConductingMaliciousCybersecurityExperiments on Crowdsourcing Platforms. InThe 2021 3rd International Conference on Big Data Engineering, BDE 2021, page 150–161. Association for Computing Machinery, 2021. ISBN 9781450389426. doi:10.1145/3468920.3468942

2021
[13]

InHCI for Cybersecurity, Privacy and Trust, pages 307– 325, Cham, 2022

AsmaaAljohaniandJamesJones.ThePitfallsofEvaluatingCyberDefenseTechniques by an Anonymous Population. InHCI for Cybersecurity, Privacy and Trust, pages 307– 325, Cham, 2022. Springer International Publishing. ISBN 978-3-031-05563-8

2022
[14]

Jennifer Allen, Baird Howland, Markus Mobius, David Rothschild, and Duncan J. Watts. Evaluating The Fake News Problem At The Scale Of The Information Ecosys- tem.Science Advances, 6(14), 2020. doi:10.1126/sciadv.aay3539

2020 doi
[15]

Arechar, Gordon Pennycook, and David G

Jennifer Allen, Antonio A. Arechar, Gordon Pennycook, and David G. Rand. Scal- ing Up Fact-checking Using The Wisdom of Crowds.Science Advances, 7(36), 2021. doi:10.1126/sciadv.abf4393

2021 doi
[16]

Using Crowdsourcing For TREC Rele- vance Assessment.Information Processing & Management, 48(6):1053–1066, 2012

Omar Alonso and Stefano Mizzaro. Using Crowdsourcing For TREC Rele- vance Assessment.Information Processing & Management, 48(6):1053–1066, 2012. doi:10.1016/j.ipm.2012.01.004

2012 doi
[17]

AReasoningModelBasedontheProductionof AcceptableArguments.AnnalsofMathematicsandArtificialIntelligence,34(1):197–215, 3 2002

LeilaAmgoudandClaudetteCayrol. AReasoningModelBasedontheProductionof AcceptableArguments.AnnalsofMathematicsandArtificialIntelligence,34(1):197–215, 3 2002. ISSN 1573-7470. doi:10.1023/A:1014490210693

2002 doi
[18]

Two Roles of Preferences in Argumentation Frame- works

Leila Amgoud and Srdjan Vesic. Two Roles of Preferences in Argumentation Frame- works. In Weiru Liu, editor,Symbolic and Quantitative Approaches to Reasoning 359 of 455 BIBLIOGRAPHY with Uncertainty, pages 86–97, Berlin, Heidelberg, 2011. Springer Berlin Heidelberg. doi:10.100...

2011 doi
[19]

Overview of RepLab 2013: Evaluating Online Reputation Monitoring Systems

Enrique Amigó, Jorge Carrillo de Albornoz, Irina Chugur, Adolfo Corujo, Julio Gon- zalo, Tamara Martín, Edgar Meĳ, Maarten de Rĳke, and Damiano Spina. Overview of RepLab 2013: Evaluating Online Reputation Monitoring Systems. In Pamela Forner, Henning Müller, Roberto Paredes, P...

2013
[20]

A General Evaluation Mea- sure for Document Organization Tasks

Enrique Amigó, Julio Gonzalo, and Felisa Verdejo. A General Evaluation Mea- sure for Document Organization Tasks. InProceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’13, page 643–652. Association for Computi...

2013
[21]

An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimen- tal Results

EnriqueAmigó,JulioGonzalo,StefanoMizzaro,andJorgeCarrillo-de-Albornoz. An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimen- tal Results. InProceedings of the 58th Annual Meeting of the Association for Computa- tionalLinguistics,pages3938–3949.As...

doi
[22]

Anderson, Pedro Domingos, and Daniel S

Corin R. Anderson, Pedro Domingos, and Daniel S. Weld. Relational Markov Mod- els and Their Application to Adaptive Web Navigation. InProceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’02, page 143–152. Association for Co...

2002
[23]

The Psychology of Sunk Cost.Organiza- tional Behavior And Human Decision Processes, 35(1):124–140, 1985

Hal R Arkes and Catherine Blumer. The Psychology of Sunk Cost.Organiza- tional Behavior And Human Decision Processes, 35(1):124–140, 1985. ISSN 0749-5978. doi:10.1016/0749-5978(85)90049-4

1985 doi
[24]

Armstrong, Ilinca A

Bonnie A. Armstrong, Ilinca A. Dutescu, Arthur Tung, Diana N. Carter, Patricia L. Trbovich, Sherman Wong, Gustavo Saposnik, and Teodor Grantcharov. Cognitive Biases in Surgery: Systematic Review.British Journal of Surgery, 110(6):645–654, 02
[25]

Automatic Fact-CheckingUsingContextandDiscourseInformation.JournalofDataandInforma- tion Quality, 11(3), 5 2019

Pepa Atanasova, Preslav Nakov, Lluís Màrquez, Alberto Barrón-Cedeño, Georgi Karadzhov, Tsvetomila Mihaylova, Mitra Mohtarami, and James Glass. Automatic Fact-CheckingUsingContextandDiscourseInformation.JournalofDataandInforma- tion Quality, 11(3), 5 2019. ISSN 1936-1955. doi:1...

2019 doi
[26]

GeneratingFactCheckingExplanations

Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. GeneratingFactCheckingExplanations. InProceedingsofthe58thAnnualMeetingofthe Association for Computational Linguistics, pages 7352–7364, Online, 7 2020. Association for Computational Linguistics. do...

2020 doi
[27]

Towards Artificial Argumentation.AI Magazine, 38(3):25–36, 10 2017

Katie Atkinson, Pietro Baroni, Massimiliano Giacomin, Anthony Hunter, Henry Prakken, Chris Reed, Guillermo Simari, Matthias Thimm, and Serena Vil- lata. Towards Artificial Argumentation.AI Magazine, 38(3):25–36, 10 2017. doi:10.1609/aimag.v38i3.2704

2017 doi
[28]

Auer, Tara S

Elena M. Auer, Tara S. Behrend, Andrew B. Collmus, Richard N. Lan- ders, and Ahleah F. Miles. Pay For Performance, Satisfaction And Reten- tion In Longitudinal Crowdsourced Research.PLOS ONE, 16(1):1–17, 01 2021. doi:10.1371/journal.pone.0245460

2021 doi
[29]

Awad and Issa Khalil

Mamoun A. Awad and Issa Khalil. Prediction of User’s Web-Browsing Behavior: Application of Markov Model.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(4):1131–1142, 2012. doi:10.1109/TSMCB.2012.2187441

2012
[30]

Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval

Leif Azzopardi. Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval. InProceedings of the 2021 Conference on Human In- formation Interaction and Retrieval, CHIIR ’21, page 27–37. Association for Computing Machinery, 2021. ISBN 97814...

2021
[31]

Bias On The Web.Communications of the ACM, 61(6):54–61, 5

Ricardo Baeza-Yates. Bias On The Web.Communications of the ACM, 61(6):54–61, 5
[32]

Associa- tion for Computing Machinery, New York, NY, USA, 2020

Ricardo Baeza-Yates.Bias in Search and Recommender Systems, page 2. Associa- tion for Computing Machinery, New York, NY, USA, 2020. ISBN 9781450375832. doi:10.1145/3383313.3418435

2020
[33]

Threat-related Attentional Bias in Anxious And Nonanxious Individuals: A Meta-analytic Study.Psychol

Yair Bar-Haim, Dominique Lamy, Lee Pergamin, Marian J Bakermans-Kranenburg, and Marinus H van Ĳzendoorn. Threat-related Attentional Bias in Anxious And Nonanxious Individuals: A Meta-analytic Study.Psychol. Bull., 133(1):1–24, 1 2007. doi:10.1037/0033-2909.133.1.1

2007 doi
[34]

James H. Barnes. Cognitive Biases and Their Impact on Strategic Planning.Strategic Management Journal, 5(2):129–137, 1984. ISSN 01432095, 10970266. URLhttp://www. jstor.org/stable/2486172

1984
[35]

Outcome Bias in Decision Evaluation.Journal of personality and social psychology, 54(4):569, 1988

Jonathan Baron and John C Hershey. Outcome Bias in Decision Evaluation.Journal of personality and social psychology, 54(4):569, 1988. URLhttps://www.sas.upenn.ed u/~baron/papers/outcomebias.pdf

1988
[36]

AbstractArgumentationFrameworks and their Semantics

P.Baroni,MartinCaminada,andM.Giacomin. AbstractArgumentationFrameworks and their Semantics. In P. Baroni, D. Gabbay, and M. Giacomin, editors,Handbook of Formal Argumentation, chapter 4, pages 159–236. College Publications, 2018. ISBN 978-1-84890-275-6. URLhttp://www.collegepu...

2018
[37]

An Introduction To Argumentation Semantics.The Knowledge Engineering Review, 26(4):365–410, 2011

Pietro Baroni, Martin Caminada, and Massimiliano Giacomin. An Introduction To Argumentation Semantics.The Knowledge Engineering Review, 26(4):365–410, 2011. doi:10.1017/S0269888911000166. 361 of 455 BIBLIOGRAPHY

2011 doi
[38]

Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media

Alberto Barrón-Cedeño, Tamer Elsayed, Preslav Nakov, Giovanni Da San Martino, Maram Hasanain, Reem Suwaileh, Fatima Haouari, Nikolay Babulkov, Bayan Ham- dan, Alex Nikolov, Shaden Shaar, and Zien Sheikh Ali. Overview of CheckThat! 2020: Automatic Identification and Verificatio...

2020
[39]

Barry and Linda Schamber

Carol L. Barry and Linda Schamber. Users’ Criteria For Relevance Evaluation: A Cross-situational Comparison.Information Processing & Management, 34(2):219–236,
[40]

Karen W. Bauer. Conducting Longitudinal Studies.New Directions for Institutional Research, 2004(121):75–90, 2004. doi:10.1002/ir.102

2004 doi
[41]

doi:10.1007/978-3-030-58219-7_17

doi
[42]

KeyingHashFunctionsforMessage Authentication

MihirBellare,RanCanetti,andHugoKrawczyk. KeyingHashFunctionsforMessage Authentication. In Neal Koblitz, editor,Advances in Cryptology — CRYPTO ’96, pages 1–15, Berlin, Heidelberg, 1996. Springer Berlin Heidelberg. ISBN 978-3-540-68697-2. doi:10.1007/3-540-68697-5_1

1996 doi
[43]

To Pay or Not to Pay? Handling Crowdsourced Participants Who Drop Out From a Research Study.Ethics and Information Technology, 25(3):34, 6

Raquel Benbunan-Fich. To Pay or Not to Pay? Handling Crowdsourced Participants Who Drop Out From a Research Study.Ethics and Information Technology, 25(3):34, 6
[44]

Trevor J. M. Bench-Capon. Persuasion in Practical Argument Using Value-based ArgumentationFrameworks.JournalofLogic andComputation, 13(3):429–448, 06 2003. ISSN 0955-792X. doi:10.1093/logcom/13.3.429

2003 doi
[45]

Behrend, David J

Tara S. Behrend, David J. Sharek, Adam W. Meade, and Eric N. Wiebe. The Viability of Crowdsourcing For Survey Research.Behavior Research Methods, 43(3):800, 3 2011. ISSN 1554-3528. doi:10.3758/s13428-011-0081-0

2011 doi
[46]

Char- acterizing User Behavior in Online Social Networks

Fabrício Benevenuto, Tiago Rodrigues, Meeyoung Cha, and Virgílio Almeida. Char- acterizing User Behavior in Online Social Networks. InProceedings of the 9th ACM SIGCOMM Conference on Internet Measurement, IMC ’09, page 49–62. Association for Computing Machinery, 2009. ISBN 978...

2009
[47]

Zhang, Connie Moon Sehat, and Tanushree Mitra

Md Momen Bhuiyan, Amy X. Zhang, Connie Moon Sehat, and Tanushree Mitra. Investigating Differences in Crowdsourced News Credibility Assessment: Raters, Tasks, and Expert Criteria.Proceedings of the ACM on Human-Computer Interaction, 4 (CSCW2), 10 2020. doi:10.1145/3415164. BIBL...

2020 doi
[48]

doi:10.1007/s10676-023-09708-8

ISSN 1572-8439. doi:10.1007/s10676-023-09708-8

doi
[49]

PedagogisktUtvecklingsarbetevidStockholmsSkolor.6

C.H.Björnsson.Läsbarhet:. PedagogisktUtvecklingsarbetevidStockholmsSkolor.6. Liber, 1968. ISBN 99-0346315-4. URLhttp://libris.kb.se/bib/8079176

1968
[50]

Trevor J. M. Bench-Caponm. Value-based Agumentation Frameworks. In Salem BenferhatandEnricoGiunchiglia,editors,Proceedingsofthe9thInternationalWorkshop on Non-Monotonic Reasoning, pages 443–454, 2002. URLhttps://nmr.cs.tu-dortm und.de/proceedings/NMR2002Proceedings.pdf

2002
[51]

Bland and Douglas G

Martin J. Bland and Douglas G. Altman. Multiple significance tests: the Bonferroni method.BMJ, 310(6973):170, 1995. ISSN 0959-8138. doi:10.1136/bmj.310.6973.170

1995 doi
[52]

Calorie Posting in Chain Restau- rants.American Economic Journal: Economic Policy, 3(1):91–128, 2011

Bryan Bollinger, Phillip Leslie, and Alan Sorensen. Calorie Posting in Chain Restau- rants.American Economic Journal: Economic Policy, 3(1):91–128, 2011. ISSN 19457731, 1945774X. doi:10.1257/pol.3.1.91

2011 doi
[53]

1 in 4 Americans Own a Smart Speaker

Kourtney Bitterly. 1 in 4 Americans Own a Smart Speaker. What Does That Mean for News?New York Times, 2019. URLhttps://open.nytimes.com/how-might-the-n ew-york-times-sound-on-smart-speakers-3b59a6a78ae3

2019
[54]

Bowman, Gabor Angeli, Christopher Potts, and Christopher D

Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Man- ning. A Large Annotated Corpus For Learning Natural Language Inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, 9 2015...

2015 doi
[55]

Martin Bland and Douglas G

J. Martin Bland and Douglas G. Altman. Statistics Notes: Bootstrap Resampling Methods.BMJ, 350, 2015. doi:10.1136/bmj.h2622

2015 doi
[56]

E-BART: JointlyPredictingandExplainingTruthfulness

Erik Brand, Kevin Roitero, Michael Soprano, and Gianluca Demartini. E-BART: JointlyPredictingandExplainingTruthfulness. InIsabelleAugenstein,PaoloPapotti, and Dustin Wright, editors,Proceedings of the 2021 Truth and Trust Online Conference, pages 18–27, 10 2021. URLhttps://tru...

2021
[57]

A Neural Model to Jointly Predict and Explain Truthfulness of Statements.Journal of Data and Information Quality, 15(1), 12 2022

ErikBrand,KevinRoitero,MichaelSoprano,AfshinRahimi,andGianlucaDemartini. A Neural Model to Jointly Predict and Explain Truthfulness of Statements.Journal of Data and Information Quality, 15(1), 12 2022. ISSN 1936-1955. doi:10.1145/3546917

2022 doi
[58]

EvaluatingVariable-LengthMarkovChainModelsfor Analysis of User Web Navigation Sessions.IEEE Transactions on Knowledge and Data Engineering, 19(4):441–452, 2007

JoseBorgesandMarkLevene. EvaluatingVariable-LengthMarkovChainModelsfor Analysis of User Web Navigation Sessions.IEEE Transactions on Knowledge and Data Engineering, 19(4):441–452, 2007. doi:10.1109/TKDE.2007.1012

2007
[59]

RandomForests.MachineLearning,45(1):5–32,102001

LeoBreiman. RandomForests.MachineLearning,45(1):5–32,102001. ISSN1573-0565. doi:10.1023/A:1010933404324. 363 of 455 BIBLIOGRAPHY

doi
[60]

The Google Effect: Googling, Blogging, Wikis and the Flattening of Expertise.International Journal of Libraries and Information Studies, 56(3):157–167, 2006

Tara Brabazon. The Google Effect: Googling, Blogging, Wikis and the Flattening of Expertise.International Journal of Libraries and Information Studies, 56(3):157–167, 2006. doi:doi:10.1515/LIBR.2006.157

2006 doi
[61]

Towards a Framework for Gami- fication Design on Crowdsourcing Systems: The G.A.M.E

Jailson Brito, Vaninha Vieira, and Adolfo Duran. Towards a Framework for Gami- fication Design on Crowdsourcing Systems: The G.A.M.E. Approach. In2015 12th International Conference on Information Technology - New Generations, pages 445–450,
[62]

ISSN 0747-5632

GabrielA.BrooksandLukeClark.TheGamblersofTheFuture? MigrationFromLoot Boxes to Gambling in a Longitudinal Study of Young Adults.Computers in Human Behavior, 141:107605, 2023. ISSN 0747-5632. doi:10.1016/j.chb.2022.107605

2023
[63]

Brashier, Emmaline Drew Eliseev, and Elizabeth J

Nadia M. Brashier, Emmaline Drew Eliseev, and Elizabeth J. Marsh. An Initial Ac- curacy Focus Prevents Illusory Truth.Cognition, 194:104054, 2020. ISSN 0010-0277. doi:10.1016/j.cognition.2019.104054

2020
[64]

Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling. Amazon’s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?Perspectives on Psycho- logical Science, 6(1):3–5, 2011. doi:10.1177/1745691610393980

2011 doi
[65]

Briguez, Maximiliano C.D

Cristian E. Briguez, Maximiliano C.D. Budán, Cristhian A.D. Deagustini, Ana G. Maguitman, Marcela Capobianco, and Guillermo R. Simari. Argument-based Mixed Recommenders And Their Application To Movie Suggestion.Expert Systems with Applications,41(14):6467–6482,2014. ISSN0957-4...

2014 doi
[66]

Brad J. Bushman. Violent Media and Hostile Appraisals: A Meta-analytic Review. Aggressive Behavior, 42(6):605–613, 2016. doi:10.1002/ab.21655

2016 doi
[67]

Bryson, and Arvind Narayanan

Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. Semantics Derived Auto- matically From Language Corpora Contain Human-like Biases.Science, 356(6334): 183–186, 2017. doi:10.1126/science.aal4230

2017 doi
[68]

e-SNLI: Natural Language Inference with Natural Language Explanations

Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e-SNLI: Natural Language Inference with Natural Language Explanations. In Samy Bengio,HannaM.Wallach,HugoLarochelle,KristenGrauman,NicolòCesa-Bianchi, and Roman Garnett, editors,Advances in Neural Infor...

2018
[69]

Language Models are Few-Shot Learners

TomBrown,BenjaminMann,NickRyder,MelanieSubbiah,JaredDKaplan,Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemen...

1901
[70]

Sports Rorts

Andrea Carson, Andrew Gibbons, Aaron Martin, and Justin B. Phillips. Does Third-Party Fact-Checking Increase Trust in News Stories? An Australian Case Study Using the “Sports Rorts” Affair.Digital Journalism, 10(5):801–822, 2022. doi:10.1080/21670811.2022.2031240

2022 arXiv
[71]

Mapping the Landscape of Artificial Intelli- gence Applications against COVID-19.Journal of Artificial Intelligence Research, 11

Joseph Bullock, Alexandra Luccioni, Katherine Hoffmann Pham, Cynthia Sin Nga Lam, and Miguel Luengo-Oroz. Mapping the Landscape of Artificial Intelli- gence Applications against COVID-19.Journal of Artificial Intelligence Research, 11
[72]

Cognitive Biases: Their ContributionforUnderstandingHumanCognitiveProcesses

Jean-Paul Caverni, Jean-Marc Fabre, and Michel Gonzalez. Cognitive Biases: Their ContributionforUnderstandingHumanCognitiveProcesses. volume68ofAdvances in Psychology, pages 7–12. North-Holland, 1990. doi:10.1016/S0166-4115(08)61311-4

1990 doi
[73]

MethodologiesforDeterminingReadingRequirementsofMilitaryOccu- pational Specialists

J.S.Caylor. MethodologiesforDeterminingReadingRequirementsofMilitaryOccu- pational Specialists. Technical report, 1973. URLhttp://files.eric.ed.gov/full text/ED074343.pdf

1973
[74]

Ceci and Wendy M

Stephen J. Ceci and Wendy M. Williams. The Psychology of Fact-checking.Scientific American, pages 7–13, 2020. URLhttps://www.scientificamerican.com/article /the-psychology-of-fact-checking1/

2020
[75]

Most Border Wall Opponents, Supporters Say Shutdown Concessions Are Unacceptable

Pew Research Center. Most Border Wall Opponents, Supporters Say Shutdown Concessions Are Unacceptable. URLhttps://www.people-press.org/2019/01/ 16/most-border-wall-opponents-supporters-say-shutdown-concessions-are -unacceptable/

2019
[76]

Workers in The Crowd: The Labor Market Impact of The Online Platform Economy.Industrial and Corporate Change, 30(6):1429– 1458, 7 2021

Michele Cantarella and Chiara Strozzi. Workers in The Crowd: The Labor Market Impact of The Online Platform Economy.Industrial and Corporate Change, 30(6):1429– 1458, 7 2021. ISSN 0960-6491. doi:10.1093/icc/dtab022

2021 doi
[77]

Assess- ing the Quality of Online Reviews Using Formal Argumentation Theory

Davide Ceolin, Giuseppe Primiero, Jan Wielemaker, and Michael Soprano. Assess- ing the Quality of Online Reviews Using Formal Argumentation Theory. In Marco Brambilla,RichardChbeir,FlaviusFrasincar,andIoanaManolescu,editors,WebEngi- neering,pages71–87.SpringerInternationalPubl...

2021 doi
[78]

Longitudinal Studies.Journal of Thoracic Disease, 7(11):537–540, 2015

Edward Joseph Caruana, Marius Roman, Jules Hernández-Sánchez, and Piergiorgio Solli. Longitudinal Studies.Journal of Thoracic Disease, 7(11):537–540, 2015. ISSN 2077-6624. doi:10.3978/j.issn.2072-1439.2015.10.63

2015
[79]

Watch ’n’ Check: Towards a Social Media Monitoring Tool 365 of 455 BIBLIOGRAPHY to Assist Fact-Checking Experts

Assunta Cerone, Elham Naghizade, Falk Scholer, Devi Mallal, Russell Skelton, and Damiano Spina. Watch ’n’ Check: Towards a Social Media Monitoring Tool 365 of 455 BIBLIOGRAPHY to Assist Fact-Checking Experts. InProceedings of the 2020 IEEE 7th International Conference on Data ...

2020
[80]

Task Design in Complex Crowdsourcing Experiments: Item Assignment Optimization.Computers & Operations Research, 148:105995, 2022

Sara Ceschia, Kevin Roitero, Gianluca Demartini, Stefano Mizzaro, Luca Di Gaspero, and Andrea Schaerf. Task Design in Complex Crowdsourcing Experiments: Item Assignment Optimization.Computers & Operations Research, 148:105995, 2022. ISSN 0305-0548. doi:10.1016/j.cor.2022.105995

2022
[81]

Estimating Clickthrough Bias in the Cascade Model

Praveen Chandar and Ben Carterette. Estimating Clickthrough Bias in the Cascade Model. InProceedingsofthe27thACMInternationalConferenceonInformationandKnowl- edge Management, CIKM ’18, page 1587–1590. Association for Computing Machinery,
[82]

Chandler and Gabriele Paolacci

Jesse J. Chandler and Gabriele Paolacci. Lie for a Dime: When Most Prescreening ResponsesAreHonestbutMostStudyParticipantsAreImpostors.SocialPsychological and Personality Science, 8(5):500–508, 2017. doi:10.1177/1948550617698203

2017 doi
[83]

Capturing the Ineffable: Col- lecting, Analysing, and Automating Web Document Quality Assessments

Davide Ceolin, Julia Noordegraaf, and Lora Aroyo. Capturing the Ineffable: Col- lecting, Analysing, and Automating Web Document Quality Assessments. In Eva Blomqvist, Paolo Ciancarini, Francesco Poggi, and Fabio Vitali, editors,Knowledge Engineer, pages 83–97, Cham, 2016. Spri...

2016 doi
[84]

Let’sAgreetoDisagree: FixingAgreementMeasuresforCrowdsourcing

AlessandroChecco,KevinRoitero,EddyMaddalena,StefanoMizzaro,andGianluca Demartini. Let’sAgreetoDisagree: FixingAgreementMeasuresforCrowdsourcing. In Steven Dow and Adam Tauman Kalai, editors,Proceedings of the Fifth AAAI Confer- ence on Human Computation and Crowdsourcing, page...

2017
[85]

Trans- parent Assessment of Information Quality Of Online Reviews Using Formal Ar- gumentation Theory.Information Systems, 110:102107, 7 2022

Davide Ceolin, Giuseppe Primiero, Michael Soprano, and Jan Wielemaker. Trans- parent Assessment of Information Quality Of Online Reviews Using Formal Ar- gumentation Theory.Information Systems, 110:102107, 7 2022. ISSN 0306-4379. doi:10.1016/j.is.2022.102107

2022
[86]

Why Stu- dents Share Misinformation on Social Media: Motivation, Gender, and Study-level Differences.TheJournalofAcademicLibrarianship,41(5):583–592,2015

Xinran Chen, Sei-Ching Joanna Sin, Yin-Leng Theng, and Chei Sian Lee. Why Stu- dents Share Misinformation on Social Media: Motivation, Gender, and Study-level Differences.TheJournalofAcademicLibrarianship,41(5):583–592,2015. ISSN0099-1333. doi:10.1016/j.acalib.2015.07.003

2015 doi
[87]

Debiasing The Framing Effect: The Effect of Warning and Involvement.Decision Support Systems, 49(3):328–334, 2010

Fei-Fei Cheng and Chin-Shan Wu. Debiasing The Framing Effect: The Effect of Warning and Involvement.Decision Support Systems, 49(3):328–334, 2010. ISSN 0167-

2010
[88]

doi:10.1145/3274306

Chun-WeiChiang,AnnaKasunic,andSaiphSavage.CrowdCoach: PeerCoachingfor CrowdWorkers’SkillGrowth.ProceedingsoftheACMonHuman-ComputerInteraction, 2(CSCW), 11 2018. doi:10.1145/3274306

2018 doi
[89]

doi:10.1145/3269206.3269315

ISBN 9781450360142. doi:10.1145/3269206.3269315
[90]

Michael Chmielewski and Sarah C. Kucker. An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results.Social Psychological and Personality Science, 11(4): 464–473, 2020. doi:10.1177/1948550619875149

2020 doi
[91]

Pairwise, Magnitude, Or Stars: What’s the Best Way For Crowds To Rate? arXiv, 2016

Alessandro Checco and Gianluca Demartini. Pairwise, Magnitude, Or Stars: What’s the Best Way For Crowds To Rate? arXiv, 2016

2016
[92]

Vanderpool

Wen-Ying Sylvia Chou, Anna Gaysynsky, and Robin C. Vanderpool. The COVID-19 Misinfodemic: Moving Beyond Fact-checking.Health Education Behavior, 48(1):9–13, 2 2021. doi:10.1177/1090198120980675

2021 doi
[93]

Charles Chen, Sungchul Kim, Hung Bui, Ryan Rossi, Eunyee Koh, Branislav Kveton, andRazvanBunescu. PredictiveAnalysisbyLeveragingTemporalUserBehaviorand UserEmbeddings.InProceedingsofthe27thACMInternationalConferenceonInformation and Knowledge Management, CIKM ’18, page 2175–21...

2018
[94]

Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini

Giovanni Luca Ciampaglia, Prashant Shiralkar, Luis M. Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini. Computational Fact Checking from Knowledge Networks.PloS One, 10(6):1–13, 6 2015. doi:10.1371/journal.pone.0128193

2015 doi
[96]

The COVID-19 Social Media Infodemic.Scientific Reports, 10(1):16598, 10 2020

MatteoCinelli,WalterQuattrociocchi,AlessandroGaleazzi,CarloMicheleValensise, EmanueleBrugnoli,AnaLuciaSchmidt,PaolaZola,FabianaZollo,andAntonioScala. The COVID-19 Social Media Infodemic.Scientific Reports, 10(1):16598, 10 2020. ISSN 2045-2322. doi:10.1038/s41598-020-73510-5

2020 doi
[97]

The Echo Chamber Effect on Social Me- dia.Proceedings of the National Academy of Sciences, 118(9):e2023301118, 2021

Matteo Cinelli, Gianmarco Morales De Francisi, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. The Echo Chamber Effect on Social Me- dia.Proceedings of the National Academy of Sciences, 118(9):e2023301118, 2021. doi:10.1073/pnas.2023301118

2021 doi
[98]

Fung, and Michael K

Wai Ki Ching, Eric S. Fung, and Michael K. Ng. Higher-order Markov Chain Mod- els For Categorical Data Sequences*.Naval Research Logistics, 51(4):557–574, 2004. doi:0.1002/nav.20017. BIBLIOGRAPHY 366 of 455

2004
[99]

AnnaE.ClarkandYoshihisaKashima. StereotypesHelpPeopleConnectWithOthers in The Community: A Situated Functional Analysis of The Stereotype Consistency Bias in Communication.Journal of Personality and Social Psychology, 93(6):1028–1039, 12 2007. doi:10.1037/0022-3514.93.6.1028

2007 doi
[100]

Zhang, Wendy Armstrong, Daniel Cho, and Susan M

Junsang Cho, Grayson W. Zhang, Wendy Armstrong, Daniel Cho, and Susan M. Culican. Crowdsourcing and Its Applications to Ophthalmology.Expert Review of Ophthalmology, 18(2):113–119, 2023. doi:10.1080/17469899.2023.2200935

2023
[101]

Charles L. A. Clarke, Saira Rizvi, Mark D. Smucker, Maria Maistro, and Guido Zuc- con. OverviewoftheTREC2020HealthMisinformationTrack. InEllenM.Voorhees andAngelaEllis,editors,ProceedingsoftheTwenty-NinthTextREtrievalConference,vol- ume 1266 ofNIST Special Publication. Nationa...

2020
[102]

Cognitive Biases AmongMillennialIndianInvestors: DoPersonalityandDemographicFactorsMatter? FIIB Business Review, 13(1):106–117, 2024

Sangita Choudhary, Mohit Yadav, and Anugamini Priya Srivastava. Cognitive Biases AmongMillennialIndianInvestors: DoPersonalityandDemographicFactorsMatter? FIIB Business Review, 13(1):106–117, 2024. doi:10.1177/23197145211057343

2024 doi
[103]

Elsevier,Cambridge, MA, USA, 1977

JacobCohen.StatisticalPowerAnalysisForTheBehavioralSciences. Elsevier,Cambridge, MA, USA, 1977. ISBN 978-0-12-179060-8. doi:10.1016/C2013-0-10517-X

1977 doi
[104]

doi:10.1037/h0076540

MeriColemanandT.L.Liau.AComputerReadabilityFormulaDesignedForMachine Scoring.Journal of Applied Psychology, 60:283–284, 1975. doi:10.1037/h0076540

1975 doi
[105]

Johnson, and Myrle M

William Jay Conover, Mark E. Johnson, and Myrle M. Mohnson. A Com- parative Study Of Tests For Homogeneity Of Variances, With Applications To The Outer Continental Shelf Bidding Data.Technometrics, 23(4):351–361, 1981. doi:10.1080/00401706.1981.10487680

1981
[106]

A Comparative Study Of Tests For Homogeneity Of Variance

WilliamJayConover, ArmandoJesúsGuerrero-Serrano, andVíctorGustavoTercero- Gómez. An Update On "A Comparative Study Of Tests For Homogeneity Of Variance".Journal of Statistical Computation and Simulation, 88(8):1454–1469, 2018. doi:10.1080/00949655.2018.1438437

2018
[107]

CoordinatedInauthenticBehaviorandInformationSpreadingonTwitter.Deci- sionSupportSystems,160:113819,2022

Matteo Cinelli, Stefano Cresci, Walter Quattrociocchi, Maurizio Tesconi, and Paola Zola. CoordinatedInauthenticBehaviorandInformationSpreadingonTwitter.Deci- sionSupportSystems,160:113819,2022. ISSN0167-9236. doi:10.1016/j.dss.2022.113819

2022
[108]

Training Attention Improves Decision Making in Individuals With Elevated Self-reported Depressive Symptoms.Cognitive, Affective, & Behavioral Neuroscience, 14(2):729–741, 6 2014

Jessica A Cooper, Marissa A Gorlick, Taylor Denny, Darrell A Worthy, Christopher G Beevers, and W Todd Maddox. Training Attention Improves Decision Making in Individuals With Elevated Self-reported Depressive Symptoms.Cognitive, Affective, & Behavioral Neuroscience, 14(2):729–...

2014 doi
[109]

A Frame- work For Using Crowdsourcing In Government

Benjamin Y Clark, Nicholas Zingale, Joseph Logan, and Jeffrey Brudney. A Frame- work For Using Crowdsourcing In Government. InSocial Entrepreneurship: Concepts, Methodologies, Tools, and Applications, pages 405–425. 2019. doi:10.4018/978-1-5225- 8182-6.ch020. 367 of 455 BIBLIOGRAPHY

2019 doi
[110]

SelectingExtensionsinWeightedArgumentationFrameworks

SylvieCoste-Marquis, SébastienKonieczny, PierreMarquis, andMohandAkliOuali. SelectingExtensionsinWeightedArgumentationFrameworks. InBartVerheĳ,Stefan Szeider, and Stefan Woltran, editors,Proceedings of The 4th International Conference on Computational Models of Argument, volum...

2012 doi
[111]

Extracting Dialogical Explana- tionsforReviewAggregationswithArgumentativeDialogicalAgents

Oana Cocarascu, Antonio Rago, and Francesca Toni. Extracting Dialogical Explana- tionsforReviewAggregationswithArgumentativeDialogicalAgents. InProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AA- MAS ’19, page 1261–1269. Internatio...

2019
[112]

An Experimental Comparison of Click Position-Bias Models

Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. An Experimental Comparison of Click Position-Bias Models. InProceedings of the 2008 International Conference on Web Search and Data Mining, WSDM ’08, page 87–94. Association for Computing Machinery, 2008. ISBN 978159...

2008
[113]

Automation Bias in Intelligent Time Critical Decision Support Systems

Mary Cummings. Automation Bias in Intelligent Time Critical Decision Support Systems. InProceedings of the AIAA 1st Intelligent Systems Technical Conference, pages 1–6, 2004. doi:10.2514/6.2004-6313

2004 doi
[114]

Argumentative XAI: A Survey

KristĳonasCyras,AntonioRago,EmanueleAlbini,PietroBaroni,andFrancescaToni. Argumentative XAI: A Survey. In Zhi-Hua Zhou, editor,Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages 4392–4399. ĳcai.org, 2021. doi:10.24963/ĳcai.2021/600

2021
[115]

An Evaluation of the Belief in Science Scale.Frontiers in Psychology, 10, 2019

Neil Dagnall, Andrew Denovan, Kenneth Graham Drinkwater, and Andrew Parker. An Evaluation of the Belief in Science Scale.Frontiers in Psychology, 10, 2019. ISSN 1664-1078. doi:10.3389/fpsyg.2019.00861

2019
[116]

Automatic Deception Detection: Methods for Finding Fake News.Proceedings of the Association for Information Science and Technology, 52(1):1–4, 2015

Niall J Conroy, Victoria L Rubin, and Yimin Chen. Automatic Deception Detection: Methods for Finding Fake News.Proceedings of the Association for Information Science and Technology, 52(1):1–4, 2015. doi:10.1002/pra2.2015.145052010082

2015
[117]

Trippas, and Hamed Zamani

Jeffrey Dalton, Sophie Fischer, Paul Owoicho, Filip Radlinski, Federico Rossetto, Jo- hanne R. Trippas, and Hamed Zamani. Conversational Information Seeking: Theory and Application. InProceedings of the 45th International ACM SIGIR Conference on Re- searchandDevelopmentinInfor...

2022
[118]

Better than Rational: Evolutionary Psychology and theInvisibleHand.TheAmericanEconomicReview,84(2):327–332,1994.ISSN00028282

Leda Cosmides and John Tooby. Better than Rational: Evolutionary Psychology and theInvisibleHand.TheAmericanEconomicReview,84(2):327–332,1994.ISSN00028282. URLhttp://www.jstor.org/stable/2117853

1994
[119]

ISSN 1438-8871

Ting Dang, Jing Han, Tong Xia, Dimitris Spathis, Erika Bondareva, Chloë Siegele- Brown, Jagmohan Chauhan, Andreas Grammenos, Apinan Hasthanasombat, R An- dresFloto,PietroCicuta,andCeciliaMascolo.ExploringLongitudinalCough,Breath, and Voice Data for COVID-19 Progression Predict...

2022 doi
[120]

Weighted Attacks in Argumentation Frameworks

SylvieCoste-Marquis, SébastienKonieczny, PierreMarquis, andMohandAkliOuali. Weighted Attacks in Argumentation Frameworks. InProceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning, KR’12, page593–597.AAAIPress,2012. ISBN97...

2012
[121]

Das and Bing-Sheng Teng

T.K. Das and Bing-Sheng Teng. Cognitive Biases and Strategic Decision Processes: An Integrative Perspective.Journal of Management Studies, 36(6):757–778, 1999. doi:10.1111/1467-6486.00157

1999
[122]

Dayton, W

L. Dayton, W. Song, I. Kaloustian, E.L. Eschliman, J.C. Strickland, and C. Latkin. A Longitudinal Study of COVID-19 Disclosure Stigma and COVID-19 Testing Hes- itancy in the United States.Public Health, 212:14–21, 2022. ISSN 0033-3506. doi:10.1016/j.puhe.2022.08.003

2022 doi
[123]

Are we all in the same boat?

Esra Cemre Su de Groot and Ujwal Gadiraju. "Are we all in the same boat?" Cus- tomizable and Evolving Avatars to Improve Worker Engagement and Foster a Sense of Community in Online Crowd Work. InProceedings of the CHI Conference on Hu- man Factors in Computing Systems, CHI ’24...

2024
[124]

J. R. De Haan, R. Wehrens, S. Bauerschmidt, E. Piek, R. C. van Schaik, and L. M. C. Buydens. Interpretation of ANOVA Models For Microarray Data Using PCA.Bioin- formatics,23(2):184–190,112006. ISSN1367-4803. doi:10.1093/bioinformatics/btl572

doi
[125]

Edgar Dale and Jeanne S. Chall. A Formula for Predicting Readability.Educational Research Bulletin, 27(1):11–28, 1948. ISSN 15554023. URLhttps://www.jstor.org/ stable/1473169

1948
[126]

An Introduction to Hybrid Human-Machine Information Systems.Foundations and Trends In Web Science, 7(1):1–87, 12 2017

Gianluca Demartini, Djellel Eddine Difallah, Ujwal Gadiraju, and Michele Catasta. An Introduction to Hybrid Human-Machine Information Systems.Foundations and Trends In Web Science, 7(1):1–87, 12 2017. ISSN 1555-077X. doi:10.1561/1800000025

2017 doi
[127]

SwappingBricksForClicks: Crowdsourcing Longitudinal Data on Amazon Turk.Journal of Business Research, 68(12):2603–2609,

TimothyM.DalyandRajanNataraajan. SwappingBricksForClicks: Crowdsourcing Longitudinal Data on Amazon Turk.Journal of Business Research, 68(12):2603–2609,
[128]

doi:10.1016/j.jbusres.2015.05.001

2015 doi
[129]

Crowdsourcing a Crisis Response for COVID-19 in Oncology.Nature Cancer, 1(5):473–476, 4 2020

AakashDesai,JeremyWarner,NicoleKuderer,MikeThompson,CorriePainter,Gary Lyman, and Gilberto Lopes. Crowdsourcing a Crisis Response for COVID-19 in Oncology.Nature Cancer, 1(5):473–476, 4 2020. doi:10.1038/s43018-020-0065-z

2020 doi
[130]

The State of Human-centered NLP Technology for Fact-checking.Information Processing & Man- agement, 60(2):103219, 2023

Anubrata Das, Houjiang Liu, Venelin Kovatchev, and Matthew Lease. The State of Human-centered NLP Technology for Fact-checking.Information Processing & Man- agement, 60(2):103219, 2023. ISSN 0306-4573. doi:10.1016/j.ipm.2022.103219. 369 of 455 BIBLIOGRAPHY

2023
[131]

BERT: Pre- training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre- training of Deep Bidirectional Transformers for Language Understanding. InPro- ceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human L...

2019 doi
[132]

Diaz.Through the Google Goggles, pages 11–34

A. Diaz.Through the Google Goggles, pages 11–34. Springer Berlin Heidelberg, 2008. doi:10.1007/978-3-540-75829-7_2

2008 doi
[133]

Scaling-Up the Crowd: Micro-Task Pricing Schemes for Worker Retention and Latency Improvement

DjellelDifallah,MicheleCatasta,GianlucaDemartini,andPhilippeCudré-Mauroux. Scaling-Up the Crowd: Micro-Task Pricing Schemes for Worker Retention and Latency Improvement. InProceedings of the AAAI Conference on Human Com- putation and Crowdsourcing, volume 2, pages 50–58, Washi...

2014 doi
[135]

Stanley, and Walter Quattrociocchi

Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni, Antonio Scala, Guido Caldarelli, Eugene H. Stanley, and Walter Quattrociocchi. The Spreading of MisinformationOnline.ProceedingsoftheNationalAcademyofSciences,113(3):554–559,
[136]

ATask-BasedTaxonomyofCognitiveBiasesforInformationVisu- alization.IEEE Transactions on Visualization and Computer Graphics, 26(2):1413–1432,

Evanthia Dimara, Steven Franconeri, Catherine Plaisant, Anastasia Bezerianos, and PierreDragicevic. ATask-BasedTaxonomyofCognitiveBiasesforInformationVisu- alization.IEEE Transactions on Visualization and Computer Graphics, 26(2):1413–1432,
[137]

A New Markov Model For Web Ac- cess Prediction.Computing in Science & Engineering, 4(6):34–39, 2002

Xing Dongshan and Shen Junyi. A New Markov Model For Web Ac- cess Prediction.Computing in Science & Engineering, 4(6):34–39, 2002. doi:10.1109/MCISE.2002.1046594

2002 arXiv
[138]

Human-in-the-loopArti- ficialIntelligenceforFightingOnlineMisinformation: ChallengesandOpportunities

GianlucaDemartini,StefanoMizzaro,andDamianoSpina. Human-in-the-loopArti- ficialIntelligenceforFightingOnlineMisinformation: ChallengesandOpportunities. IEEE Data Engineering Bulletin, 43(3):65–74, 2020. URLhttp://sites.computer.org /debull/A20sept/p65.pdf

2020
[139]

Linked Credibility Reviews for Explainable Misinformation Detection

Ronald Denaux and Jose Manuel Gomez-Perez. Linked Credibility Reviews for Explainable Misinformation Detection. In Jeff Z. Pan, Valentina Tamma, Claudia d’Amato, Krzysztof Janowicz, Bo Fu, Axel Polleres, Oshani Seneviratne, and Lalana Kagal, editors,The Semantic Web – ISWC 202...

2020 doi
[140]

The Effects of Crowd Worker Biases in 371 of 455 BIBLIOGRAPHY Fact-Checking Tasks

Tim Draws, David La Barbera, Michael Soprano, Kevin Roitero, Davide Ceolin, Alessandro Checco, and Stefano Mizzaro. The Effects of Crowd Worker Biases in 371 of 455 BIBLIOGRAPHY Fact-Checking Tasks. In2022 ACM Conference on Fairness, Accountability, and Trans- parency, FAccT ’...

2022
[141]

Selective Markov Models for Predicting Web Page Accesses.ACM Transactions on Internet Technology, 4(2):163–184, 5 2004

Mukund Deshpande and George Karypis. Selective Markov Models for Predicting Web Page Accesses.ACM Transactions on Internet Technology, 4(2):163–184, 5 2004. ISSN 1533-5399. doi:10.1145/990301.990304. BIBLIOGRAPHY 370 of 455

2004
[142]

Optimism Bias During The COVID-19 Pandemic: Empirical Evidence From Romania And Italy.Games, 11(3),

Elena Druică, Fabio Musso, and Rodica Ianole-Călin. Optimism Bias During The COVID-19 Pandemic: Empirical Evidence From Romania And Italy.Games, 11(3),
[143]

Effects of Assessor Training on Information Use.Journal of Applied Psychology, 73(4):743–748, 1988

Beverly Dugan. Effects of Assessor Training on Information Use.Journal of Applied Psychology, 73(4):743–748, 1988. doi:10.1037/0021-9010.73.4.743

1988 doi
[144]

CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreement

Anca Dumitrache, Oana Inel, Lora Aroyo, Benjamin Timmermans, and Chris Welty. CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreement. In Lora Aroya, Anca Dumitrache, Praveen K. Paritosh, Alexander J. Quinn, Chris Welty, Alessandro Checco, Gianluca Demartini, Ujwal...

2018
[145]

On The Acceptability Of Arguments And Its Fundamental Role In Monmonotonic Reasoning, Logic Programming And n-person Games.Artificial Intelligence, 77(2):321–357, 1995

Phan Minh Dung. On The Acceptability Of Arguments And Its Fundamental Role In Monmonotonic Reasoning, Logic Programming And n-person Games.Artificial Intelligence, 77(2):321–357, 1995. ISSN 0004-3702. doi:10.1016/0004-3702(94)00041-X

1995 doi
[147]

Dunne, Anthony Hunter, Peter McBurney, Simon Parsons, and Michael Wooldridge

Paul E. Dunne, Anthony Hunter, Peter McBurney, Simon Parsons, and Michael Wooldridge. Weighted Argument Systems: Basic Definitions, Algorithms, And Complexity Results.Artificial Intelligence, 175(2):457–486, 2011. ISSN 0004-3702. doi:10.1016/j.artint.2010.09.005

2011 doi
[148]

doi:10.1109/TVCG.2018.2872577

2018
[149]

The Dunning–Kruger Effect: On Being Ignorant of One’s Own Ignorance.Advances in Experimental Social Psychology, 44:247–296, 2011

David Dunning. The Dunning–Kruger Effect: On Being Ignorant of One’s Own Ignorance.Advances in Experimental Social Psychology, 44:247–296, 2011. ISSN 0065-

2011
[150]

A Checklist to Combat Cognitive Biases in Crowdsourcing.Proceedings of the Ninth AAAI Conference on Human Computation and Crowdsourcing, 9(1):48–59, 10 2021

Tim Draws, Alisa Rieger, Oana Inel, Ujwal Gadiraju, and Nava Tintarev. A Checklist to Combat Cognitive Biases in Crowdsourcing.Proceedings of the Ninth AAAI Conference on Human Computation and Crowdsourcing, 9(1):48–59, 10 2021. doi:10.1609/hcomp.v9i1.18939

2021 doi
[151]

SIGIR ’21

Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Tim- mermans.This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics, page 295–305. SIGIR ’21. Association for Com- puting Machinery, 2021. ISBN 9...

2021
[152]

The Effects of Visualization on Judgment and Decision-making: A Systematic Literature Review.Management Review Quarterly, 73(1):167–214, 2 2023

Karin Eberhard. The Effects of Visualization on Judgment and Decision-making: A Systematic Literature Review.Management Review Quarterly, 73(1):167–214, 2 2023. ISSN 2198-1639. doi:10.1007/s11301-021-00235-8

2023 doi
[153]

The Principles of the Truth-O-Meter: Politi- Fact’s methodology for independent fact-checking, 2021

Angie Drobnic Holan. The Principles of the Truth-O-Meter: Politi- Fact’s methodology for independent fact-checking, 2021. URLhttps: //www.politifact.com/article/2018/feb/12/principles-truth-o-meter-p olitifacts-methodology-i/#Truth-O-Meter%20ratings. (Accessed: 20.04.2021)

2021
[154]

Improving Reactions to Rejection in Crowdsourcing Through Self-Reflection

Tom Edixhoven, Sihang Qiu, Lucie Kuiper, Olivier Dikken, Gwennan Smitskamp, and Ujwal Gadiraju. Improving Reactions to Rejection in Crowdsourcing Through Self-Reflection. InProceedings of the 13th ACM Web Science Conference 2021, WebSci ’21, page 74–83. Association for Computi...

2021
[155]

doi:10.3390/g11030039

ISSN 2073-4336. doi:10.3390/g11030039

doi
[156]

Cognitive Biases in Crowdsourcing

Carsten Eickhoff. Cognitive Biases in Crowdsourcing. InProceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM ’18, page 162–170. Association for Computing Machinery, 2018. ISBN 9781450355810. doi:10.1145/3159652.3159654

2018
[157]

de Vries

Carsten Eickhoff and Arjen P. de Vries. Increasing Cheat Robustness of Crowd- sourcing Tasks.Information Retrieval, 16(2):121–137, 4 2013. ISSN 1573-7659. doi:10.1007/s10791-011-9181-9

2013 doi
[158]

Confidence in Judgment: Persistence of The Illusion of Validity.Psychological Review, 85(5):395–416, 1978

Hillel J Einhorn and Robin M Hogarth. Confidence in Judgment: Persistence of The Illusion of Validity.Psychological Review, 85(5):395–416, 1978. doi:10.1037/0033- 295X.85.5.395

1978 doi
[159]

Multiple Comparisons Using Rank Sums.Technometrics, 6(3):241– 252, 1964

Olive Jean Dunn. Multiple Comparisons Using Rank Sums.Technometrics, 6(3):241– 252, 1964. doi:10.2307/1266041

1964 doi
[160]

Embretson

Susan E. Embretson. Item Response Theory Models and Spurious Interaction Effects in Factorial ANOVA Designs.Applied Psychological Measurement, 20(3):201–212, 1996. doi:10.1177/014662169602000302. 373 of 455 BIBLIOGRAPHY

1996 doi
[161]

Dunning, D

D. Dunning, D. W. Griffin, J. D. Milojkovic, and L. Ross. The Overconfidence Effect in Social Prediction.Journal of Personality and Social Psychology, 58(4):568–581, 4 1990. doi:10.1037/0022-3514.58.4.568

1990 doi
[162]

Association for Computing Machinery, New York, NY, USA, 2020

ZivEpstein,GordonPennycook,andDavidRand.WilltheCrowdGametheAlgorithm? Using Layperson Judgments to Combat Misinformation on Social Media by Downranking Distrusted Sources, page 1–11. Association for Computing Machinery, New York, NY, USA, 2020. ISBN 9781450367080. doi:10.1145/...

2020
[163]

GPOWER:AGeneralPowerAnalysis Program.Behavior Research Methods, Instruments, & Computers, 28(1):1–11, 3 1996

EdgarErdfelder,FranzFaul,andAxelBuchner. GPOWER:AGeneralPowerAnalysis Program.Behavior Research Methods, Instruments, & Computers, 28(1):1–11, 3 1996. doi:10.3758/BF03203630

1996 doi
[164]

The Nature of Crowd Work and its Effects on Individuals’ Work Perception.Journal of Management Information Systems, 37(1):66–95, 2020

David Durward, Ivo Blohm, and Jan Marco Leimeister. The Nature of Crowd Work and its Effects on Individuals’ Work Perception.Journal of Management Information Systems, 37(1):66–95, 2020. doi:10.1080/07421222.2019.1705506. BIBLIOGRAPHY 372 of 455

2020
[165]

Gregory Eady, Jonathan Nagler, Andy Guess, Jan Zilinsky, and Joshua A. Tucker. How Many People Live in Political Bubbles on Social Media? Evidence From Linked Survey and Twitter Data.SAGE Open, 9(1):2158244019832705, 2019. doi:10.1177/2158244019832705

2019 doi
[166]

Declinism: IsTheWorldActuallyGettingWorse.TheGuardian,15:1087– 1089,2015

PeteEtchells. Declinism: IsTheWorldActuallyGettingWorse.TheGuardian,15:1087– 1089,2015. URLhttps://www.theguardian.com/science/head-quarters/2015/ja n/16/declinism-is-the-world-actually-getting-worse

2015
[167]

ExplicitWarnings Reduce But Do Not Eliminate The Continued Influence of Misinformation.Memory & Cognition, 38(8):1087–1100, 12 2010

UllrichK.H.Ecker,StephanLewandowsky,andDavidT.W.Tang. ExplicitWarnings Reduce But Do Not Eliminate The Continued Influence of Misinformation.Memory & Cognition, 38(8):1087–1100, 12 2010. ISSN 1532-5946. doi:10.3758/MC.38.8.1087

2010 doi
[168]

FactCheck.org.OurProcess.https://www.factcheck.org/our-process/,2020.(Ac- cessed: 15–12–2021)

2020
[169]

Ehrlinger, W.O

J. Ehrlinger, W.O. Readinger, and B. Kim. Decision-Making and Cognitive Biases. In Howard S. Friedman, editor,Encyclopedia of Mental Health (Second Edition), pages 5–12. Academic Press, Oxford, second edition edition, 2016. ISBN 978-0-12-397753-3. doi:10.1016/B978-0-12-397045-...

2016 doi
[170]

CrowdCO-OP: Sharing Risks and Rewards in Crowdsourcing.Proceedings of ACM Human-Computer Interaction, 4(CSCW2), 10 2020

Shaoyang Fan, Ujwal Gadiraju, Alessandro Checco, and Gianluca Demartini. CrowdCO-OP: Sharing Risks and Rewards in Crowdsourcing.Proceedings of ACM Human-Computer Interaction, 4(CSCW2), 10 2020. doi:10.1145/3415203

2020 doi
[171]

Emergent: a Novel Data-set For Stance Clas- sification

William Ferreira and Andreas Vlachos. Emergent: a Novel Data-set For Stance Clas- sification. In Kevin Knight, Ani Nenkova, and Owen Rambow, editors,NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Lang...

2016 doi
[172]

A General Linear Mixed Models Approach to Study System Component Effects

Nicola Ferro and Gianmaria Silvello. A General Linear Mixed Models Approach to Study System Component Effects. InProceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’16, page 25–34. ACM, 2016. ISBN 978145034069...

2016
[173]

Overview Of The CLEF- 2019 CheckThat! Lab: Automatic Identification And Verification Of Claims

Tamer Elsayed, Preslav Nakov, Alberto Barrón-Cedeño, Maram Hasanain, Reem Suwaileh, Giovanni Da San Martino, and Pepa Atanasova. Overview Of The CLEF- 2019 CheckThat! Lab: Automatic Identification And Verification Of Claims. In Fabio Crestani,MartinBraschler,JacquesSavoy,Andre...

2019
[174]

Using Collection Shards to Study Retrieval Performance Effect Sizes.ACM Transactions on Information Systems, 37(3), 3

Nicola Ferro, Yubin Kim, and Mark Sanderson. Using Collection Shards to Study Retrieval Performance Effect Sizes.ACM Transactions on Information Systems, 37(3), 3
[175]

Barnum Effect

Catherine S. Fichten and Betty Sunerton. Popular Horoscopes and the “Barnum Effect”.The Journal of Psychology, 114(1):123–134, 1983. doi:10.1080/00223980.1983.9915405

1983
[176]

Robertson

Robert Epstein and Ronald E. Robertson. The Search Engine Manipulation Effect (SEME) And Its Possible Impact On The Outcomes Of Elections.Proceedings of the National Academy of Sciences, 112(33), 2015. doi:10.1073/pnas.1419828112

2015 doi
[177]

Seth Flaxman, Sharad Goel, and Justin M. Rao. Filter Bubbles, Echo Chambers, and Online News Consumption.Public Opinion Quarterly, 80(S1):298–320, 03 2016. ISSN 0033-362X. doi:10.1093/poq/nfw006

2016 doi
[178]

The Equivalence of Weighted Kappa and the Intr- aclass Correlation Coefficient as Measures of Reliability.Educational and Psychological Measurement, 33(3):613–619, 1973

Joseph L Fleiss and Jacob Cohen. The Equivalence of Weighted Kappa and the Intr- aclass Correlation Coefficient as Measures of Reliability.Educational and Psychological Measurement, 33(3):613–619, 1973. doi:10.1177/001316447303300309

1973 doi
[179]

Some Thoughts on a Framework for Crowdsourcing

Thomas Erickson. Some Thoughts on a Framework for Crowdsourcing. InCHI 2011 Workshop on Crowdsourcing and Human Computation, pages 1–4, 2011. URLhttps: //www.humancomputation.com/crowdcamp/chi2011/papers/erickson.pdf

2011
[180]

Who Falls For Fake News? Psychological And Clinical Pro- filing Evidence of Fake News Consumers.Personality and Individual Differences, 200,

Alex Escola-Gascon, Neil Dagnall, Andrew Denovan, Kenneth Drinkwater, and Miriam Diez-Bosch. Who Falls For Fake News? Psychological And Clinical Pro- filing Evidence of Fake News Consumers.Personality and Individual Differences, 200,
[181]

doi:10.1016/j.paid.2022.111893

ISSN 0191-8869. doi:10.1016/j.paid.2022.111893

2022
[182]

Some Common Mistakes In IR Evaluation, And How They Can Be Avoided.SIGIR Forum, 51(3):32–41, 2 2018

Norbert Fuhr. Some Common Mistakes In IR Evaluation, And How They Can Be Avoided.SIGIR Forum, 51(3):32–41, 2 2018. ISSN 0163-5840. doi:10.1145/3190580.3190586

2018
[183]

News Prove- nance: Revealing News Text Reuse at Web-Scale in an Augmented News Search Experience

Nathan Evans, Darren Edge, Jonathan Larson, and Christopher White. News Prove- nance: Revealing News Text Reuse at Web-Scale in an Augmented News Search Experience. InExtended Abstracts of the 2020 CHI Conference on Human Factors in Com- puting Systems, CHI EA ’20, page 1–8. A...

2020
[184]

A Literature Review of The Anchor- ing Effect.The Journal of Socio-Economics, 40(1):35–42, 2011

Adrian Furnham and Hua Chu Boo. A Literature Review of The Anchor- ing Effect.The Journal of Socio-Economics, 40(1):35–42, 2011. ISSN 1053-5357. doi:10.1016/j.socec.2010.10.008. 375 of 455 BIBLIOGRAPHY

2011 doi
[185]

ICrowd: An Adaptive Crowdsourcing Framework

Ju Fan, Guoliang Li, Beng Chin Ooi, Kian-lee Tan, and Jianhua Feng. ICrowd: An Adaptive Crowdsourcing Framework. InProceedings of the 2015 ACM SIGMOD Inter- national Conference on Management of Data, SIGMOD ’15, page 1015–1030. Association forComputingMachinery,2015. ISBN97814...

2015
[186]

Human Beyond the Machine: Challenges and Opportunities of Microtask Crowdsourcing

Ujwal Gadiraju, Gianluca Demartini, Ricardo Kawase, and Stefan Dietze. Human Beyond the Machine: Challenges and Opportunities of Microtask Crowdsourcing. IEEE Intelligent Systems, 30(4):81–85, 2015. doi:10.1109/MIS.2015.66

2015 doi
[187]

Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments

Ujwal Gadiraju, Alessandro Checco, Neha Gupta, and Gianluca Demartini. Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(3), 2017

2017
[188]

Crowdsourcing Versus the Laboratory: Towards Human-Centered Experiments Using the Crowd

Ujwal Gadiraju, Sebastian Möller, Martin Nöllenburg, Dietmar Saupe, Sebastian Egger-Lampl, Daniel Archambault, and Brian Fisher. Crowdsourcing Versus the Laboratory: Towards Human-Centered Experiments Using the Crowd. In Daniel Archambault, Helen Purchase, and Tobias Hoßfeld, ...

2017 doi
[189]

Toward an Anatomy of IR System Component Performances.Journal of the Association for Information Science and Technology, 69(2): 187–200, 2018

Nicola Ferro and Gianmaria Silvello. Toward an Anatomy of IR System Component Performances.Journal of the Association for Information Science and Technology, 69(2): 187–200, 2018. doi:10.1002/asi.23910

2018 doi
[190]

Snehalkumar(Neil)S.Gaikwad,DurimMorina,AdamGinzberg,CatherineMullings, Shirish Goyal, Dilrukshi Gamage, Christopher Diemert, Mathias Burton, Sharon Zhou, Mark Whiting, Karolina Ziulkoski, Alipta Ballav, Aaron Gilbee, Senad- hipathige S. Niranga, Vibhor Sehgal, Jasmine Lin, Leo...

2016
[191]

doi:10.1145/3310364

ISSN 1046-8188. doi:10.1145/3310364

doi
[192]

A Context-Aware Approach for Detecting Worth-Checking Claims in Po- litical Debates

Pepa Gencheva, Preslav Nakov, Lluís Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. A Context-Aware Approach for Detecting Worth-Checking Claims in Po- litical Debates. InProceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, ...

2017 doi
[193]

Fisher and Meir Statman

Kenneth L. Fisher and Meir Statman. Cognitive Biases in Market Fore- casts.The Journal of Portfolio Management, 27(1):72–81, 2000. ISSN 0095-4918. doi:10.3905/jpm.2000.319785
[194]

Ipeirotis

Anindya Ghose and Panagiotis G. Ipeirotis. Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteris- tics.IEEE Transactions on Knowledge and Data Engineering, 23(10):1498–1512, 2011. doi:10.1109/TKDE.2010.188

2011 doi
[195]

SemEval-2015 Task 11: Sentiment Analysis of Figurative LanguageinTwitter

AniruddhaGhosh,GuofuLi,TonyVeale,PaoloRosso,EkaterinaShutova,JohnBarn- den, and Antonio Reyes. SemEval-2015 Task 11: Sentiment Analysis of Figurative LanguageinTwitter. InProceedingsofthe9thInternationalWorkshoponSemanticEvalu- ation (SemEval 2015), pages 470–478, Denver, Colo...

2015 doi
[196]

A New Readability Yardstick.Journal of Applied Psychology, 32: 221–233, 1948

Rudolph Flesch. A New Readability Yardstick.Journal of Applied Psychology, 32: 221–233, 1948. doi:10.1037/h0057532

1948 doi
[197]

Follett and Thomas M

Katherine J. Follett and Thomas M. Hess. Aging, Cognitive Complexity, and the FundamentalAttributionError.TheJournalsofGerontology: SeriesB,57(4):P312–P323, 7 2002. ISSN 1079-5014. doi:10.1093/geronb/57.4.P312

2002 doi
[198]

Cognitive Reflection and Decision Making.Journal of Economic Perspectives, 19(4):25–42, 12 2005

Shane Frederick. Cognitive Reflection and Decision Making.Journal of Economic Perspectives, 19(4):25–42, 12 2005. doi:10.1257/089533005775196732

2005 doi
[199]

AutomationBias: ASystematic ReviewofFrequency,EffectMediators,andMitigators.JournaloftheAmericanMedical Informatics Association, 19(1):121–127, 06 2011

KateGoddard,AbdulRoudsari,andJeremyC.Wyatt. AutomationBias: ASystematic ReviewofFrequency,EffectMediators,andMitigators.JournaloftheAmericanMedical Informatics Association, 19(1):121–127, 06 2011. ISSN 1067-5027. doi:10.1136/amiajnl- 2011-000089

2011 doi
[200]

Cooperation in Crowd Work: Attitude and Perception of Freelancers on a Knowledge Work Platform

Zachary Fulker and Christoph Riedl. Cooperation in Crowd Work: Attitude and Perception of Freelancers on a Knowledge Work Platform. arXiv, 2023

2023
[201]

ISSN 2949-8759

S.R.Goodwin,M.J.Dwyer,S.L.Caliva,C.A.Burrows,andB.R.Raiff.UsingRedditasa RecruitmentStrategyforAddictionScienceResearch.JournalofSubstanceUseandAd- diction Treatment, 148:209011, 2023. ISSN 2949-8759. doi:10.1016/j.josat.2023.209011

2023
[202]

Understanding Worker Moods and Reac- tions to Rejection in Crowdsourcing

Ujwal Gadiraju and Gianluca Demartini. Understanding Worker Moods and Reac- tions to Rejection in Crowdsourcing. InProceedings of the 30th ACM Conference on HypertextandSocialMedia,HT’19,page211–220.AssociationforComputingMachin- ery, 2019. ISBN 9781450368858. doi:10.1145/3342...

2019
[203]

Annetta Grant, Henri Weĳo, and Peter A. Dacin. How Institutional Logics Shape Fairness in Crowdsourcing: The Case of Threadless.International Journal of Research inMarketing,40(2):378–397,2023. ISSN0167-8116. doi:10.1016/j.ĳresmar.2022.10.002

2023 doi
[204]

Understanding the Promise and Limits of Automated Fact-Checking,

Lucas Graves. Understanding the Promise and Limits of Automated Fact-Checking,
[206]

Clarity is a Worthwhile Quality: On the Role of Task Clarity in Microtask Crowdsourcing

Ujwal Gadiraju, Jie Yang, and Alessandro Bozzon. Clarity is a Worthwhile Quality: On the Role of Task Clarity in Microtask Crowdsourcing. InProceedings of the 28th ACM Conference on Hypertext and Social Media, HT ’17, page 5–14. Association for Computing Machinery, 2017. ISBN ...

2017
[207]

Validating a Scale For Citizen Trust In Government Organizations.International Review of Administrative Sciences, 83(3):583– 601, 2017

Stephan Grimmelikhuĳsen and Eva Knies. Validating a Scale For Citizen Trust In Government Organizations.International Review of Administrative Sciences, 83(3):583– 601, 2017. doi:10.1177/0020852315585950

2017 doi
[208]

Assessing the risks of ‘infodemics’ in response to COVID-19 epi- demics, 12 2020

Riccardo Gallotti, Francesco Valle, Nicola Castaldo, Pierluigi Sacco, and Manlio De Domenico. Assessing the risks of ‘infodemics’ in response to COVID-19 epi- demics, 12 2020. ISSN 2397-3374

2020
[209]

InProceedingsofthe34thInternationalConferenceonMachineLearning -Volume70,ICML’17,page1321–1330.JMLR.org,2017

ChuanGuo,GeoffPleiss,YuSun,andKilianQ.Weinberger.OnCalibrationofModern NeuralNetworks. InProceedingsofthe34thInternationalConferenceonMachineLearning -Volume70,ICML’17,page1321–1330.JMLR.org,2017. doi:10.5555/3305381.3305518

2017
[210]

Catching Zika Fever: Application of Crowd- sourcing and Machine Learning for Tracking Health Misinformation on Twitter

Amira Ghenai and Yelena Mejova. Catching Zika Fever: Application of Crowd- sourcing and Machine Learning for Tracking Health Misinformation on Twitter. In2017 IEEE International Conference on Healthcare Informatics, pages 518–518, 2017. doi:10.1109/ICHI.2017.58

2017 doi
[211]

Has the Standard Cognitive Reflection Test Become a Victim of Its Own Success?Advances in Cognitive Psychology, 12(3):145, 2016

Matthew Haigh. Has the Standard Cognitive Reflection Test Become a Victim of Its Own Success?Advances in Cognitive Psychology, 12(3):145, 2016. doi:10.5709/acp- 0193-5

2016 doi
[212]

Hamilton and Robert K

David L. Hamilton and Robert K. Gifford. Illusory Correlation In Interpersonal Perception: ACognitiveBasisofStereotypicJudgments.JournalofExperimentalSocial Psychology, 12(4):392–407, 1976. ISSN 0022-1031. doi:10.1016/S0022-1031(76)80006-6

1976 doi
[213]

Association for Computing Machinery, 2020

Anastasia Giachanou and Paolo Rosso.The Battle Against Online Harmful Information: The Cases of Fake News and Hate Speech, pages 3503–3504. Association for Computing Machinery, 2020. ISBN 9781450368599. doi:10.1145/3340531.3412169

2020
[214]

BoundedandRational

GerdGigerenzerandR.Selten. BoundedandRational. InPhilosophie: Grundlagenund Anwendungen/Philosophy: Foundations and Applications, pages 233–257. Brill | Mentis,
[215]

On Transforming Relevance Scales

LeiHan,KevinRoitero,EddyMaddalena,StefanoMizzaro,andGianlucaDemartini. On Transforming Relevance Scales. InProceedings of the 28th ACM International Con- ference on Information and Knowledge Management, CIKM ’19, page 39–48. Association forComputingMachinery,2019. ISBN97814503...

2019
[216]

Thomas Gillier, Cédric Chaffois, Mustapha Belkhouja, Yannig Roth, and Barry L. Bayus. The Effects of Task Instructions in Crowdsourcing Innovative Ideas. Technological Forecasting and Social Change, 134:35–44, 2018. ISSN 0040-1625. doi:10.1016/j.techfore.2018.05.005

2018 doi
[217]

TheImpactofTaskAbandonmentinCrowdsourcing.IEEE Transactions on Knowledge & Data Engineering, 1(1):1–1, 10 2019

TomL.Han,KevinRoitero,UjwalGadiraju,C.Sarasua,A.Checco,EddyMaddalena, andGianlucaDemartini. TheImpactofTaskAbandonmentinCrowdsourcing.IEEE Transactions on Knowledge & Data Engineering, 1(1):1–1, 10 2019. ISSN 1558-2191. doi:10.1109/TKDE.2019.2948168

2019
[218]

Identifying effective cognitive biases in information retrieval.Journal of Information Science, 49(2):348–358, 2023

Gisoo Gomroki, Hassan Behzadi, Rahmatolloah Fattahi, and Javad Salehi Fadardi. Identifying effective cognitive biases in information retrieval.Journal of Information Science, 49(2):348–358, 2023. doi:10.1177/01655515211001777

2023 doi
[219]

NeuralCheck-WorthinessRankingwithWeakSupervision: Finding Sentences for Fact-Checking

Casper Hansen, Christian Hansen, Stephen Alstrup, Jakob Grue Simonsen, and ChristinaLioma. NeuralCheck-WorthinessRankingwithWeakSupervision: Finding Sentences for Fact-Checking. InCompanion Proceedings of The 2019 World Wide Web Conference, WWW ’19, page 994–1000. Association ...

2019
[220]

Accurate Evaluation of Segment-level Machine Translation Metrics

Yvette Graham, Timothy Baldwin, and Nitika Mathur. Accurate Evaluation of Segment-level Machine Translation Metrics. InProceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1183–119...

2015 doi
[221]

The Fundamental Attribution Error

John H. Harvey, Jerri P. Town, and Kerry L. Yarkin. How Fundamental Is "The Fundamental Attribution Error"?Journal of Personality and Social Psychology, 40(2): 346–349, 1981. doi:10.1037/0022-3514.40.2.346

1981 doi
[222]

Haselton and Daniel Nettle

Martie G. Haselton and Daniel Nettle. The Paranoid Optimist: An Integrative Evo- lutionary Model of Cognitive Biases.Personality and Social Psychology Review, 10(1): 47–66, 2006. doi:10.1207/s15327957pspr1001_3. 379 of 455 BIBLIOGRAPHY

2006 doi
[223]

URLhttps://reutersinstitute.politics.ox.ac.uk/our-research/unde rstanding-promise-and-limits-automated-fact-checking
[224]

TheQuestToAutomateFact-Checking

Naeemul Hassan, Bill Adair, James T Hamilton, Chengkai Li, Mark Tremayne, Jun Yang,andCongYu. TheQuestToAutomateFact-Checking. InProceedingsofthe2015 Computation + Journalism Symposium, 10 2015. URLhttp://cj2015.brown.columbia .edu/papers/automate-fact-checking.pdf

2015
[225]

InProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, page 1809–1818

MihajloGrbovic,VladanRadosavljevic,NemanjaDjuric,NarayanBhamidipati,Jaikit Savla,VarunBhagwan,andDougSharp.E-CommerceinYourInbox: ProductRecom- mendations at Scale. InProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, p...

2015
[226]

EffectsoftheUseoftheAvailabilityHeuristic on Ethical Decision-Making in Organizations.Journal of Business Ethics, 84:151–165,

SefaHayiborandDavidM.Wasieleski. EffectsoftheUseoftheAvailabilityHeuristic on Ethical Decision-Making in Organizations.Journal of Business Ethics, 84:151–165,
[227]

Psychology Press, 2016

David Groome and Michael Eysenck.An Introduction to Applied Cognitive Psychology. Psychology Press, 2016. ISBN 9781138840133

2016
[228]

McKinney, and Matthew Lease

Danula Hettiachchi, Mike Schaekermann, Tristan J. McKinney, and Matthew Lease. The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Help.Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), 10 2021. doi:10.1145/3476073

2021 doi
[229]

Lee, Adam C

Ashish Gurung, Sami Baral, Morgan P. Lee, Adam C. Sales, Aaron Haim, Kirk P. Vanacore, Andrew A. McReynolds, Hilary Kreisberg, Cristina Heffernan, and Neil T. Heffernan. How Common Are Common Wrong Answers? Crowdsourcing Reme- diation at Scale. InProceedings of the Tenth ACM C...

2023
[230]

TowardaSynthesisofCognitiveBiases: HowNoisyInformationProcess- ing Can Bias Human Decision Making.Psychology Bullettin, 138(2):211–237, 3 2012

M.Hilbert. TowardaSynthesisofCognitiveBiases: HowNoisyInformationProcess- ing Can Bias Human Decision Making.Psychology Bullettin, 138(2):211–237, 3 2012. doi:10.1037/a0025940

2012 doi
[231]

Martin Hinton and Jean H. M. Wagemans. Evaluating Reasoning in Natural Argu- ments: A Procedural Approach.Argumentation, 36(1):61–84, 3 2022. ISSN 1572-8374. doi:10.1007/s10503-021-09555-1

2022 doi
[232]

A Spatial MobileCrowdsourcingFrameworkforEventReporting.IEEETransactionsonCompu- tational Social Systems, 7(2):477–491, 2020

Aymen Hamrouni, Hakim Ghazzai, Mounir Frikha, and Yehia Massoud. A Spatial MobileCrowdsourcingFrameworkforEventReporting.IEEETransactionsonCompu- tational Social Systems, 7(2):477–491, 2020. doi:10.1109/TCSS.2020.2967585. BIBLIOGRAPHY 378 of 455

2020
[234]

Harry L. Hom. Perspective-taking and Hindsight Bias: When The Target Is Oneself And/or A Peer.Current Psychology, 1 2022. ISSN 1936-4733. doi:10.1007/s12144-021- 02413-z

2022 doi
[235]

Modelling User Behavior Dynamics with Embeddings

Lei Han, Alessandro Checco, Djellel Difallah, Gianluca Demartini, and Shazia Sadiq. Modelling User Behavior Dynamics with Embeddings. InProceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 445–454. Association for Computing...

2020
[236]

The Rise of Crowdsourcing.Wired Magazine, 14(6):1–4, 2006

Jeff Howe. The Rise of Crowdsourcing.Wired Magazine, 14(6):1–4, 2006. URLhttps: //www.wired.com/2006/06/crowds/

2006
[237]

AndreasHanselowski,HaoZhang,ZileLi,DaniilSorokin,BenjaminSchiller,Claudia Schulz, and Iryna Gurevych. UKP-Athene: Multi-Sentence Textual Entailment for ClaimVerification.InProceedingsoftheFirstWorkshoponFactExtractionandVERification (FEVER), pages 103–108, Brussels, Belgium, 1...

2018 doi
[238]

Crowd- sourcing for Sustainable Urban Logistics: Exploring the Factors Influencing Crowd Workers’ Participative Behavior.Sustainability, 12(8):1–20, 4 2020

Lĳuan Huang, Guojie Xie, John Blenkinsopp, Raoyi Huang, and Hou Bin. Crowd- sourcing for Sustainable Urban Logistics: Exploring the Factors Influencing Crowd Workers’ Participative Behavior.Sustainability, 12(8):1–20, 4 2020. ISSN 2071-1050. doi:10.3390/su12083091

2020 doi
[239]

Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P. Bigham. A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk. InProceedings of the 2018 CHI Conference on Human Factors in Com- putingSystems,CHI’18,page1–14.Associa...

2018
[240]

Understanding and Mitigat- ing Worker Biases in the Crowdsourced Collection of Subjective Judgments

Christoph Hube, Besnik Fetahu, and Ujwal Gadiraju. Understanding and Mitigat- ing Worker Biases in the Crowdsourced Collection of Subjective Judgments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, page 1–12. Association for Computin...

2019
[241]

ISO/IEC 25012:2008 Software Engi- neering—SoftwareProductQualityRequirementsandEvaluation(SQuaRE)—Data Quality M<Sodel

International Organization for Standardization. ISO/IEC 25012:2008 Software Engi- neering—SoftwareProductQualityRequirementsandEvaluation(SQuaRE)—Data Quality M<Sodel. Technical report, ISO, 2008. URLhttps://www.iso.org/standa rd/35736.html

2008
[242]

Haselton, Daniel Nettle, and Damian R

Martie G. Haselton, Daniel Nettle, and Damian R. Murray.The Evolution of Cognitive Bias, chapter 41, pages 1–20. John Wiley & Sons, Ltd, 2015. ISBN 9781119125563. doi:10.1002/9781119125563.evpsych241

2015 doi
[243]

MohsenJavdaniandHa-JoonChang.WhoSaidOrWhatSaid? EstimatingIdeological Bias In Views Among Economists.Cambridge Journal of Economics, 47(2):309–339, 4
[244]

Bernstein

Kenji Hata, Ranjay Krishna, Li Fei-Fei, and Michael S. Bernstein. A Glimpse Far into the Future: Understanding Long-Term Crowd Worker Quality. InProceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Com- puting, CSCW ’17, page 889–901. Associ...

2017
[245]

Comparing In Situ and Multidimensional RelevanceJudgments

Jiepu Jiang, Daqing He, and James Allan. Comparing In Situ and Multidimensional RelevanceJudgments. InProceedingsofthe40thInternationalACMSIGIRConferenceon ResearchandDevelopmentinInformationRetrieval,SIGIR’17,page405–414.Association forComputingMachinery,2017. ISBN97814503502...

2017
[246]

Accu- rately Interpreting Clickthrough Data as Implicit Feedback

ThorstenJoachims,LauraGranka,BingPan,HeleneHembrooke,andGeriGay. Accu- rately Interpreting Clickthrough Data as Implicit Feedback. InProceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval,SIGIR’05,page154–161.Asso...

2005
[247]

Madeline E. Heilman. Gender Stereotypes And Workplace Bias.Research in Organi- zational Behavior, 32:113–135, 2012. ISSN 0191-3085. doi:10.1016/j.riob.2012.11.003

2012 doi
[248]

Jones.The Courtesy Bias in South-East Asian Surveys, pages 253–9

Emily L. Jones.The Courtesy Bias in South-East Asian Surveys, pages 253–9. 1993. URL https://unesdoc.unesco.org/ark:/48223/pf0000016829

1993
[249]

A Survey on Task As- signment in Crowdsourcing.ACM Computing Surveys, 55(3), 2 2022

Danula Hettiachchi, Vassilis Kostakos, and Jorge Goncalves. A Survey on Task As- signment in Crowdsourcing.ACM Computing Surveys, 55(3), 2 2022. ISSN 0360-0300. doi:10.1145/3494522

2022 doi
[250]

A Hybrid High-Order Markov Chain Model for Computer Intrusion Detection.Journal of Computational and Graphical Statistics, 10(2): 277–295, 2001

Wen-Hua Ju and Yehuda Vardi. A Hybrid High-Order Markov Chain Model for Computer Intrusion Detection.Journal of Computational and Graphical Statistics, 10(2): 277–295, 2001. ISSN 10618600. URLhttp://www.jstor.org/stable/1391012

2001
[251]

Great Chain of Agents: TheRoleofMetaphoricalRepresentationofAgentsinConversationalCrowd- sourcing

Ji-Youn Jung, Sihang Qiu, Alessandro Bozzon, and Ujwal Gadiraju. Great Chain of Agents: TheRoleofMetaphoricalRepresentationofAgentsinConversationalCrowd- sourcing. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22. Association for Computin...

2022
[252]

(Accessed: 20–06–2023)

AngieDrobnicHolan.ThePrinciplesofTheTruth-O-Meter: PolitiFact’sMethodology ForIndependentFact-checking.https://www.politifact.com/article/2018/feb/ 12/principles-truth-o-meter-politifacts-methodology-i/, 2018. (Accessed: 20–06–2023)

2018
[253]

Assessing The Reliability of The M5-120 on Amazon’s Mechanical Turk.Computers in Human Behavior, 29(4): 1749–1754, 2013

Christopher J Holden, Trevor Dennie, and Adam D Hicks. Assessing The Reliability of The M5-120 on Amazon’s Mechanical Turk.Computers in Human Behavior, 29(4): 1749–1754, 2013. ISSN 0747-5632. doi:10.1016/j.chb.2013.02.020. BIBLIOGRAPHY 380 of 455

2013 doi
[254]

Farrar,StrausandGiroux,NewYork,2011

DanielKahneman.Thinking,FastandSlow. Farrar,StrausandGiroux,NewYork,2011. ISBN 9780374275631. doi:10.1007/978-3-642-36121-8

2011 doi
[255]

This Just In: Fake News Packs A Lot In Title, Uses Simpler, Repetitive Content in Text Body, More Similar To Satire Than Real News

Benjamin Horne and Sibel Adali. This Just In: Fake News Packs A Lot In Title, Uses Simpler, Repetitive Content in Text Body, More Similar To Satire Than Real News. Proceedings of the International AAAI Conference on Web and Social Media, 11(1), 5 2017. URLhttps://ojs.aaai.org/...

2017
[256]

OnThePsychologyofPrediction.Psychological Review, 80(4):237–251, 1973

DanielKahnemanandAmosTversky. OnThePsychologyofPrediction.Psychological Review, 80(4):237–251, 1973. doi:10.1037/h0034747. BIBLIOGRAPHY 382 of 455

1973 doi
[257]

Hsiu-Fang Hsieh and Sarah E. Shannon. Three Approaches to Qualita- tive Content Analysis.Qualitative Health Research, 15(9):1277–1288, 2005. doi:10.1177/1049732305276687

2005 doi
[258]

Can You Verifi This? Studying Uncertainty and Decision-Making About Misinformation Using Visual Analytics

Alireza Karduni, Ryan Wesslen, Sashank Santhanam, Isaac Cho, Svitlana Volkova, Dustin Arendt, Samira Shaikh, and Wenwen Dou. Can You Verifi This? Studying Uncertainty and Decision-Making About Misinformation Using Visual Analytics. In Proceedings of the International AAAI Conf...
[259]

A Tutorial On Calibration Measurements And Calibration Models For Clinical Prediction Models.Journal of the American Medical Informatics Association, 27 (4):621–633, 02 2020

Yingxiang Huang, Wentao Li, Fima Macheret, Rodney A Gabriel, and Lucila Ohno- Machado. A Tutorial On Calibration Measurements And Calibration Models For Clinical Prediction Models.Journal of the American Medical Informatics Association, 27 (4):621–633, 02 2020. ISSN 1527-974X....

2020 doi
[260]

Artifact,Bias,AndComplexityofAssessment: TheABCsofReliabil- ity.JournalofAppliedBehaviorAnalysis,10(1):141–150,1977.doi:10.1901/jaba.19P77.10- 141

AlanEKazdin. Artifact,Bias,AndComplexityofAssessment: TheABCsofReliabil- ity.JournalofAppliedBehaviorAnalysis,10(1):141–150,1977.doi:10.1901/jaba.19P77.10- 141

1977 doi
[261]

SystemsPerspectiveofAmazonMe- chanical Turk for Organizational Research: Review and Recommendations.Frontiers in Psychology, 8:1359, 2017

MelissaG.Keith,LouisTay,andPeterD.Harms. SystemsPerspectiveofAmazonMe- chanical Turk for Organizational Research: Review and Recommendations.Frontiers in Psychology, 8:1359, 2017. doi:10.3389/fpsyg.2017.01359

2017
[262]

Irani and M

Lilly C. Irani and M. Six Silberman. Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, page 611–620. Association for Computing Machinery,
[263]

Waggoner, Ryan Jewell, and Nicholas J

Ryan Kennedy, Scott Clifford, Tyler Burleigh, Philip D. Waggoner, Ryan Jewell, and Nicholas J. G. Winter. The Shape of And Solutions To The MTurk Quality Crisis. Political Science Research and Methods, 8(4):614–629, 2020. doi:10.1017/psrm.2020.6

2020 doi
[264]

TheMeant, theSaid,andtheUnderstood: ConversationalArgumentSearchandCognitiveBiases

JohannesKiesel,DamianoSpina,HenningWachsmuth,andBennoStein. TheMeant, theSaid,andtheUnderstood: ConversationalArgumentSearchandCognitiveBiases. InProceedingsofthe3rdConferenceonConversationalUserInterfaces,CUI’21.Association forComputingMachinery,2021. ISBN9781450389983. doi:1...

2021
[265]

doi:10.1093/cje/beac071

ISSN 0309-166X. doi:10.1093/cje/beac071

doi
[266]

Issue Framing and Engagement: Rhetorical Strategy in Public Policy Debates.Political Behavior, 30(1):1–24, 2008

Jennifer Jerit. Issue Framing and Engagement: Rhetorical Strategy in Public Policy Debates.Political Behavior, 30(1):1–24, 2008. ISSN 01909320, 15736687. URLhttp: //www.jstor.org/stable/40213302. 381 of 455 BIBLIOGRAPHY

2008
[267]

Framing Effects in Younger And Older Adults.The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 60(4):P215–P218, 2005

Sunghan Kim, David Goldstein, Lynn Hasher, and Rose T Zacks. Framing Effects in Younger And Older Adults.The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 60(4):P215–P218, 2005. doi:10.1093/geronb/60.4.p215

2005 doi
[268]

Deriva- tion Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel

JPeterKincaid,RobertPFishburneJr,RichardLRogers,andBradSChissom. Deriva- tion Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Research branch report 8-75, Chief of Naval Technical Training: Naval...

1975
[269]

The Evolution of Error: Error Management, Cognitive Constraints, And Adaptive Decision-making Biases.Trends in Ecology & Evolution, 28(8):474–481, 2013

DominicD.P.Johnson,DanielT.Blumstein,JamesH.Fowler,andMartieG.Haselton. The Evolution of Error: Error Management, Cognitive Constraints, And Adaptive Decision-making Biases.Trends in Ecology & Evolution, 28(8):474–481, 2013. ISSN 0169-5347. doi:10.1016/j.tree.2013.05.014

2013 doi
[270]

Chi, and Bongwon Suh

Aniket Kittur, Ed H. Chi, and Bongwon Suh. Crowdsourcing User Studies With Mechanical Turk. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, pages 453–456. ACM, 2008. ISBN 9781605580111. doi:10.1145/1357054.1357127

2008
[271]

Sage Publications,

Garth S Jowett and Victoria O’Donnell.Propaganda & Persuasion. Sage Publications,
[272]

URLhttps://us.sagepub.com/en-us/nam/propaganda -persuasion/book250869

ISBN 9781506371344. URLhttps://us.sagepub.com/en-us/nam/propaganda -persuasion/book250869
[273]

Evalu- ating content quality and helpfulness of online product reviews: The interplay of review helpfulness vs

Nikolaos Korfiatis, Elena García-Bariocanal, and Salvador Sánchez-Alonso. Evalu- ating content quality and helpfulness of online product reviews: The interplay of review helpfulness vs. review content.Electronic Commerce Research and Applications, 11(3):205–217, 2012. ISSN 156...

2012 doi
[274]

Integrating On-demand Fact-checking with Public Dialogue

TravisKriplean,CaitlinBonnar,AlanBorning,BoKinney,andBrianGill. Integrating On-demand Fact-checking with Public Dialogue. InProceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW ’14, page 1188–1199. Association for Computing Mac...

2014
[275]

Choice-supportive Bias

Mahdi Kafaee, Hanie Marhamati, and Shahriar Gharibzadeh. “Choice-supportive Bias” In Science: Explanation and Mitigation.Accountability in Research, 28(8):528– 543, 2021. doi:10.1080/08989621.2021.1872377

2021
[276]

InformationQualityBench- marks: Product and Service Performance.Communications of the ACM, 45(4):184–192, 4 2002

BeverlyK.Kahn,DianeM.Strong,andRichardY.Wang. InformationQualityBench- marks: Product and Service Performance.Communications of the ACM, 45(4):184–192, 4 2002. ISSN 0001-0782. doi:10.1145/505248.506007

2002
[278]

Representativeness Revisited: Attribute Substitution in Intuitive Judgment.Heuristics and biases: The psychology of intuitive judgment, pages 49–81, 2002

Daniel Kahneman and Shane Frederick. Representativeness Revisited: Attribute Substitution in Intuitive Judgment.Heuristics and biases: The psychology of intuitive judgment, pages 49–81, 2002. doi:10.1017/CBO9780511808098.004

2002 doi
[280]

National Institute of Informatics (NII)

Noriko Kando, editor.Proceedings of the 7th NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, Question Answering and Clross- Lingual Information Access, NTCIR-7, 2008. National Institute of Informatics (NII). ISBN 978-4-86049-044-7...

2008
[282]

doi:10.1609/icwsm.v12i1.15014

doi
[283]

TheOstrichEffect: Selective Attention To Information.Journal of Risk and Uncertainty, 38(2):95–115, 4 2009

NiklasKarlsson,GeorgeLoewenstein,andDuaneSeppi. TheOstrichEffect: Selective Attention To Information.Journal of Risk and Uncertainty, 38(2):95–115, 4 2009. ISSN 1573-0476. doi:10.1007/s11166-009-9060-6

2009 doi
[286]

How Many Results per Page? A Study of SERP Size, Search Behavior and User Experience

Diane Kelly and Leif Azzopardi. How Many Results per Page? A Study of SERP Size, Search Behavior and User Experience. InProceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, page 183–192. Association for Com...

2015
[289]

Homogeneity-Based Transmissive Process to Model True and False News in Social Networks

Jooyeon Kim, Dongkwan Kim, and Alice Oh. Homogeneity-Based Transmissive Process to Model True and False News in Social Networks. InProceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM ’19, page 348–356. Association for Computing Machinery...

2019
[290]

Semantic Sentence Matching with Densely-Connected Recurrent and Co-Attentive Information

Seonhoon Kim, Inho Kang, and Nojun Kwak. Semantic Sentence Matching with Densely-Connected Recurrent and Co-Attentive Information. InProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Ap- plications of Artificial Intelligence...

2019 doi
[293]

Identifying The Bandwagon Effect In Two-round Elections.Public Choice, 160(3):327–344, 9 2014

Áron Kiss and Gábor Simonovits. Identifying The Bandwagon Effect In Two-round Elections.Public Choice, 160(3):327–344, 9 2014. ISSN 1573-7101. doi:10.1007/s11127- 013-0146-y

2014 doi
[295]

Aniket Kittur, Boris Smus, Susheel Khamkar, and Robert E. Kraut. CrowdForge: Crowdsourcing Complex Work. InProceedings of the 24th Annual ACM Symposium on UserInterfaceSoftwareandTechnology,UIST’11,page43–52.AssociationforComputing Machinery, 2011. ISBN 9781450307161. doi:10.1...

2011
[296]

Kleinbaum, K

David G. Kleinbaum, K. Dietz, M. Gail, and M. Klein.Logistic Regression. Springer,
[300]

Computing Krippendorff’s Alpha-Reliability.UPENN Libraries, 1:43, 2008

Klaus Krippendorff. Computing Krippendorff’s Alpha-Reliability.UPENN Libraries, 1:43, 2008. URLhttps://repository.upenn.edu/asc_papers/43

2008
[1998]

doi:10.1016/S0306-4573(97)00078-2

ISSN 0306-4573. doi:10.1016/S0306-4573(97)00078-2

doi
[2002]

doi:10.1007/978-1-4419-1742-3

doi
[2008]

doi:10.30965/9783969750056_016

ISBN 9783969750056. doi:10.30965/9783969750056_016

doi
[2009]

URLhttp://www.jstor.org/stable/40294779

ISSN 01674544, 15730697. URLhttp://www.jstor.org/stable/40294779
[2013]

doi:10.1145/2470654.2470742

ISBN 9781450318990. doi:10.1145/2470654.2470742
[2015]

doi:10.1109/ITNG.2015.78

2015 doi
[2016]

doi:10.1073/pnas.1517441113

doi
[2018]

doi:10.1145/3209581

ISSN 0001-0782. doi:10.1145/3209581

doi
[2019]

ISBN 978-3-030-28577-7

Springer International Publishing. ISBN 978-3-030-28577-7. doi:10.1007/978-3- 030-28577-7_25

doi
[2020]

doi:10.1613/jair.1.12162

doi
[2023]

doi:10.1093/bjs/znad004

ISSN 1365-2168. doi:10.1093/bjs/znad004

doi
[2601]

doi:10.1016/B978-0-12-385522-0.00005-6

doi
[9236]

doi:10.1016/j.dss.2010.04.002

2010 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

doi:10.1007/978-3-642-40802-1_31

doi:10.1007/978-3-642-40802-1_31

[2] [2]

Goal-Setting Behavior of Workers on Crowd- sourcing Platforms: An Exploratory Study on MTurk and Prolific

Tahir Abbas and Ujwal Gadiraju. Goal-Setting Behavior of Workers on Crowd- sourcing Platforms: An Exploratory Study on MTurk and Prolific. In Jane Hsu and Ming Yin, editors,Proceedings of the Tenth AAAI Conference on Human Computa- tion and Crowdsourcing, volume 10, pages 2–13, Washington, DC, 2022. AAAI Press. doi:10.1609/hcomp.v10i1.21983

doi:10.1609/hcomp.v10i1.21983 2022

[3] [3]

Tukey’s Honestly Significant Difference (HSD) Test.EncyclopediaofResearchDesign,3(1):1–5,2010

Hervé Abdi and Lynne J Williams. Tukey’s Honestly Significant Difference (HSD) Test.EncyclopediaofResearchDesign,3(1):1–5,2010. URLhttps://personal.utdalla s.edu/~Herve/abdi-HSD2010-pretty.pdf

2010

[4] [4]

Cognitive Attraction and Online Misinformation.Palgrave Communi- cations, 5(1), 2 2019

Alberto Acerbi. Cognitive Attraction and Online Misinformation.Palgrave Communi- cations, 5(1), 2 2019. ISSN 2055-1045. doi:10.1057/s41599-019-0224-y

doi:10.1057/s41599-019-0224-y 2019

[5] [5]

Wiley Series in Probability and Statistics.Wiley,2ndedition,2010

Alan Agresti.Analysis of Ordinal Categorical Data. Wiley Series in Probability and Statistics.Wiley,2ndedition,2010. ISBN9780470082898. doi:10.1002/9780470594001

doi:10.1002/9780470594001 2010

[6] [6]

Explainable Fact Checking with Probabilistic Answer Set Programming

Naser Ahmadi, Joohyung Lee, Paolo Papotti, and Mohammed Saeed. Explainable Fact Checking with Probabilistic Answer Set Programming. In Maria Liakata and Andreas Vlachos, editors,Proceedings of the 2019 Truth and Trust Online Conference (TTO 2019), 2019. URLhttps://truthandtrustonline.com/wp-content/uploads /2019/09/paper_15.pdf

2019

[7] [7]

Aiken.Rating Scales And Checklists: Evaluating Behavior, Personality, And Attitudes.Ratingscalesandchecklists: Evaluatingbehavior,personality,andattitudes

Lewis R. Aiken.Rating Scales And Checklists: Evaluating Behavior, Personality, And Attitudes.Ratingscalesandchecklists: Evaluatingbehavior,personality,andattitudes. John Wiley & Sons, Oxford, England, 1996. ISBN 0-471-12787-6

1996

[8] [8]

FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP

Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP. In Waleed Ammar, Annie Louis, and Nasrin Mostafazadeh, editors,Proceedings of the 357 BIBLIOGRAPHY 358 of 455 2019 Conference of the North American Chapter of the Association for Computational Lin- guisti...

doi:10.18653/v1/n19-4010 2019