Pith. sign in

REVIEW 4 major objections 6 minor 53 references

Using only prompts, Claude 3.5 Sonnet identifies misleading YouTube thumbnails with 93.8% accuracy across eight countries, matching or beating a supervised detector trained for the task.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Claude 3.5 Sonnet, prompted with thumbnails, subtitles, and video summaries, detects misleading YouTube thumbnails with up to 93.8% accuracy on a new cross-country dataset.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A genuinely useful new dataset and a plausible LLM-prompting result, but the headline comparison with CHECKER does not survive contact with the actual numbers. the 4 major comments →

arxiv 2509.04714 v1 pith:RQNI4JEI submitted 2025-09-05 cs.SI

ThumbnailTruth: A Multi-Modal LLM Approach for Detecting Misleading YouTube Thumbnails Across Diverse Cultural Settings

Wajiha Naveed , Zartash Afzal Uzmi , Zafar Ayyub Qazi This is my paper
classification cs.SI
keywords misleading thumbnailsYouTubemultimodal LLMclickbait detectiondynamic few-shot promptingcontent moderationcross-cultural datasetsemantic alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a general-purpose multimodal LLM, prompted rather than trained for the task, can detect misleading YouTube thumbnails at levels competitive with specialised supervised systems. On a new dataset of 2,843 videos from eight countries—1,359 of them judged misleading, with over 7.6 billion collective views—the authors feed each model the thumbnail image, subtitle transcript, and an automatically generated video description, and ask it to judge semantic alignment. Claude 3.5 Sonnet with dynamic few-shot prompting reaches 93.8% accuracy, with precision above 92% and recall above 94% in several settings, and its F1 of 0.7227 edges out CHECKER's 0.7153, a supervised pipeline built for this exact task. Why it matters: if the result holds, platforms could screen thumbnails before upload without training bespoke detectors, and adapt to local clickbait styles by swapping in example videos rather than retraining.

Core claim

The paper's central claim is that a general-purpose multimodal LLM, used with engineered prompts and no fine-tuning, can detect misleading YouTube thumbnails as accurately as a specialised supervised detector. Feeding the model three signals—the thumbnail image, the subtitle transcript, and an automatically generated scene-by-scene video description—and asking it to judge whether the thumbnail matches the content, Claude 3.5 Sonnet reaches 93.8% accuracy on a dataset of 2,843 videos from eight countries, with precision above 92%, specificity above 93%, and recall above 94% in several settings. Benchmarking against CHECKER, the best supervised pipeline for this task, the prompted Claude confi

What carries the argument

The load-bearing object is the semantic-alignment judgment itself, assembled from three modalities per video: the thumbnail image, the subtitle transcript, and a model-generated scene-by-scene video description. The mechanism that sharpens it is dynamic few-shot prompting: Sentence-BERT embeddings of the video description retrieve one known-misleading and one known-honest example semantically closest to the test video, and those examples, with thumbnail descriptions and rationales, are inserted into the prompt. This turns the task into a comparison against local exemplars, which is what the authors credit for the cross-country accuracy gains, including at least 8 percentage points in Italy,

Load-bearing premise

The paper treats the two annotators' agreement (Cohen's kappa 0.9633) as the ground truth for what makes a thumbnail misleading in all eight countries, so every accuracy figure is really a measure of agreement with that subjective codebook—if the codebook misses regional expectations, the headline 93.8% is not a true detection rate.

What would settle it

Take a stratified sample of the 2,843 videos and have a larger panel of annotators from each of the eight countries label thumbnails as misleading without seeing the paper's codebook, then compare the panel labels with the paper's labels; substantial disagreement (say, more than 10% on any country) would show the 93.8% accuracy is an artifact of the original two annotators' shared biases rather than a measure of genuine deception.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Pre-upload screening becomes feasible: the pipeline uses only signals available before public release, so a misleading thumbnail can be flagged for human review at upload time rather than after viewers are exposed.
  • Cultural adaptation can be achieved by prompt selection, not retraining: swapping in locally relevant few-shot examples raised accuracy by at least 8 percentage points in Italy, Brazil, and Pakistan.
  • A general LLM can match a specialised supervised detector on this task: Claude's F1 of 0.7227 exceeds CHECKER's 0.7153, despite zero task-specific training.
  • High specificity (Claude above 93%) keeps false flags low enough that flagged thumbnails could be routed to human moderators without overwhelming them.
  • The same pipeline transfers across video categories: balanced per-category accuracy ranged from 0.875 for Science & Technology to 0.953 for Sports, suggesting the mechanism is not niche to one genre.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the 93.8% figure is an agreement rate with a two-annotator codebook, not a measurement against viewer deception; a genuinely independent gold standard, such as post-viewing surveys per country, would likely lower the headline number, especially for culturally coded clickbait.
  • Beyond the paper: because the pipeline omits titles, comments, likes, and view counts by design, it cannot catch thumbnails that are deceptive only in relation to metadata or social signals; a hybrid system would be needed for post-hoc moderation.
  • Beyond the paper: dynamic few-shot retrieval depends on the quality of the generated video descriptions; if those descriptions contain errors, the retrieved examples can be off-topic, and the accuracy gain could shrink on videos with sparse or poor subtitles.
  • Beyond the paper: one could test the mechanism's limits by adversarial generation—feeding the pipeline thumbnails designed by an LLM to be subtly misleading—to see whether the semantic mismatch judgment degrades against adaptive creators.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ThumbnailTruth, a dataset of 2,843 YouTube videos from eight countries, with 1,359 videos labeled as having misleading thumbnails and 1,484 as non-misleading, along with a codebook, scripts, and a subset of data made public. The detection pipeline combines thumbnail images, subtitle transcripts, and generated video-to-text descriptions, and is evaluated with four multimodal LLMs (Claude 3.5 Sonnet, Gemini 1.5 Flash, GPT-4o, GPT-4o Mini) under zero-shot, fixed few-shot, and dynamically retrieved few-shot prompting. The headline result is that Claude 3.5 Sonnet with dynamic few-shot prompting reaches 93.8% accuracy, with high precision and recall. The paper also compares the best Claude configuration to CHECKER, a supervised multimodal detector, reporting an F1 of 0.7227 versus 0.7153, and concludes that prompt-based LLMs can match or exceed state-of-the-art task-specific models without training. Additional analyses cover per-category performance, cultural/geographic variation, cost, and a modality ablation.

Significance. If the dataset and evaluation protocol are accepted, the paper makes a useful contribution: a publicly released cross-cultural dataset for a real content-moderation problem, a reproducible multimodal LLM pipeline, and systematic prompt ablations. I credit the authors for releasing the dataset, codebook, and scripts, and for evaluating multiple commercial LLMs under a consistent input setup. However, the central comparative claim against CHECKER is not supported by the data as reported, the ablation table contradicts the accompanying text, and the evaluation protocol has a transductive flavor that makes the headline accuracy hard to interpret as a prospective detection rate. These issues are load-bearing for the paper's main claims, so the manuscript needs substantial revision rather than minor polishing.

major comments (4)
  1. [Section 5, Table 3] The F1 comparison with CHECKER is invalid as presented. Claude 3.5 Sonnet + Dynamic Examples has F1 = 0.7227 computed on the ThumbnailTruth dataset (2,843 videos, ~48% positive), while CHECKER + Block (τ=0.30) has F1 = 0.7153 taken from the original CHECKER 197-video test set (64 clickbait, 133 non-clickbait, ~32.5% positive). These are different evaluation sets with different class priors, content distributions, and annotation protocols. F1 values are not portable across benchmarks, so the statement that 'Claude matched or outperformed CHECKER on every metric' and the conclusion that prompt-based LLMs can reach state-of-the-art accuracy without task-specific training do not follow. Both systems must be run on a shared held-out test set, or the comparison must be removed/reframed as indicative only.
  2. [Section 6, Table 4] The ablation table contradicts the text. The paper states that 'the best overall performance is observed in the full-input setting, Claude-Zero Shot', but the table shows full-input accuracy 0.8920, which is below ABL-NS (0.9076) and ABL-ND (0.9077); the full-input recall 0.8430 is also below ABL-ND (0.8987) and ABL-NS (0.8856). The likely explanation is that incomplete responses (refusals) were excluded differently across rows, so metrics are not computed on a comparable subset. The authors should report the number of excluded cases per configuration, recompute metrics on the common subset, and correct the claims about complementarity and the superiority of the full input.
  3. [Section 2.4 and Section 4.2] The evaluation protocol is transductive and the headline accuracy lacks uncertainty quantification. Dynamic few-shot examples are retrieved from the same dataset by semantic similarity to the input video's description, and the best prompt is selected after seeing the full evaluation dataset. This means the reported 93.8% is not a prospective or train/test estimate and cannot be directly compared with supervised models evaluated on separate test sets. The paper should use a held-out split for prompt selection and retrieval, state explicitly whether ground-truth labels of test instances are used anywhere in prompt construction, and report confidence intervals or significance tests for the differences in Table 1 (e.g., 93.8% vs 84.8%).
  4. [Section 2.2] The gold standard is the subjective judgment of two annotators (one author and one graduate student), with Cohen's kappa 0.9633. All reported accuracy, precision, recall, and specificity numbers measure agreement with these two annotators' codebook, not with an independent platform or audience ground truth. Given the paper's cross-cultural claims, this is a load-bearing concern: the 93.8% figure may reflect inter-rater agreement with the codebook rather than detection of 'actually misleading' thumbnails. The authors should provide an external validation (e.g., additional annotators from each country, or comparison with platform outcomes), or at minimum report per-country disagreement rates and borderline-case resolution.
minor comments (6)
  1. [Section 2.1] The text says country classification used the UN World Economic Situation and Prospects 2024 report, but reference [5] is a Statista URL about YouTube users. The citation does not match the source described.
  2. [Section 3.2] Typo: 'We acknowledge that certain video types. particularly...' should be 'certain video types, particularly...'.
  3. [Table 2] The per-category table reports accuracy and F1 on a 'balanced subset' but does not give the number of videos per category. Without support counts, the F1 values are hard to interpret, especially for categories with small N.
  4. [Table 6] The caption says 'Average Number of Videos Processed by Each Model' but the rows appear to be total counts (e.g., Claude processed 2,759 videos). Clarify whether these are counts or averages, and report the denominator for the full dataset.
  5. [Section 6] The subsection titled 'Limitations and Areas for Improvement' discusses only model weaknesses (celebrity recognition, etc.). The paper should also include a limitations subsection covering dataset biases, annotation subjectivity, and the transductive evaluation issues raised above.
  6. [Section 7] The text refers to an 'anonymized GitHub repository', but Section 1 gives a non-anonymized repository URL and the author names are present. For a double-blind review, this should be anonymized consistently, or the wording should be changed.

Circularity Check

0 steps flagged

No significant circularity: the paper's claims are empirical measurements, not derivations; the CHECKER comparison and dynamic-few-shot evaluation raise validity concerns but do not reduce to the paper's own inputs by construction.

full rationale

The paper contains no mathematical derivation chain in which an output is shown equal to an input by construction. The central claims (Claude reaches 93.8% accuracy; dynamic few-shot prompting improves over zero-shot) are reported measurements on a newly annotated dataset. The gold labels were produced by two annotators following a codebook, and the LLM prompt uses a definition of 'misleading thumbnail' drawn from the same sources; this makes the metric a measure of agreement with that operationalization, but the model was not trained on those labels and the measurement is not equivalent to the input by definition. Two validity concerns are real but are not circularity under the specified patterns: (1) Section 5, Table 3 compares Claude's F1 (0.7227 on the authors' 2,843-video set) with CHECKER's F1 (0.7153 on CHECKER's original 197-video test set), so the 'matched or outperformed CHECKER' claim lacks a shared test set; this is a comparability/evidence problem, not a reduction of the conclusion to its premise. (2) Dynamic few-shot examples are retrieved from the same dataset and include ground-truth labels, and the best prompt was selected after seeing the evaluation set; this creates transductive leakage/selection bias that can inflate reported accuracy, but the prediction is not defined as the retrieved label nor forced by construction. No load-bearing self-citation or imported uniqueness theorem appears. Accordingly, no circular step is identified.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central result rests on human annotation labels, LLM-generated video summaries, and prompt choices made after inspecting the same dataset. None of these are verified by an independent benchmark or external ground truth. No new particles, forces, or entities are introduced; the only new artifact is the dataset and the operational definition of 'misleading thumbnail'.

free parameters (3)
  • number of in-context examples (dynamic and fixed few-shot) = 2 (one misleading, one non-misleading)
    Hand-picked; no sweep over 1, 3, or 4 examples is reported, so the effect of this choice on the 93.8% figure is unknown.
  • context truncation lengths = 200 words for subtitles/descriptions; 29m55s for long videos
    Selected after empirical testing that longer inputs hurt accuracy; the exact trade-off is not quantified and no ablation is shown.
  • Claude frame sample count = 20 evenly spaced frames
    Used because Claude 3.5 Sonnet does not accept video input; no ablation over frame count is reported.
axioms (5)
  • domain assumption Two annotators' labels (Cohen's kappa 0.9633) are an accurate ground truth for misleading thumbnails across all eight countries.
    Section 2.2: labels were assigned by one author and one graduate using a codebook; all metrics are computed against these labels.
  • domain assumption LLM-generated video descriptions (from Gemini, Claude, Twelve Labs) faithfully summarize the video content.
    Section 2.3: descriptions are used as the main content signal; the paper notes they occasionally introduce inaccuracies.
  • domain assumption YouTube's default hqdefault.jpg thumbnail is the thumbnail viewers see and matches the phenomenon under study.
    Section 2.3: only the default high-quality thumbnail is downloaded; alternative thumbnail variants are not considered.
  • domain assumption Google Translate preserves enough meaning in non-English thumbnails and subtitles to support classification.
    Section 2.2: non-English text was translated to English; translation errors could affect the comparison between thumbnail and content.
  • ad hoc to paper The best model and prompt can be selected on the full evaluation dataset without invalidating the reported accuracy.
    Sections 2.4 and 4.2: no train/validation/test split is described; dynamic few-shot examples are retrieved from the same dataset using ground-truth labels.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ThumbnailTruth: A Multi-Modal LLM Approach for Detecting Misleading YouTube Thumbnails Across Diverse Cultural Settings." pith.science (2026). https://pith.science/paper/RQNI4JEI

@misc{pith2026250904714,
  author       = {Pith},
  title        = {Pith review of: ThumbnailTruth: A Multi-Modal LLM Approach for Detecting Misleading YouTube Thumbnails Across Diverse Cultural Settings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RQNI4JEI}},
  note         = {Machine review of arXiv:2509.04714}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Misleading video thumbnails on platforms like YouTube are a pervasive problem, undermining user trust and platform integrity. This paper proposes a novel multi-modal detection pipeline that uses Large Language Models (LLMs) to flag misleading thumbnails. We first construct a comprehensive dataset of 2,843 videos from eight countries, including 1,359 misleading thumbnail videos that collectively amassed over 7.6 billion views -- providing a unique cross-cultural perspective on this global issue. Our detection pipeline integrates video-to-text descriptions, thumbnail images, and subtitle transcripts to holistically analyze content and flag misleading thumbnails. Through extensive experimentation and prompt engineering, we evaluate the performance of state-of-the-art LLMs, including GPT-4o, GPT-4o Mini, Claude 3.5 Sonnet, and Gemini-1.5 Flash. Our findings show the effectiveness of LLMs in identifying misleading thumbnails, with Claude 3.5 Sonnet consistently showing strong performance, achieving an accuracy of 93.8\%, precision over 92\%, and recall exceeding 94\% in certain scenarios. We discuss the implications of our findings for content moderation, user experience, and the ethical considerations of deploying such systems at scale. Our findings pave the way for more transparent, trustworthy video platforms and stronger content integrity for audiences worldwide.

Figures

Figures reproduced from arXiv: 2509.04714 by Wajiha Naveed, Zafar Ayyub Qazi, Zartash Afzal Uzmi.

Figure 1
Figure 1. Figure 1: Overall categorical distribution of MTVs and [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Radar plot of best prompt accuracy for four models [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Model accuracies for each prompt significant limitations in celebrity recognition. While Claude partially compensated with logical reasoning to infer content without explicitly identifying celebrities, other models strug￾gled to recognize prominent figures, which led to misclassi￾fications in scenarios where identification was essential. This limitation highlights a shared weakness across all models, excep… view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy using Claude 3.5 Sonnet for each prompt [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Graph comparing model performances across [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: illustrates the radar plot for Claude, showcas￾ing its performance across four metrics: Accuracy, Recall, Precision, and Specificity. Similarly, [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Radar plot for Gemini for all prompts [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 10
Figure 10. Figure 10: Top 3 Models with their Highest Overall Accu [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Heatmap for Model Accuracies Across Prompts [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Claude Accuracies Across Prompts for Each [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 44 canonical work pages · 2 internal anchors

  1. [1]

    The impact of misleading thumbnails on user engagement,

    C. Srinivasan, J. Smith, and E. Johnson, “The impact of misleading thumbnails on user engagement,” inProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, 2021, pp. 1–12

    work page 2021

  2. [2]

    Americans and digital misinfor- mation,

    Pew Research Center, “Americans and digital misinfor- mation,” Pew Research Center, Tech. Rep., 2022. [On- line]. Available: https://www.pewresearch.org/internet/2022/01/31/ americans-and-digital-misinformation/

    work page 2022

  3. [3]

    Youtube for press,

    YouTube, “Youtube for press,” https://blog.youtube/press/, 2024

    work page 2024

  4. [4]

    Checker: Detecting clickbait thumbnails with weak supervision and co-teaching,

    T. Xie, T. Le, and D. Lee, “Checker: Detecting clickbait thumbnails with weak supervision and co-teaching,” inMachine Learning and Knowledge Discovery in Databases. Applied Data Science Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part V 21. Springer, 2021, pp. 415–430

    work page 2021

  5. [5]

    Number of monthly unique youtube users,

    Statista, “Number of monthly unique youtube users,” https://www.statista.com/statistics/280685/ number-of-monthly-unique-youtube-users/, 2024

    work page 2024

  6. [6]

    Measuring misinformation in video search platforms: An audit study on youtube,

    E. Hussein, P. Juneja, and T. Mitra, “Measuring misinformation in video search platforms: An audit study on youtube,”Proceedings of the ACM on Human-Computer Interaction, vol. 4, no. CSCW1, pp. 1–27, 2020

    work page 2020

  7. [7]

    Adolescents’ experiences and (re)action towards fake news on social media: Perspectives from norway,

    F. M. Lindstøl, C. E. Tømte, and O. E. Hatlevik, “Adolescents’ experiences and (re)action towards fake news on social media: Perspectives from norway,”Nordic Journal of Comparative and International Education (NJCIE), vol. 7, no. 1, pp. 1–18, 2023. [Online]. Available: https://doi.org/10.7577/njcie.4883

    work page doi:10.7577/njcie.4883 2023

  8. [8]

    Towards crowdsourcing clickbait labels for youtube videos,

    J. Qu, A. M. Hißbach, T. Gollub, and M. Potthast, “Towards crowdsourcing clickbait labels for youtube videos,” inProceedings of the Workshop and Poster Track of the 6th International Conference on the Analysis of Images, Social Networks and Texts (AIST 2017), ser. CEUR Workshop Proceedings, vol. 2210. CEUR-WS.org, 2018. [Online]. Available: http://ceur-ws...

    work page 2017

  9. [9]

    Youtube data api,

    Google Cloud, “Youtube data api,” https://console.cloud.google.com/ marketplace/product/google/youtube.googleapis.com, 2024

    work page 2024

  10. [10]

    Pytubefix repository,

    JuanBindez, “Pytubefix repository,” https://github.com/JuanBindez/ pytubefix, 2024

    work page 2024

  11. [11]

    Gemini 1.5 flash model,

    Google Cloud, “Gemini 1.5 flash model,” https://console.cloud. google.com/vertex-ai/publishers/google/model-garden/gemini-1. 5-flash-002, 2024

    work page 2024

  12. [12]

    Claude 3.5 sonnet model,

    ——, “Claude 3.5 sonnet model,” https://console.cloud.google. com/vertex-ai/publishers/anthropic/model-garden/claude-3-5-sonnet, 2024

    work page 2024

  13. [13]

    Twelve labs: The platform for video understanding,

    Twelve Labs, “Twelve labs: The platform for video understanding,” https://www.twelvelabs.io, 2024

    work page 2024

  14. [14]

    Long context rag: Performance with llms,

    Databricks, “Long context rag: Performance with llms,” https:// www.databricks.com/blog/long-context-rag-performance-llms, 2024, accessed: 2025-01-20

    work page 2024

  15. [15]

    Ruler: What’s the real context size of your long-context language models?

    C.-P. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, Y . Zhang, and B. Ginsburg, “Ruler: What’s the real context size of your long-context language models?” 2024. [Online]. Available: https://arxiv.org/abs/2404.06654

    Pith/arXiv arXiv 2024

  16. [16]

    Prompting techniques guide,

    Prompting Guide, “Prompting techniques guide,” https://www. promptingguide.ai/techniques, 2024

    work page 2024

  17. [17]

    Large language models are zero-shot reasoners,

    T. Kojima, S. S. Gu, M. Reid, Y . Matsuo, and Y . Iwasawa, “Large language models are zero-shot reasoners,” 2023. [Online]. Available: https://arxiv.org/abs/2205.11916

    Pith/arXiv arXiv 2023

  18. [18]

    The good, the bad and the bait: Detecting and characterizing clickbait on youtube,

    S. Zannettou, S. Chatzis, K. Papadamou, and M. Sirivianos, “The good, the bad and the bait: Detecting and characterizing clickbait on youtube,” in2018 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 2018, pp. 163–171

    work page 2018

  19. [19]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” 2019. [Online]. Available: https: //arxiv.org/abs/1908.10084

    Pith/arXiv arXiv 2019

  20. [20]

    Long-context llms struggle with long in-context learning,

    T. Li, G. Zhang, Q. D. Do, X. Yue, and W. Chen, “Long-context llms struggle with long in-context learning,” 2024. [Online]. Available: https://arxiv.org/abs/2404.02060

    Pith/arXiv arXiv 2024

  21. [21]

    Openai models documentation,

    OpenAI, “Openai models documentation,” https://platform.openai. com/docs/models, 2024

    work page 2024

  22. [22]

    Thumbnails policy,

    YouTube, “Thumbnails policy,” https://support.google.com/youtube/ answer/9229980?hl=en, 2024

    work page arXiv 2024

  23. [23]

    Community guidelines,

    ——, “Community guidelines,” https://www.youtube.com/ howyoutubeworks/policies/community-guidelines/, 2024

    work page 2024

  24. [24]

    Gemini api safety settings,

    Google AI, “Gemini api safety settings,” https://ai.google.dev/ gemini-api/docs/safety-settings, 2024

    work page 2024

  25. [25]

    Configure safety filters for generative ai studio,

    Google Cloud, “Configure safety filters for generative ai studio,” https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/ configure-safety-filters, 2024

    work page 2024

  26. [26]

    Safety at openai,

    OpenAI, “Safety at openai,” https://openai.com/safety/, 2024

    work page 2024

  27. [27]

    Acceptable use policy,

    Anthropic, “Acceptable use policy,” https://www.anthropic.com/legal/ aup, 2024

    work page 2024

  28. [28]

    The ethics of interaction: Mitigating security threats in llms,

    A. Kumar, S. Singh, S. V . Murty, and S. Ragupathy, “The ethics of interaction: Mitigating security threats in llms,”arXiv preprint arXiv:2401.12273, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Safety layers of aligned large language models: The key to llm security,

    S. Li, L. Yao, L. Zhang, and Y . Li, “Safety layers of aligned large language models: The key to llm security,”arXiv preprint arXiv:2408.17003, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Chain-of-thought prompting,

    Anthropic, “Chain-of-thought prompting,” https://docs.anthropic.com/ en/docs/build-with-claude/prompt-engineering/chain-of-thought, 2024, accessed: 2024-12-07

    work page 2024

  31. [31]

    Twelve labs pricing,

    Twelve Labs, “Twelve labs pricing,” https://www.twelvelabs.io/ pricing, 2024

    work page 2024

  32. [32]

    Batch processing guide,

    OpenAI, “Batch processing guide,” https://platform.openai.com/docs/ guides/batch, 2024

    work page 2024

  33. [33]

    Message batches api,

    Anthropic, “Message batches api,” https://www.anthropic.com/news/ message-batches-api, 2024

    work page 2024

  34. [34]

    Prompt caching,

    ——, “Prompt caching,” https://www.anthropic.com/news/ prompt-caching, 2024

    work page 2024

  35. [35]

    Context cache overview,

    Google Cloud, “Context cache overview,” https://cloud.google.com/ vertex-ai/generative-ai/docs/context-cache/context-cache-overview, 2024

    work page 2024

  36. [36]

    Api prompt caching,

    OpenAI, “Api prompt caching,” https://openai.com/index/ api-prompt-caching/, 2024

    work page 2024

  37. [37]

    Batch prediction with gemini models,

    Google Cloud, “Batch prediction with gemini models,” https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/ batch-prediction-gemini, 2024

    work page 2024

  38. [38]

    Ageval: A benchmark for zero-shot and few- shot plant stress phenotyping with multimodal llms,

    M. A. Arshad, T. Z. Jubery, T. Roy, R. Nassiri, A. K. Singh, A. Singh, C. Hegde, B. Ganapathysubramanian, A. Balu, A. Kr- ishnamurthyet al., “Ageval: A benchmark for zero-shot and few- shot plant stress phenotyping with multimodal llms,”arXiv preprint arXiv:2407.19617, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Evaluating the utility- truthfulness trade-off in large language model agents: A comparative study of chatgpt, gemini, and claude

    K. Whitbeck, L. Brown, and S. Abernathy, “Evaluating the utility- truthfulness trade-off in large language model agents: A comparative study of chatgpt, gemini, and claude.”

    work page

  40. [40]

    Efficacy of synthetic data as a benchmark,

    G. Maheshwari, D. Ivanov, and K. E. Haddad, “Efficacy of synthetic data as a benchmark,”arXiv preprint arXiv:2409.11968, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types

    N. Sinha, V . Jain, and A. Chadha, “Guiding vision-language model selection for visual question-answering across tasks, domains, and knowledge types,”arXiv preprint arXiv:2409.09269, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  42. [42]

    Infinibench: A comprehensive benchmark for large multimodal models in very long video understanding,

    K. Ataallah, C. Gou, E. Abdelrahman, K. Pahwa, J. Ding, and M. Elhoseiny, “Infinibench: A comprehensive benchmark for large multimodal models in very long video understanding,”arXiv preprint arXiv:2406.19875, 2024

    arXiv 2024

  43. [43]

    Localizing youtube: Language, cultural regions, and digital platforms,

    S. Mohan and A. Punathambekar, “Localizing youtube: Language, cultural regions, and digital platforms,”International journal of cul- tural studies, vol. 22, no. 3, pp. 317–333, 2019

    work page 2019

  44. [44]

    Exploring audience en- gagement with chatgpt-related content on youtube: Implications for content creators and ai tool developers,

    K. Hussain, M. L. Khan, and A. Malik, “Exploring audience en- gagement with chatgpt-related content on youtube: Implications for content creators and ai tool developers,”Digital Business, vol. 4, no. 1, p. 100071, 2024

    work page 2024

  45. [45]

    Digital services act (dsa),

    European Commission, “Digital services act (dsa),” 2024. [On- line]. Available: https://commission.europa.eu/strategy-and-policy/ priorities-2019-2024/europe-fit-digital-age/digital-services-act en

    work page 2024

  46. [46]

    Mis- leading metadata detection on youtube,

    P. Palod, A. Patwari, S. Bahety, S. Bagchi, and P. Goyal, “Mis- leading metadata detection on youtube,” inAdvances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part II 41. Springer, 2019, pp. 140–147

    work page 2019

  47. [47]

    Towards re- liable online clickbait video detection: A content-agnostic approach,

    L. Shang, D. Y . Zhang, M. Wang, S. Lai, and D. Wang, “Towards re- liable online clickbait video detection: A content-agnostic approach,” Knowledge-Based Systems, vol. 182, p. 104851, 2019

    work page 2019

  48. [48]

    Disinformation in multimedia annotation: Misleading metadata detection on youtube,

    P. Bajaj, M. Kavidayal, P. Srivastava, M. N. Akhtar, and P. Ku- maraguru, “Disinformation in multimedia annotation: Misleading metadata detection on youtube,” inProceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion, 2016, pp. 53–61

    work page 2016

  49. [49]

    Baitradar: A multi-model clickbait detection algorithm using deep learning,

    B. Gamage, A. Labib, A. Joomun, C. H. Lim, and K. Wong, “Baitradar: A multi-model clickbait detection algorithm using deep learning,” inICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 2665–2669

    work page 2021

  50. [50]

    Chatgpt outperforms crowd workers for text-annotation tasks,

    F. Gilardi, M. Alizadeh, and M. Kubli, “Chatgpt outperforms crowd workers for text-annotation tasks,”Proceedings of the National Academy of Sciences, vol. 120, no. 30, p. e2305016120, 2023

    work page 2023

  51. [51]

    Benchmarking LLMs in Political Content Text-Annotation: Proof-of-Concept with Toxicity and Incivility Data

    B. Gonz ´alez-Bustamante, “Benchmarking llms in political content text-annotation: Proof-of-concept with toxicity and incivility data,” arXiv preprint arXiv:2409.09741, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  52. [52]

    {PIXELMOD}: Improving soft moderation of visual misleading information on twit- ter,

    P. Paudel, C. Ling, J. Blackburn, and G. Stringhini, “{PIXELMOD}: Improving soft moderation of visual misleading information on twit- ter,” in33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 5125–5142. Appendix

    work page 2024

  53. [53]

    Supplementary Data The following tables and figures provide supplementary data that support the main findings of our study. These include detailed steps used for classifying YouTube thumb- nails, the number of videos processed by each model, ac- curacy metrics for each model across different prompting strategies shown using radar plots for better visualiz...

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.