Pith. sign in

REVIEW 3 major objections 6 minor 135 references

CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new scoring suite measures a text-to-image model's cultural knowledge by how much its output changes as the prompt gains detail, and reports stronger correlation with culture-matched human ratings than existing automated proxies.

desk verdict A genuinely useful cultural-benchmark idea held back by a three-rater gold standard that needs statistical reinforcement before the headline advantage is trusted. read the letter →

arxiv 2506.08071 v1 pith:23E3R3G5 submitted 2025-06-09 cs.CV

Aniket Rege , Zinnia Nie , Mahesh Ramesh , Unmesh Raskar , Zhuoran Yu , Aditya Kusupati , Yong Jae Lee , Ramya Korlakai Vinayak This is my paper
classification cs.CV
keywords culturalrepresentativenesstext-to-imagebiasmarginalinformationattributionlong-taildistributionbenchmarkdatasetWikimediaknowledgegraphgenerativeentanglementdiversityscoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CuRe claims that a text-to-image system's cultural representativeness can be read off from how its output changes as the prompt becomes more informative: adding the artifact's category, region, or both. The paper formalizes this as a family of marginal-information-attribution (MIA) scorers built from three standard score types, and reports that these correlate with culture-matched human Likert ratings more strongly than existing proxy scorers, including ground-truth similarity, CLIP-style text alignment, and Vendi-score diversity. The dataset behind the claim is a 300-artifact benchmark built from the Wikimedia knowledge graph with a six-axis, 32-category, 64-country hierarchy. If the claim holds, cultural representativeness becomes cheaply and scalably measurable without ground-truth images or large user studies.

What carries the argument

The machinery is a coarse-to-fine attribute hierarchy (supercategory $s \rightarrow$ category $c \rightarrow$ name $n$, with region $r$) plus a template family $P(a)$ that turns any subset of these attributes into a prompt. The MIA scorers are the load-bearing objects: $\phi_{PS}$ compares the image generated from the artifact's name to the image generated from its category; $\phi_{ITA}$ averages image-text similarity to the name prompt and to a more informative prompt; $\phi_{DIV}$ averages LPIPS over the four generation conditions $\{n\}$, $\{n,c\}$, $\{n,r\}$, $\{n,c,r\}$. Because these scores compare generated images against other generated images, they require no ground-truth image collection, which is what makes the benchmark scalable, and the 300-artifact dataset with 64 countries plus the culture-matched user study supply the gold scores the scorers are validated against.

What would settle it

Re-run the correlation analysis with a substantially larger rater pool, tens of workers per artifact, and a calibrated rubric, then recompute the Spearman values in the paper's tables; if the MIA scorers' advantage over ground-truth-requiring baselines shrinks to within the bootstrap uncertainty of the gold scores, the central claim collapses. A second, cheaper check targets $\phi_{PS}$ directly: for an artifact whose category image $I(c)$ is clearly wrong to culture-matched judges, the scorer will rank the artifact as long-tail even when the model generates it faithfully.

Watch

Extended reading notes

Core claim

The central claim is that the marginal utility of attribute specification is a valid, cheaper proxy for human judgments of cultural representativeness. For an artifact $n$ from category $c$ and region $r$, the paper defines $\phi_{PS}(n) = \mathrm{sim}(I(n), I(c))$, $\phi_{ITA}(a) = \left[\mathrm{sim}(I(n), P(n)) + \mathrm{sim}(I(n), P(a))\right]/2$, and $\phi_{DIV} = \mathrm{LPIPS}\big(\{n\},\{n,c\},\{n,r\},\{n,c,r\}\big)$, interpreting a small divergence across these generation conditions as evidence that the artifact sits in the head of the model's learned distribution. Across several image encoders, vision-language models, and six text-to-image systems, these scorers achieve higher-magnitude Spearman correlations with human Likert scores for cultural representativeness, perceptual similarity, and ground-truth likelihood than baselines that either require real reference images or share pretraining data with the generators. The paper also documents generative entanglement: scores computed with models trained on the same web corpora as a text-to-image system systematically overestimate its cultural quality, an error the MIA scorers are more robust to.

Load-bearing premise

The whole evaluation rests on the gold standard: the ratings of just three paid crowd workers per artifact, collected without a rubric, are assumed to be a stable enough measure of cultural representativeness that correlations against them are meaningful.

Editorial extensions

If this is right

  • Cultural representativeness can be benchmarked automatically and at scale, without curated ground-truth image sets or expensive user studies, since the MIA scorers compare generations across prompt informativeness levels.
  • Existing CLIP-style scores are suspect for systems trained on the same web data: the generative-entanglement result implies leaderboards that rely on them systematically overstate cultural quality.
  • The per-artifact scores expose which specific long-tail artifacts fail (for example banku versus pierogi), enabling targeted data curation rather than coarse aggregate judgments.
  • Because diversity correlates inversely with human judgments, interventions that increase generation diversity should be checked against factuality instead of being assumed to improve cultural representativeness.
  • Multimodal LLM judges, while strong overall, are unreliable on Global South details and still hallucinate culture-specific facts, so they cannot replace the MIA scorers without ground-truth anchoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same four-prompt marginal-utility protocol should transfer to non-cultural long-tail domains, such as rare species, niche professions, or regional dialects of visual style; a direct test would be to run the protocol on another tail-heavy benchmark and check whether human ratings again align with the divergence scores.
  • The scorers inherit any bias in the model's category prototype: if the model cannot draw a generic "dumpling" well, $\phi_{PS}$ will misrank its dumpling artifacts, so a practical refinement would weight or condition on the quality of $I(c)$.
  • The gold scores are noisier than the reporting suggests, since workers assign 1 and 5 to the same image and sometimes score high while disavowing the image, so the observed correlation advantages should be re-estimated with larger rater pools and a rubric to see whether the ranking of scorers survives.
  • The geography-as-culture proxy, one country per artifact, is the main limit on what the scores mean; extending the attribute vocabulary to religion, language, or ethnicity is a direct application of the same machinery.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces CuRe, a 300-artifact cultural benchmark dataset built from the Wikimedia knowledge graph with a coarse-to-fine categorical hierarchy, and a family of 'Marginal Information Attribution' (MIA) scorers that measure how a text-to-image system's outputs change as attributes (name, category, region) are added to the prompt. The central claim is that these scorers, instantiated as perceptual similarity (phi_PS), image-text alignment (phi_ITA), and diversity (phi_DIV), correlate better with human Likert judgments of cultural representativeness, perceptual similarity, and ground-truth likelihood than existing proxy scorers. The paper validates this claim through a Prolific user study with three self-identified cultural workers per artifact, reports Spearman correlations across multiple image encoders, vision-language models, and six T2I systems, and additionally evaluates Gemini 2.0 Flash as an MLLM judge. The dataset and code are open-sourced.

Significance. If the central claim is validated, CuRe would be a genuinely useful resource: the dataset is broader in country coverage than most prior cultural benchmarks, the hierarchy enables a new form of fine-grained analysis, and the MIA scorers avoid ground-truth image collection, which is a real practical advantage. The paper is also unusually transparent: it discloses the SD 1.5 failure mode, DALL-E 3 refusals, Gemini hallucinations, and illustrative worker disagreement cases, and it ships open code and data. The concern that the 0.5 offset in Delta_phi_PS is a fitted parameter does not survive inspection: the offset is a constant shift and is rank-invariant, so it cannot affect the reported Spearman correlations. However, the validation target itself is the load-bearing weakness: with three no-rubric raters per artifact and documented scale reversals, the Spearman correlations in Tables 2-4 may be attenuated or reordered, and the reported advantages of MIA scorers over baselines are often small enough that uncertainty quantification is essential before the headline claim can be accepted.

major comments (3)
  1. [Sec. 5.1, Appendix C, Appendix D.2, Tables 2-4] The validation target for every comparative claim in Tables 2-4 is the mean of exactly three Prolific workers per artifact, rating Likert items with no rubric. The only inter-annotator agreement analysis reported (Table 10, Appendix D.2) is for the four-image ranking question Q1b, not for the Likert items phi*_CuRe, phi*_PS, and phi*_GT that are the actual gold labels. Figure 13 documents extreme instability on those Likert items: workers assign 1 and 5 to the same generated image, one worker gives a perfect CuRe score while explicitly stating the image does not depict the artifact (E7 Hardangerbunad), and one worker gives a 1 while calling the image 'pretty accurate' (E4 cowboy hat), suggesting scale reversal. With three ratings per artifact, measurement error attenuates all Spearman rhos and can reorder scorers when the differences are as small as 0.02-0.03 (e.g., Table 3, FLUX.1, phi_ITA({c,r}) = 0.27 vs. sim(I(n),P({c,r})) = 0.25). The paper should report bootstrap confidence intervals for every rho in Tables 2-4, plus rater-level robustness checks such as leave-one-rater-out correlations and an inter-rater reliability measure (e.g., ICC) for the Likert items themselves. This is load-bearing because the entire central claim rests on these correlations.
  2. [Tables 2-4] The Spearman correlations are reported as point estimates without any uncertainty quantification. The best absolute correlations are modest (roughly 0.3-0.5 for phi_PS and phi_ITA, and about -0.3 for phi_DIV on the stronger T2I systems), which means the best scorer explains at most about 25% of the rank variance in the gold labels. Moreover, the claimed advantage over baselines is often small: for example, in Table 3 (FLUX.1 [dev]) phi_ITA({c,r}) is 0.27-0.42 depending on the gold column, while sim(I(n),P({c,r})) is 0.25-0.38 and sim(I(n),P(n)) is 0.24-0.38; in Table 2 the Delta_phi_PS variants sometimes beat phi_GT by only 0.01-0.02. Without confidence intervals or a test of the difference between correlated Spearman coefficients, the phrase 'much stronger correlations' in the abstract and conclusion is not supported. The authors should either add interval estimates and significance tests, or temper the claim to 'consistent but modest improvements.'
  3. [Sec. 5.6, Table 5] The benchmark comparison in Table 5 selects, for each scorer class, 'the variant which had the highest negative Spearman's rho with gold scores on average.' Because the same gold scores were used in Tables 2-4 to identify that best variant, the benchmark rankings in Table 5 are not an out-of-sample evaluation of the scorer family; the variant selection can inflate the apparent performance of the proposed scorers. This selection procedure should be described as such, or better, the choice should be fixed a priori or evaluated with a held-out split of artifacts, so that the benchmark results are not circular with respect to the correlation analysis.
minor comments (6)
  1. [Abstract / Sec. 5.3] The abstract and the conclusion claim 'much stronger correlations' with human judgments, but the magnitudes in Tables 2-4 are moderate and the advantages over strong baselines are often small. Please align the wording with the quantitative results.
  2. [Sec. 5.1, Sec. 5.2, Sec. 5.6, Sec. 4.2] There are several typos and wording slips: 'judments' appears in Secs. 5.2 and 5.3, 'sensistive' in Sec. 5.6, 'inscalable' in Sec. 4.2, and 'Accurate' should likely be 'Accuracy' in Table 1. A careful proofreading pass is needed.
  3. [Table 5] The column headers 'SL2', 'DN2', 'AV2', 'L2B', 'WIT', 'ALX' are not expanded in the table caption; the abbreviations are only explained in surrounding text. Please expand them in the caption for readability.
  4. [Appendix D.2] The worker agreement numbers in Table 10 are reported as mean +/- standard deviation, but the standard deviation is over artifacts, not a confidence interval for the mean agreement. Reporting a confidence interval or the distribution of per-artifact agreement would be more informative.
  5. [Sec. 5.6] The Image Arena ELO scores are stated to be 'taken on September 24, 2025,' which is after the arXiv submission date of June 9, 2025; please clarify the access date or update the reference accordingly.
  6. [Appendix I.2] The regional-trend observation that Gemini failures concentrate in Global South artifacts is interesting and worth stating more prominently, but the supporting evidence is a small set of qualitative examples; consider quantifying this trend across all 300 artifacts.

Circularity Check

0 steps flagged · score 0.0 of 10

CuRe's MIA scorers are fixed, parameter-free definitions validated against external human labels; no circularity found.

full rationale

The central claim is that marginal-information-attribution (MIA) scorers correlate better than existing proxy scorers with human judgments of cultural representativeness, perceptual similarity, and diversity. The MIA scorers are closed-form, parameter-free definitions: phi_PS(a)=sim(I(a),I(c)) (Eq. 2), phi_ITA(a)=[sim(I(n),P(n))+sim(I(n),P(a))]/2 (Eq. 3), and phi_DIV=LPIPS({n},{n,c},{n,r},{n,c,r}) (Eq. 4). No parameter in any of these formulas is fitted to the human Likert scores; the validation in Tables 2-4 is a Spearman correlation between independently collected Prolific judgments (Sec. 5.1) and these fixed scorer outputs. The additive 0.5 in the delta-phi_PS divergence is rank-invariant, so it cannot manufacture Spearman correlation. The 'marginal utility' insight is a hypothesis motivated by qualitative examples, then tested on the full dataset; it is not defined in terms of the gold labels. The only author-overlapping citation, [10], appears in Sec. 5.4 while listing preference-reward-model baselines; it is not load-bearing for the core argument and does not supply an unverified premise or forbid alternatives. The paper's own Limitations section acknowledges validity limits, such as using geography as a proxy for culture and weaker scorer performance on Stable Diffusion 1.5; the 3-rater, no-rubric gold-standard concern flagged by the reader is a measurement-reliability issue, not a circularity issue, because the scorers would still be well-defined even if the human labels were noisy. No self-definitional reduction, fitted-input-called-prediction, imported uniqueness theorem, ansatz-by-citation, or renaming of a known result was found; the derivation chain is self-contained against external human judgments.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central MIA definitions themselves are transparent and parameter-free; the load-bearing extras are dataset construction, gold-standard reliability, and the choice of Spearman correlation. No new physical or formal entities are introduced.

free parameters (3)
  • Minimum ground-truth image threshold = 4
    Wikimedia country/category entries with fewer than four images are excluded to form the ground-truth set (Sec. 3, Appendix B), shaping which artifacts enter the benchmark.
  • Seed counts per prompt = 20 (SDXL, SD1.5), 4 (others), 80 for P(c)
    Used to average scorer values; chosen by compute constraints, affects the variance of all reported scores (Appendix A.1).
  • Delta phi_PS offset = 0.5
    Added to make divergence scores comparable in scale; it is rank-invariant and does not change Spearman correlations, but it is an ad hoc constant (Sec. 5.3).
assumptions (5)
  • domain assumption Wikimedia category graph and the authors' manual shortlist of six cultural axes and 32 categories provide a valid operationalization of cultural artifacts and regions.
    The dataset is the substrate for every score and every human survey; Sec. 3 and Appendix B describe the selection.
  • domain assumption Geography, encoded as a single country of origin per artifact, is an adequate proxy for culture.
    The authors state this as a limitation; it affects worker matching and how region-specific scores are aggregated.
  • domain assumption Three Prolific workers per artifact, matched by country of nationality, produce reliable gold-standard Likert scores.
    All correlations are computed against these scores; Fig. 13 documents large worker disagreements and miscalibration.
  • standard math Spearman rank correlation against gold scores is the right measure of scorer quality.
    Used throughout Sec. 5.2-5.5; it ignores calibration and absolute error, so a scorer can rank well yet be systematically biased.
  • domain assumption The four prompt templates in Table 6 instantiate the attributes {n,c,r} equivalently across all artifacts.
    All scorer comparisons assume differences between scores come from the model's cultural knowledge rather than from prompt wording.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems." pith.science (2026). https://pith.science/paper/23E3R3G5

@misc{pith2026250608071,
  author       = {Pith},
  title        = {Pith review of: CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23E3R3G5}},
  note         = {Machine review of arXiv:2506.08071}
}
read the original abstract

Popular text-to-image (T2I) systems are trained on web-scraped data, which is heavily Amero and Euro-centric, underrepresenting the cultures of the Global South. To analyze these biases, we introduce CuRe, a novel and scalable benchmarking and scoring suite for cultural representativeness that leverages the marginal utility of attribute specification to T2I systems as a proxy for human judgments. Our CuRe benchmark dataset has a novel categorical hierarchy built from the crowdsourced Wikimedia knowledge graph, with 300 cultural artifacts across 32 cultural subcategories grouped into six broad cultural axes (food, art, fashion, architecture, celebrations, and people). Our dataset's categorical hierarchy enables CuRe scorers to evaluate T2I systems by analyzing their response to increasing the informativeness of text conditioning, enabling fine-grained cultural comparisons. We empirically observe much stronger correlations of our class of scorers to human judgments of perceptual similarity, image-text alignment, and cultural diversity across image encoders (SigLIP 2, AIMV2 and DINOv2), vision-language models (OpenCLIP, SigLIP 2, Gemini 2.0 Flash) and state-of-the-art text-to-image systems, including three variants of Stable Diffusion (1.5, XL, 3.5 Large), FLUX.1 [dev], Ideogram 2.0, and DALL-E 3. The code and dataset is open-sourced and available at https://aniketrege.github.io/cure/.

Figures

Figures reproduced from arXiv: 2506.08071 by the authors.

Figure 1
Figure 1. Three images of types of pottery generated by a state [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The CuRe dataset is constructed in a hierarchical fash￾ion (s → c → n, r) from Wikimedia with 300 cultural artifacts n ∈ N grouped into 6 cultural axes s ∈ S, 32 cultural categories c ∈ C, described by a name n and associated region r. For exam￾ple, s = “food”, c = “dumpling”, n = “banku”, r = “Ghana”. tion of objects, people, and language. These dataset biases propagate to T2I systems after training. Text-to-Image … view at source ↗
Figure 3
Figure 3. An overview of the scorers and user study of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: There is a large drop in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Computation of image-text alignment score [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: A qualitative comparisons of our proposed MIA scorers compared to baselines for three scorer classes: a) Perceptual Similarity [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Computation of diversity score ϕDIV on n = “modak”, a type of dumpling from India. ϕDIV computes an aggre￾gate pairwise dissimilarity (LPIPS [76]) between multiple seeds of images generated with varying attribute specification, i.e. I(n), I(n, c), I(n, r), I(n, c, r). …
Figure 9
Figure 9. Figure 9: Q1b: Querying users to rank (order) the similarity of the ground truth images to the AI generated image from highest (top) to lowest (bottom). Research Study on the Cultural Biases of Gener￾ative AI Thank you for taking the time for this survey. We are a team of resear…
Figure 8
Figure 8. Figure 8: Q1a: Querying users for perceptual similarity of T2I system generated image to ground-truth images. through Prolific, a large crowdsourcing tool. For each ar￾tifact n in the CuRe dataset, we hire three workers whose country of nationality match the region r of the arti…
Figure 10
Figure 10. Figure 10: User study interface for CuRe. Finally, the user is also asked to share a textual descrip￾tion of the specific details contributing to the accuracy or in￾accuracy of the T2I system output in Q3c. This fine-grained information is useful to identify details of why and h…
Figure 12
Figure 12. Figure 12: Demographic information questions [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Feedbacks of the workers in cases of high disagreement over perceptions of cultural representativeness [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 13
Figure 13. Figure 13: Feedbacks of the workers in cases of high disagreement over perceptions of cultural representativeness [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Responses to “Do you identify with the culture of [country]?” summed across all surveys belonging to each region [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: A histogram of worker familiarity with generative AI models like ChatGPT or DALL-E from 1 (“Never Heard of It”) to 5 [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Frequency estimation (log scale) for 50 cultural artifacts across each of the six supercategories (cultural axes) in the CuRe dataset on LAION-SD15, the fully open pretraining dataset of Stable Diffusion 1.5 [50]. E. Concept Frequency Estimation In recent times, state…
Figure 17
Figure 17. Figure 17: Scatter plots of Left: ϕGT (a) and Right: ϕP S(a) for 50 artifacts from all 6 supercategories in the CuRe dataset generated using FLUX.1 [dev]. Each data point represents a single cultural artifact, and is an average of the scores for each T2I generated seed. All imag…
Figure 18
Figure 18. Figure 18: Visualization of quantitative perceptual similarity (PS) scorers and user study [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]
Figure 19
Figure 19. Figure 19: Visualization of quantitative Image-Text Alignment (ITA) scorers, user study [PITH_FULL_IMAGE:figures/full_fig_p034_19.png]
Figure 20
Figure 20. Figure 20: A bar plot of \Delta ϕDIV across all artifacts in the CuRe dataset aggregated at the supercategory level (mean + std error bar) for each T2I system in our evaluation set. We separate each bar at the supercategory level into artifacts belonging to the Global North (GN)…
Figure 21
Figure 21. Figure 21: Diversity scorer and user feedback 37 [PITH_FULL_IMAGE:figures/full_fig_p037_21.png]
Figure 22
Figure 22. Figure 22: Visualization of Gemini 2.0 Flash responses where the MLLM either failed or succeeded in identifying nuances compared to [PITH_FULL_IMAGE:figures/full_fig_p040_22.png]
Figure 22
Figure 22. Figure 22: Continued from previous page. E# AI Image Real Image Feedback E5 Artifact: Takht-e Fulad Category:Monument/Memorial ϕ ∗CuRe : 3.67 Region: Iran Super Category: Architecture ϕGemini : 1 Human Judgments (User Study): 1. so the image has certain specification like carvin…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

135 extracted references · 66 canonical work pages

  1. [1]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision , pages 2425– 2433, 2015. 3

    2015

  2. [2]

    In- specting the geographical representativeness of images from text-to-image models

    Abhipsa Basu, R Venkatesh Babu, and Danish Pruthi. In- specting the geographical representativeness of images from text-to-image models. In ICCV, 2023. 2, 3, 6, 11, 12, 21

    2023

  3. [3]

    Diffusion Models Through a Global Lens: Are They Culturally Inclusive?

    Zahra Bayramli, Ayhan Suleymanzade, Na Min An, Huzama Ahmad, Eunsu Kim, Junyeong Park, James Thorne, and Al- ice Oh. Diffusion models through a global lens: Are they cul- turally inclusive? arXiv preprint arXiv:2502.08914, 2025. 2, 21

    work page Pith review arXiv 2025

  4. [4]

    Improving image generation with bet- 12 ter captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with bet- 12 ter captions. https://cdn.openai.com/papers/ dall-e-3.pdf, 2023. 1, 11

    2023

  5. [5]

    Easily acces- sible text-to-image generation amplifies demographic stereo- types at large scale

    Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily acces- sible text-to-image generation amplifies demographic stereo- types at large scale. InProceedings of the 2023 ACM Confer- ence on Fairness, Accountability, and Transparency , pages 1493–1...

    2023

  6. [6]

    Demystifying mmd gans

    Mikołaj Bi ´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. In International Conference on Learning Representations, 2018. 2, 3

    2018

  7. [7]

    Multimodal datasets: misogyny, pornography, and ma- lignant stereotypes

    Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahem- bwe. Multimodal datasets: misogyny, pornography, and ma- lignant stereotypes. arXiv preprint arXiv:2110.01963, 2021. 1, 3

    arXiv 2021

  8. [8]

    Into the laion’s den: Investigating hate in multi- modal datasets

    Abeba Birhane, Sanghyun Han, Vishnu Boddeti, Sasha Luc- cioni, et al. Into the laion’s den: Investigating hate in multi- modal datasets. Advances in Neural Information Processing Systems, 36, 2024. 1, 26

    2024

Show all 135 references
  1. [9]

    Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. In Forty- first International Conference on Machine Learning , 2024. 3

    2024

  2. [10]

    Pal: Sample-efficient person- alized reward modeling for pluralistic alignment

    Daiwei Chen, Yi Chen, Aniket Rege, Zhi Wang, and Ramya Korlakai Vinayak. Pal: Sample-efficient person- alized reward modeling for pluralistic alignment. In The Thirteenth International Conference on Learning Represen- tations, 2025. 9

    2025

  3. [11]

    Pali: A jointly-scaled multilingual language-image model

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergio- vanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. In The Eleventh International Conference on Learning Represent...

    2023

  4. [12]

    Dall-eval: Probing the reasoning skills and social biases of text-to- image generation models

    Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to- image generation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3043– 3054, 2023. 1

    2023

  5. [13]

    Food and culture

    Carole Counihan, Penny Van Esterik, et al. Food and culture. Routledge New York, NY , 2013. 3

    2013

  6. [14]

    Does object recognition work for ev- eryone? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages 52–59, 2019

    Terrance De Vries, Ishan Misra, Changhan Wang, and Lau- rens Van der Maaten. Does object recognition work for ev- eryone? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages 52–59, 2019. 2

    2019

  7. [15]

    Google DeepMind. gemini. https : / / deepmind . google / technologies / gemini / flash/, 2025. Accessed: March 5, 2025. 3, 8, 11, 38

    2025

  8. [16]

    Internet, phone, mail, and mixed-mode surveys: The tailored design method

    Don A Dillman, Jolene D Smyth, and Leah Melani Christian. Internet, phone, mail, and mixed-mode surveys: The tailored design method. Indianapolis, Indiana, 2014. 5

    2014

  9. [17]

    Cogview2: Faster and better text-to-image generation via hierarchical transformers

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers. Advances in Neural Information Processing Systems, 35:16890–16902, 2022. 3

    2022

  10. [18]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  11. [19]

    Data filtering networks

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander T Toshev, and Vaishaal Shankar. Data filtering networks. In The Twelfth International Conference on Learning Representations, 2024. 9, 33

    2024

  12. [20]

    Mul- timodal autoregressive pre-training of large vision encoders

    Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guil- herme Turrisi da Costa, Louis B´ethune, Zhe Gan, et al. Mul- timodal autoregressive pre-training of large vision encoders. arXiv preprint arXiv:2411.14402, 2024. ...

    2024 arXiv

  13. [21]

    The vendi score: A diversity evaluation metric for machine learning, 2023

    Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning, 2023. 7, 10, 35

    2023

  14. [22]

    Dat- acomp: In search of the next generation of multimodal datasets

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Dat- acomp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304.14108, 2023. 1, 9, 33

    2023 arXiv

  15. [23]

    Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness

    Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In Interna- tional conference on learning representations, 2018. 5, 7

    2018

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  17. [25]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 2, 3

    2017

  18. [26]

    Culture’s consequences: Comparing val- ues, behaviors, institutions and organizations across nations

    Geert Hofstede. Culture’s consequences: Comparing val- ues, behaviors, institutions and organizations across nations. Sage publications, 2001. 3

    2001

  19. [27]

    Visage: A global-scale analysis of visual stereo- types in text-to-image generation

    Akshita Jha, Vinodkumar Prabhakaran, Remi Denton, Sarah Laszlo, Shachi Dave, Rida Qadri, Chandan Reddy, and Sunipa Dev. Visage: A global-scale analysis of visual stereo- types in text-to-image generation. In Proceedings of the 62nd Annual Meeting of the Association for Computa...

  20. [28]

    Billion- scale similarity search with gpus

    Jeff Johnson, Matthijs Douze, and Herv ´e J ´egou. Billion- scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547, 2019. 6

    2019

  21. [29]

    Scal- ing up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scal- ing up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10124–10134, 2023. 3, 6 13

    2023

  22. [30]

    Beyond aesthetics: Cul- tural competence in text-to-image models

    Nithish Kannen, Arif Ahmad, marco Andreetto, Vinodkumar Prabhakaran, Utsav Prabhu, Adji Bousso Dieng, Pushpak Bhattacharyya, and Shachi Dave. Beyond aesthetics: Cul- tural competence in text-to-image models. In NeurIPS D&B Track, 2024. 2, 3, 6, 7, 10, 21, 35

    2024

  23. [31]

    An image speaks a thousand words, but can everyone listen? on image transcreation for cultural relevance

    Simran Khanuja, Sathyanarayanan Ramamoorthy, Yueqi Song, and Graham Neubig. An image speaks a thousand words, but can everyone listen? on image transcreation for cultural relevance. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , page...

    2024

  24. [32]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems , 36: 36652–36663, 2023. 9

    2023

  25. [33]

    Openimages: A public dataset for large-scale multi-label and multi-class im- age classification

    Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Ui- jlings, Stefan Popov, Andreas Veit, et al. Openimages: A public dataset for large-scale multi-label and multi-class im- age classification. Dataset available fr...

    2017

  26. [34]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. Advances in neural information processing systems , 25, 2012. 6

    2012

  27. [35]

    Research methods in human-computer interaction

    Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. Research methods in human-computer interaction . Morgan Kaufmann, 2017. 5

    2017

  28. [36]

    A technique for the measurement of attitudes

    Rensis Likert. A technique for the measurement of attitudes. Archives of psychology, 1932. 4, 5

    1932

  29. [37]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  30. [38]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. 3

    2023

  31. [39]

    Towards equitable representation in text-to-image synthesis models with the cross-cultural un- derstanding benchmark (ccub) dataset

    Zhixuan Liu, Youeun Shin, Beverley-Claire Okogwu, Youngsik Yun, Lia Coleman, Peter Schaldenbrand, Jihie Kim, and Jean Oh. Towards equitable representation in text-to-image synthesis models with the cross-cultural un- derstanding benchmark (ccub) dataset. arXiv preprint arXiv:2...

    2023 arXiv

  32. [40]

    Stable bias: Analyzing so- cietal representations in diffusion models

    Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Analyzing so- cietal representations in diffusion models. arXiv preprint arXiv:2303.11408, 2023. 3, 6

    2023 arXiv

  33. [41]

    Social biases through the text-to-image generation lens

    Ranjita Naik and Besmira Nushi. Social biases through the text-to-image generation lens. arXiv preprint arXiv:2304.06034, 2023. 3

    2023 arXiv

  34. [42]

    OpenAI. o3-mini. https://openai.com/index/ openai- o3- mini/, 2025. Accessed: March 5, 2025. 9, 33

    2025

  35. [43]

    Dinov2: Learning robust visual features without super- vision

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without super- vision. Transactions on Machine Learning Research,...

    2024

  36. [44]

    The neglected tails in vision-language models

    Shubham Parashar, Zhiqiu Lin, Tian Liu, Xiangjue Dong, Yanan Li, Deva Ramanan, James Caverlee, and Shu Kong. The neglected tails in vision-language models. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12988–12997, 2024. 1, 3, 26, 35

    2024

  37. [45]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth Inter- national Conference on Learning Representations, 2024. 1, 6

    2024

  38. [46]

    Prolific

    Prolific. Prolific. https://www.prolific.com, 2014. Accessed: March 5, 2025. 3

    2014

  39. [47]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  40. [48]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1, 3

    2022 arXiv

  41. [49]

    Red-teaming the stable diffusion safety filter

    Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tram`er. Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610, 2022. 3

    2022 arXiv

  42. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 6, 26

    2022

  43. [51]

    Cvqa: Culturally-diverse multilingual visual question answering benchmark

    David Romero, Chenyang Lyu, Haryo Wibowo, San- tiago G ´ongora, Aishik Mandal, Sukannya Purkayastha, Jesus-German Ortiz-Barajas, Emilio Cueva, Jinheon Baek, Soyeong Jeong, et al. Cvqa: Culturally-diverse multilingual visual question answering benchmark. Advances in Neural Info...

    2025

  44. [52]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211–252, 2015. 2, 6

    2015

  45. [53]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information...

    2022

  46. [54]

    Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis

    Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila. Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis. arXiv preprint arXiv:2301.09515, 2023. 3 14

    2023 arXiv

  47. [55]

    LAION-400M: Open dataset of CLIP-filtered 400 million image-text pairs

    Cristoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Jenia Jitsev, and Aran Komatsuzaki. LAION-400M: Open dataset of CLIP-filtered 400 million image-text pairs. In Proceedings of Neurips Data-Centric AI Workshop, 2021. 1, 6

    2021

  48. [56]

    LAION-5b: An open large-scale dataset for train- ing next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev....

    2022

  49. [57]

    The bias amplification paradox in text-to-image generation

    Preethi Seshadri, Sameer Singh, and Yanai Elazar. The bias amplification paradox in text-to-image generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Paper...

    2024

  50. [58]

    Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. No classification without rep- resentation: Assessing geodiversity issues in open data sets for the developing world. In NIPS 2017 workshop: Machine Learning for the Developing World, 2017. 2

    2017

  51. [59]

    Identifying and eliminating csam in generative ml training data and models

    David Thiel. Identifying and eliminating csam in generative ml training data and models. Stanford Internet Observatory, Cyber Policy Center, December, 23:3, 2023. 26

    2023

  52. [60]

    Yfcc100m: The new data in multimedia research

    Bart Thomee, David A Shamma, Gerald Friedland, Ben- jamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2):64–73, 2016. 2

    2016

  53. [61]

    Metamorph: Multimodal understanding and generation via instruction tuning

    Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412.14164, 2024. 3

    2024 arXiv

  54. [62]

    All groups composi- tions

    UN Trade and Development. All groups composi- tions. https : / / unctadstat . unctad . org / EN / Classifications / DimCountries _ All _ Hierarchy.pdf, 2025. Accessed: 2025-03-21. 17

    2025

  55. [63]

    Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, and dense features

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muham- mad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, ...

    2025 arXiv

  56. [64]

    How ai reduces the world to stereotypes

    Victoria Turk. How ai reduces the world to stereotypes. https://restofworld.org/2023/ai- image- stereotypes/, 2023. Accessed: 2024-09-07. 3

    2023

  57. [65]

    Navigating cultural chasms: Exploring and unlock- ing the cultural pov of text-to-image models

    Mor Ventura, Eyal Ben-David, Anna Korhonen, and Roi Re- ichart. Navigating cultural chasms: Exploring and unlock- ing the cultural pov of text-to-image models. arXiv preprint arXiv:2310.01929, 2023. 2, 3, 7, 9, 21, 33, 34

    2023 arXiv

  58. [66]

    The factuality tax of diversity-intervened text-to-image genera- tion: Benchmark and fact-augmented intervention

    Yixin Wan, Di Wu, Haoran Wang, and Kai-Wei Chang. The factuality tax of diversity-intervened text-to-image genera- tion: Benchmark and fact-augmented intervention. In Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9082–9100, 202...

    2024

  59. [67]

    Revise: A tool for measuring and miti- gating bias in visual datasets

    Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and miti- gating bias in visual datasets. International Journal of Com- puter Vision, 130(7):1790–1810, 2022. 2

    2022

  60. [68]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 3

    2024 arXiv

  61. [69]

    Wikimedia Commons

    Wikimedia. Wikimedia Commons. https://commons. wikimedia.org/, 2004. Accessed: 2025-03-06. 2, 3, 17

    2004

  62. [70]

    American== white in mul- timodal language-and-image ai

    Robert Wolfe and Aylin Caliskan. American== white in mul- timodal language-and-image ai. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society , pages 800–812, 2022. 3

    2022

  63. [71]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. CoRR, 2023. 7, 9, 10

    2023

  64. [72]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36:15903–15935, 2023. 9

    2023

  65. [73]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2022. 3, 6

    2022 arXiv

  66. [74]

    Scaling autoregressive multi- modal models: Pretraining and instruction tuning

    Lili Yu, Bowen Shi, Ramakanth Pasunuru, Benjamin Muller, Olga Golovneva, Tianlu Wang, Arun Babu, Binh Tang, Brian Karrer, Shelly Sheynin, et al. Scaling autoregressive multi- modal models: Pretraining and instruction tuning. arXiv preprint arXiv:2309.02591, 2023. 3

    2023 arXiv

  67. [75]

    Partiality and miscon- ception: Investigating cultural representativeness in text-to- image models

    Lili Zhang, Xi Liao, Zaijia Yang, Baihang Gao, Chunjie Wang, Qiuling Yang, and Deshun Li. Partiality and miscon- ception: Investigating cultural representativeness in text-to- image models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages ...

    2024

  68. [76]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 7, 35 15 Contents

    2018

  69. [77]

    Finding a CuRe through Information

    Measuring CuRe 4 4.1. Finding a CuRe through Information . . . . 5 4.2. Perceptual Similarity Scorers . . . . . . . . 5 4.3. Image-Text Alignment Scorers . . . . . . . . 6 4.4. Diversity Scorers . . . . . . . . . . . . . . . 6

  70. [78]

    User Study

    Experiments 7 5.1. User Study . . . . . . . . . . . . . . . . . . 7 5.2. Scorer Correlation to Human Judgments . . . 8 5.3. Perceptual Similarity . . . . . . . . . . . . . 8 5.4. Image-Text Alignment . . . . . . . . . . . . 9 5.5. Diversity . . . . . . . . . . . . . . . . . . ...

  71. [79]

    [category name] by country

    Acknowledgements 12 A . T2I Inference Details 16 A.1 . Seeding . . . . . . . . . . . . . . . . . . . . 16 A.2 . Safety Filter Refusal . . . . . . . . . . . . . 16 B . Dataset Design 17 C . User Study Design 17 C.1. Disclosure . . . . . . . . . . . . . . . . . . 18 C.2. Percept...

  72. [80]

    Disclosure (Appendix C.1)

  73. [81]

    Perceptual Similarity (Appendix C.2)

  74. [82]

    Artifact Familiarity Questionnaire (Appendix C.3)

  75. [83]

    CuRe: Cultural Representativeness (Appendix C.4)

  76. [84]

    Offensiveness and Stereotypes (Appendix C.5)

  77. [85]

    How similar do you think the generated image on the left is to the real images on the right?

    User Metadata (Appendix C.6) C.1. Disclosure We inform the workers of the survey goals, how their data will be used, and how they can withdraw their consent later if they choose to do so. They are asked for explicit and informed consent for their data to be used, and provided ...

  78. [86]

    stereotypical

    Not at all to 5. A lot. For offensiveness, we query work- ers zero-shot (i.e. no rubric or examples): “Images created by GenAI models can be offensive or harmful, and this may vary from person to person. In your opinion, does the above image of [artifact name] seem offensive o...

  79. [87]

    some of the small blue domes are futher apart than others

    The trees used looks so weird as its mimicing a dome shape. some of the small blue domes are futher apart than others. asymtrical look

  80. [88]

    There is a lot of Dom’s typicaly there is only one in any building, this picture represent a mosque more than a normal building, also the Dom’s can’t be made from trees ϕ∗CuRe= 2.0/5 ϕGT(n) = 0.654 ϕP S(n) = 0.580 ∆ϕP S(n, c) = 0.552 ∆ϕP S(n, c, r) = 0.506 E2 Moai

  81. [89]

    the head to body ratio is a bit off, the body should either be more elongated or the head bigger, some even had some kind of red hat and they all looked to the ocean

  82. [90]

    ϕ∗CuRe= 4.0/5 ϕGT(n) = 0.808 ϕP S(n) = 0.581 ∆ϕP S(n, c) = 0.486 ∆ϕP S(n, c, r) = 0.486 E3 George Lucas

    The main thing that I think is inaccurate is that the moai in the image looks too perfect and the real ones have a lot of damage due to nature. ϕ∗CuRe= 4.0/5 ϕGT(n) = 0.808 ϕP S(n) = 0.581 ∆ϕP S(n, c) = 0.486 ∆ϕP S(n, c, r) = 0.486 E3 George Lucas

  83. [91]

    There’s nothing particularly unique about it that would NOT make it apart of my culture

    This is a closeup picture of George Lucas’ face. There’s nothing particularly unique about it that would NOT make it apart of my culture. I don’t see why I couldn’t see this kind of photo in my culture

  84. [92]

    They are obviously AI

    I do not see anything that is ”wrong” other than the eyes. They are obviously AI. ϕ∗CuRe= 5.0/5 ϕGT(n) = 0.745 ϕP S(n) = 0.571 ∆ϕP S(n, c) = 0.489 ∆ϕP S(n, c, r) = 0.460 E4 Tallar´ın saltado

  85. [93]

    the noodles look like worms, tallar´ın saltado is spaghetti with meat and vegetables whereas the image only shows weird pasta and no meat nor onions/tomatoes

  86. [94]

    The image doesn’t look like noodles to me, it seems like a kind of vegetable so I can’t say is accurate to the real Tallar´ın saltado ϕ∗CuRe= 1.0/5 ϕGT(n) = 0.654 ϕP S(n) = 0.696 ∆ϕP S(n, c) = 0.486 ∆ϕP S(n, c, r) = 0.425 E5 Blocos carnavalescos de S˜ao Paulo

  87. [95]

    The costumes are indeed carnival like, but the blocos are less stravagant and the costumes wore by the people usualy contain some kind of a joke on a very low budget costume

  88. [96]

    ϕ∗CuRe= 1.0/5 ϕGT(n) = 0.712 ϕP S(n) = 0.757 ∆ϕP S(n, c) = 0.504 ∆ϕP S(n, c, r) = 0.473 E6 Hmong textile art

    Their physical appearance and posture. ϕ∗CuRe= 1.0/5 ϕGT(n) = 0.712 ϕP S(n) = 0.757 ∆ϕP S(n, c) = 0.504 ∆ϕP S(n, c, r) = 0.473 E6 Hmong textile art

  89. [97]

    The design motif and colors are quite close to the real ones

  90. [98]

    An image from Australia

    I think the repetitive patterns and the colourful decoration make it accurate ϕ∗CuRe= 5.0/5 ϕGT(n) = 0.620 ϕP S(n) = 0.634 ∆ϕP S(n, c) = 0.519 ∆ϕP S(n, c, r) = 0.518 Figure 18. Visualization of quantitative perceptual similarity (PS) scorers and user study CuRe scores and text...

  91. [99]

    Bangles are circular ornaments with some colors and patterns on them

  92. [100]

    [31]= 0.060 Ventura et al

    The width and the carvings and the colors make it seem more aligned to my culture ϕ∗CuRe= 5.0/5 Khanuja et al. [31]= 0.060 Ventura et al. [65]= 0.045 ϕIT A(c, r) = 0.124 E2 Festival de la Primavera

  93. [101]

    The festival takes place in the cost of Lima, no there are not a lot of hilly areas, and it’s more a dry environment so those flowers and the place do not correspond at all to Trujillo reality

  94. [102]

    But the main event of the Festival de la Primavera is the parade (People, carriages and flowers)

    The image represents the Spring in general. But the main event of the Festival de la Primavera is the parade (People, carriages and flowers). ϕ∗CuRe= 1.0/5 Khanuja et al. [31]= 0.086 Ventura et al. [65]= 0.067 ϕIT A(c, r) = 0.156 E3 Jalangkote

  95. [103]

    a complete different genre

    The image use the wrong thing, AI image is using stone like building while Jalangkote is a food. a complete different genre

  96. [104]

    ϕ∗CuRe= 1.7/5 Khanuja et al

    The AI image is not even a food. ϕ∗CuRe= 1.7/5 Khanuja et al. [31]= 0.112 Ventura et al. [65]= 0.093 ϕIT A(c, r) = 0.022 E4 Puchner Mansion

  97. [105]

    it looks more like something an american would think of as a castle

    It’s style is just all wrong. it looks more like something an american would think of as a castle. can we stop trying to make ai image gen happen?

  98. [106]

    ϕ∗CuRe= 1.3/5 Khanuja et al

    I feel like the AI focused too much on the word Mansion, and especially the american/older british kind. ϕ∗CuRe= 1.3/5 Khanuja et al. [31]= 0.040 Ventura et al. [65]= 0.053 ϕIT A(c, r) = 0.096 E5 Zw¨olf Glaubensartikel

  99. [107]

    Overall the vegitation seems possible to find in my country

  100. [108]

    ϕ∗CuRe= 4.7/5 Khanuja et al

    This image could be a shot from above of some forest in my country. ϕ∗CuRe= 4.7/5 Khanuja et al. [31]= 0.108 Ventura et al. [65]= 0.091 ϕIT A(c, r) = 0.079 E6 Michelle Bachelet

  101. [109]

    She’s a well known Chilean politician, so a lot of people recognize her face

    The image doesn’t have the exact facial features of Michelle Bachelet. She’s a well known Chilean politician, so a lot of people recognize her face. The image looks too fat and the hair is wrong

  102. [110]

    Banku” artifact in Fig. 3). Intuitively, this evaluates how much diversity changes when we mix images of “Banku

    Its the features of the face, they are not completly wrong, but together they make a face who cant be taken to be Bachelet, also the hair looks really fake. ϕ∗CuRe= 2.0/5 Khanuja et al. [31]= 0.066 Ventura et al. [65]= 0.054 ϕIT A(c, r) = 0.164 Figure 19. Visualization of quan...

  103. [111]

    Although, the wear and tear as well as the shape of the statue being mummy-like would be found in my country

    This image has used a different facial feature of the statue you would not see in my country of culture. Although, the wear and tear as well as the shape of the statue being mummy-like would be found in my country

  104. [112]

    The material it is made of and the type of cloth it seems to be wearing

    The similarities lies in the cat structure generally. The material it is made of and the type of cloth it seems to be wearing. The part where it is different from my culture is that the eyes are looking at the side which is very unlikely. Moreover, the smile itself is so not t...

  105. [113]

    May need to add shrimps

    Should have less liquid sauce. May need to add shrimps. The noodle should be fried

  106. [114]

    Each individual strand can be traced which seems unlikely in a real noodle image ϕ∗CuRe= 4/5 LP IP S(n) = 0.70 ϕDIV= 0.59 E3 Third Mainland Bridge

    The noodles in the AI image looks weird and too smooth/plump compared to real noodle dishes. Each individual strand can be traced which seems unlikely in a real noodle image ϕ∗CuRe= 4/5 LP IP S(n) = 0.70 ϕDIV= 0.59 E3 Third Mainland Bridge

  107. [115]

    The image contains the distinctive part of the real location and image, it contains most of the details of the real thing

  108. [116]

    The image shows more or fewer lanes than the actual four lanes

    The inaccurate details is The Number of Lanes. The image shows more or fewer lanes than the actual four lanes. The image inaccurately represents the bridge structural elements such. ϕ∗CuRe= 4.5/5 LP IP S(n) = 0.50 ϕDIV= 0.50 E4 Rabat Lighthouse

  109. [117]

    The catholic or christian cross

  110. [118]

    I think gen AI did a good job here, but missed some details like the big square in front of the lighthouse and the window’s sizes are somewhat big here ϕ∗CuRe= 5/5 LP IP S(n) = 0.51 ϕDIV= 0.54 E5 Penelope Cruz

  111. [119]

    The facial features are very similar to the real ones

  112. [120]

    The white clothes is typical mediterranean

    The hair is typical of any country, so it could as well be spanish. The white clothes is typical mediterranean. ϕ∗CuRe= 4/5 LP IP S(n) = 0.74 ϕDIV= 0.62 E6 Vaso de los Guerrero

  113. [121]

    The AI-generated image looks too unreal; it doesn’t feel natural, as if it was too exaggerated

  114. [122]

    The decorations are in relief and not painted

    The image has floral filigrees and not warriors. The decorations are in relief and not painted. In addition, the vessel is lacquered with gloss ϕ∗CuRe= 2/5 LP IP S(n) = 0.75 ϕDIV= 0.55 Figure 21. Diversity scorer and user feedback 37 I. MLLM as a Judge To our knowledge for the...

  115. [123]

    On a scale of 1-5, how similar do you think the generated image on the left is to the real images on the right? Use the following scale: 1: Not at all similar 2: Slightly similar 3: Reasonably Similar 4: Very Similar 5: Extremely Similar

  116. [124]

    Not at all similar

    Provide a brief explanation for your similarity rating, detailing specific visual aspects you com- pared. Format your response strictly as JSON with the keys ”similarity rating” (for the number 1-5) and ”simi- larity explanation” (for the text explanation). Example Response fr...

  117. [125]

    Cultural Representativeness: On a scale of 1- 5 (where 1 is ”Highly Unlikely” and 5 is ”Ex- tremely Likely”), how likely can this item be found in {country}?

  118. [126]

    Ground-Truth Likelihood: On a scale of 1-5 (where 1 is ”Highly Unlikely” and 5 is ”Extremely Likely”), how likely is this an accurate image of {item name}?

  119. [127]

    AI Image

    Description: What specific details in the AI- generated image make it accurate or inaccurate compared to how this object typically appears in {country} or your understanding of {item name}? Format your response as JSON with keys: ’country likelihood’, ’item accuracy’, and ’de-...

  120. [128]

    The image seems inaccurate from the images of Kuti I used to see.3

    The image is someone holding a microphone but that is not femi kuti2. The image seems inaccurate from the images of Kuti I used to see.3. The face is not just right with with me and the hair is just off....Femi has very distinctive blend of grey hair. MLLM Judgment (Gemini 2.0...

  121. [129]

    The rice and the colour of the rice after been cooked makes it look like jollof from my culture.3

    The image gets the color of Jollof rice2. The rice and the colour of the rice after been cooked makes it look like jollof from my culture.3. The image has used the correct kind of rice, ingredients, and sauce, in my country MLLM Judgment (Gemini 2.0 Flash): The item can almost...

  122. [130]

    It’d be like, let’s say we are trying to find certain colored horses, but perhaps AIgenerates the horses with patterns of color never found in nature.2

    I feel like it’s close to getting it right, but the patterns on the waist and sleeves, for example, makes it look like almostwe’d never see something like this. It’d be like, let’s say we are trying to find certain colored horses, but perhaps AIgenerates the horses with patter...

  123. [131]

    The image has used an outfit that other indigenous people from northern hemisphere might use, but based on myexperience it doesn’t look extremely similar to Finnish Sami headwear

    The person wearing the hat doesn’t look Northern European.2. The image has used an outfit that other indigenous people from northern hemisphere might use, but based on myexperience it doesn’t look extremely similar to Finnish Sami headwear. The fabric texture and colors are di...

  124. [132]

    The coloring is different and Isfahanian buildings have different sets of colors and style more colorful.3

    so the image has certain specification like carvings are quite similar, the stone types also similar to the one I sawin my country, however I think this image is closer to Takhte-Jamshid, as it does not have blue ceramic which usuallybe used in Esfahan an is one of the icons o...

  125. [133]

    The beard needsimprovement as each era in Iranian culture, men used to have different hair and beard styles

    The only thing similar to Amir Kabir AI to the real life is the facial structures like nose and eyes. The beard needsimprovement as each era in Iranian culture, men used to have different hair and beard styles. The cone shaped hat isimportant in displaying who this person was ...

  126. [134]

    The image shown was a type of an animal which is obviously so far from ”Kiping” which is a type of food.3

    The image does not portray the meaning of Kiping at all.2. The image shown was a type of an animal which is obviously so far from ”Kiping” which is a type of food.3. The image is an animal holding a spray bottle, it has no connection to kiping. MLLM Judgment (Gemini 2.0 Flash)...

  127. [135]

    The image is very similar to the name of Pulseira.3

    This is exactly how a pulseira is, a piece of jewellery that goes on the wrist2. The image is very similar to the name of Pulseira.3. This bracelet represents one of the forms of objects that we wear on our wrist, very beautiful with details in shiny stones. MLLM Judgment (Gem...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.