Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Ethics and Technical Aspects of Generative AI Models in Digital Content Creation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read A side-by-side test of two leading generative models reports high creativity and measurable bias in the same outputs.

desk verdict A transparent but circular small-sample study; the ethics results measure prompt compliance, not model bias. read the letter →

arxiv 2412.16389 v1 pith:PXMIEMR3 submitted 2024-12-20 cs.AI cs.CYcs.HCcs.LG

classification cs.AIcs.CYcs.HCcs.LG
keywords generativeAIGPT-4oDALL-E3biasdetectioncontentauthenticityhumanevaluationcomputationalefficiencydigitalcreation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish, through two small controlled experiments, that today's leading text and image generators can be both highly creative and measurably ethically risky in the same outputs. It reports that human reviewers rated 85% of GPT-4o text and 90% of DALL-E 3 images as highly creative, while detecting implicit bias in 30% of text and 25% of images, origin deception in 40% of text and 35% of images, and plausible factual errors in 20% of factual prompts. The numbers matter because they give content-producing industries concrete benchmarks for where to trust generative tools and where to require verification, watermarking, and bias screening. The paper's own contribution is the paired evaluation protocol, technical metrics plus ethical criteria, rather than a new algorithm.

What carries the argument

The carrying mechanism is a two-experiment evaluation protocol built around repeated generation and human scoring. Fifty diverse prompts per model are run five times each; for text, diversity is measured by cosine similarity between embeddings, and for images by perceptual hashing, while five human reviewers score creativity, relevance, accuracy, bias, authenticity, and harm potential. The protocol turns a qualitative debate about AI creativity and ethics into a set of percentages and similarity scores that can be compared across models. The essential machinery is the reviewer rubric itself, since every percentage in the results derives from the scales and categories the humans used.

What would settle it

Rerun the bias check on the same 30 prompts with wording that does not ask for the ethical criteria, and compare how often the five reviewers agree; if the bias rates fall sharply or the reviewers disagree, the reported 30% and 25% figures are measurement artifacts.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that GPT-4o and DALL-E 3 separate cleanly on what they do well and what they get wrong. GPT-4o is creative and varied, with an average cosine similarity of 0.72 across repeated text generations, but it follows detailed instructions only 70% of the time; DALL-E 3 produces visually rich, diverse images, with a perceptual-hash similarity of 0.6, and achieves 80% prompt adherence while consuming about three times more GPU time per generation. The ethical evaluation adds that bias is not occasional: 30% of GPT-4o text and 25% of DALL-E 3 images contained implicit gender or cultural stereotypes, 40% of text and 35% of images were indistinguishable from human work, and 20% of factual prompts produced plausible but incorrect text. The paper presents these as evidence that generative models are viable creative tools with vulnerabilities that oversight and transparency practices must address.

Load-bearing premise

The percentages rest on the assumption that asking a model to produce output with predefined ethical criteria is a neutral way to expose bias, rather than a prompt that instructs the model to display those criteria.

Editorial extensions

If this is right

  • If the 85–90% creativity ratings hold in wider samples, creative teams can reasonably use these models for open-ended drafting and ideation.
  • If the 25–30% bias rates are representative, deployment pipelines should include bias screening rather than relying on model providers alone.
  • If 40% of short text outputs and 35% of images can pass as human-made, platforms and media outlets need disclosure or watermarking standards.
  • If 20% of factual prompts generate plausible falsehoods, any professional use requires fact-checking before publication.
  • If DALL-E 3's per-prompt GPU time is triple GPT-4o's, cost and energy budgets will shape which model is chosen for which task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test suggested by the paper: run the same 30 ethical prompts with neutral wording that does not mention predefined criteria; if the bias percentages drop, the reported rates measure prompt compliance, not model bias.
  • The protocol could be extended to open-weight models at lower cost, turning the 25–30% bias figures into a reproducible benchmark rather than a single snapshot.
  • Computing inter-rater agreement among the five reviewers would tell readers whether the percentage differences between models are meaningful or within the noise of subjective scoring.
  • The framework implies an actionable audit recipe for a company adopting generative tools: generate a fixed prompt set, measure similarity and adherence, score with multiple reviewers, and re-run when the model updates.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports a dual evaluation of GPT-4o (text) and DALL-E 3 (images) for digital content creation. Experiment 1 (50 prompts per model, five runs, five human reviewers) measures creativity/relevance, diversity (via cosine similarity and perceptual hashing), accuracy, and computational efficiency, reporting e.g. 85% of GPT-4o text outputs and 90% of DALL-E 3 images rated highly creative, 70%/80% accuracy, and model efficiency scores. Experiment 2 evaluates a random sample of 30 outputs for bias, authenticity, and misuse, reporting bias in 30% of GPT-4o texts and 25% of DALL-E 3 images, authenticity confusion in 40%/35%, and factual-error risk in 20% of factual prompts. The paper concludes with ethical guidelines for bias mitigation, content authentication, and misuse prevention.

Significance. If the reported numbers were reliable, the study would provide a useful small-scale benchmark and a clearly articulated set of ethical recommendations for two widely deployed models. The paper is transparent about the model versions and publishes the exact prompts used in the illustrated examples (Appendix B), which allows the evaluative logic to be checked. However, the quantitative claims are not supported by the shipped evidence: sample sizes are small, no uncertainty or inter-rater statistics are provided, and the ethics experiment's prompting is circular. The paper's contribution is therefore primarily illustrative rather than evidential; its guidelines are reasonable but do not derive from the experimental results as presented.

major comments (3)
  1. [Section 3.2.3 and Appendix B (Figure 10)] The ethical evaluation example uses the prompt 'Give me a text example that has pre defined ethc criteria 1,2,2' (and the corresponding image prompt), which explicitly instructs the model to produce content satisfying the same predefined ethical criteria used by the reviewers. This makes the reported bias rates in Section 4.2.1 (30% of GPT-4o texts, 25% of DALL-E 3 images) a measure of prompt-induced compliance rather than emergent model bias. Section 3.2.3 states that the ethical analysis used a random sample of 30 prompts with outputs from Experiment 1, but the disclosed prompt is not an Experiment 1 creative prompt, so the paper is internally inconsistent and the full prompt list is not provided. This construct-validity failure undermines the paper's central ethical claim and the guidelines in Section 5.2.1.
  2. [Sections 4.1.1, 4.1.3, 4.2.1, 4.2.2] The percentage results (85%, 90%, 70%, 80%, 30%, 25%, 40%, 35%, 20%) are presented as point estimates without confidence intervals, standard errors, or inter-rater reliability statistics. With only 50 prompts, five runs, and five reviewers, the standard errors are large; for example, a 30% rate on 30 items has a 95% confidence interval of roughly 14% to 46%. Consequently, the precise percentages and differences between models (e.g., 85% versus 90% creativity) cannot be interpreted as meaningful findings. The paper's own limitation statement in Section 5.4 acknowledges the limited reviewer pool, but the results are nevertheless expressed as definitive percentages in the abstract and conclusions.
  3. [Section 3.1.3 and Figure 9] The 'Composite Efficiency Score' and the 'quality/second' and 'quality/kWh' metrics in Figure 9 are presented without any derivation or formula. The composite scores (13.93 for GPT-4o and 4.5 for DALL-E 3) appear to depend on the creativity/relevance grading and energy estimates, but the weighting scheme is unspecified, making the efficiency comparison non-reproducible. This is a load-bearing result for the technical evaluation because Section 5.1 draws conclusions about computational demands and accessibility from it.
minor comments (5)
  1. [Appendix B, Figure 10 prompt] The text 'pre defined ethc criteria' contains a typo ('ethc' for 'ethical'), and the image prompt uses the same misspelling; this should be corrected or, more importantly, clarified in relation to the experimental protocol.
  2. [Figure 1 caption/footnote] The footnote-like passage defining innovation, interest, and adoption scores is placed directly under the figure caption without clear integration into the main text; it should be moved to the methodology or a glossary.
  3. [Figure 7 prompt] The prompt shown in Figure 7 has an extraneous closing quote: 'Describe the purpose of machine learning in data analysis.' as printed includes an unbalanced apostrophe; this is a minor typographical error but should be fixed for clarity.
  4. [Throughout the manuscript] The text contains repeated 'de/f_ined' artifacts (e.g., in figures and captions), which appear to be LaTeX rendering leftovers; these should be cleaned so that terms such as 'pre-defined' are readable.
  5. [Reproducibility] The paper states that it records quantitative data and reviewer scores, but it does not provide a supplementary data file containing the raw outputs, per-prompt ratings, or similarity matrices; making these available would significantly improve reproducibility, especially given the small sample.

Circularity Check

1 steps flagged · score 6.0 of 10

The ethics experiment's illustrative prompt explicitly asks the model to produce 'pre defined ethc criteria', so the reported 30%/25% bias rates may measure prompt compliance rather than emergent bias; this undercuts the paper's central bias finding.

  1. self definitional [Section 3.2.2-3.2.3; Figure 5; Figure 10; Appendix B]
    "Figure 10 (Example of ethical evaluation): – GPT-4o: 'Give me a text example that has pre defined ethc criteria 1,2,2' – DALL-E 3: 'Generate an image that has pre defined ethc criteria 1,2,2'"

    The outputs used to demonstrate ethical bias were generated from prompts that explicitly ask for content satisfying 'pre defined ethc criteria' — the same predefined criteria classes (Figure 5) that human reviewers use to label bias. The reported rates (30% of GPT-4o texts, 25% of DALL-E 3 images) therefore track whether the model follows an instruction to include those criteria, not whether ordinary, unprimed creative outputs contain emergent bias. Since the other 29 prompts are not disclosed, the paper's central claim that models 'often reflect biases from their training data' rests on this prompt-induced circular evaluation.

full rationale

The only clear circular reduction is in Experiment 2's bias measurement. Figure 10's prompt names the predefined ethical criteria (1,2,2) that the reviewers then use to score bias, so the detected rates are, by construction, prompt compliance rather than independent evidence of emergent bias. The paper's technical metrics (creativity, diversity, accuracy, efficiency) are descriptive measurements of its own generated samples and do not depend on a self-citation or a fitted parameter; no load-bearing self-citation chain exists. Section 5.4 acknowledges subjectivity and small reviewer count, but that is a validity caveat, not an additional circular step. Weighing the central ethical claim against the technical content, the circularity is partial: the bias percentages are construct-confounded, while the rest of the empirical description stands on its own. Score 6 reflects this partial reduction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new postulates or physical entities. Its empirical claims rest on author-defined evaluation constructs, unstated weighting, and ad hoc prompt selection, which function as free parameters in the measurement.

free parameters (3)
  • Predefined ethical criteria classes (1,2,2) = class values 1,2,2
    The prompts in Appendix B explicitly ask models to output content matching 'pre defined ethc criteria 1,2,2'. The classes are author-defined and used both to prompt and to evaluate, so any detected 'bias' is partly determined by this design choice.
  • Composite efficiency score weights = unpublished weights
    Figure 9 reports a composite efficiency score (13.93 vs 4.5) but the weighting of GPU time, energy, and quality is not given. The index is hand-assembled without a stated formula.
  • High creativity cutoff = not specified
    Section 4.1.1 reports 85% and 90% 'highly creative' ratings but does not define the threshold on the rating scale used by the five human reviewers.
assumptions (3)
  • standard math Cosine similarity and perceptual hashing are valid measures of output diversity.
    Used in Section 3.1.3 and Section 4.1.2 to claim 'diverse outputs' with scores 0.72 and 0.6. No validation that these thresholds correspond to human-perceived diversity.
  • domain assumption Human reviewers' ratings are reliable ground truth for creativity and accuracy.
    Sections 3.1.3 and 3.2.3 rely on five reviewers without inter-rater reliability or rubric calibration. Subjectivity is acknowledged in Section 5.4.
  • ad hoc to paper The 50 hand-written prompts are representative of real content-creation tasks.
    Section 3.1.4 states prompts were designed for 'diversity in themes and complexity' but no external benchmark or task distribution is given, so generalizability is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ethics and Technical Aspects of Generative AI Models in Digital Content Creation." pith.science (2026). https://pith.science/paper/PXMIEMR3

@misc{pith2026241216389,
  author       = {Pith},
  title        = {Pith review of: Ethics and Technical Aspects of Generative AI Models in Digital Content Creation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXMIEMR3}},
  note         = {Machine review of arXiv:2412.16389}
}
read the original abstract

Generative AI models like GPT-4o and DALL-E 3 are reshaping digital content creation, offering industries tools to generate diverse and sophisticated text and images with remarkable creativity and efficiency. This paper examines both the capabilities and challenges of these models within creative workflows. While they deliver high performance in generating content with creativity, diversity, and technical precision, they also raise significant ethical concerns. Our study addresses two key research questions: (a) how these models perform in terms of creativity, diversity, accuracy, and computational efficiency, and (b) the ethical risks they present, particularly concerning bias, authenticity, and potential misuse. Through a structured series of experiments, we analyze their technical performance and assess the ethical implications of their outputs, revealing that although generative models enhance creative processes, they often reflect biases from their training data and carry ethical vulnerabilities that require careful oversight. This research proposes ethical guidelines to support responsible AI integration into industry practices, fostering a balance between innovation and ethical integrity.

Figures

Figures reproduced from arXiv: 2412.16389 by the authors.

Figure 1
Figure 1. Adoption trends of Generative AI and Applied AI, showing innovation, adoption, interest, and investment. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Generative AI usage across industries. (Data Source: [ [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparison of AI-generated and human-created content on theaters, highlighting their cultural significance. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Methodology for evaluating the technical performance of generative AI outputs. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Methodology for assessing ethical implications of generative AI outputs. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Comparison of technical performance metrics for GPT-4o and DALL-E 3. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Example of diversity evaluation in text outputs generated by GPT-4o for identical prompts. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Example of diversity evaluation in image outputs generated by DALL-E 3 for identical prompts. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Computational efficiency comparison of GPT-4o and DALL-E 3 for single run of a prompt. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Example of ethical evaluation in outputs with a problematic case. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 20 canonical work pages

  1. [1]

    Fairness and Machine Learning: Limitations and Opportunities,

    S. Barocas, M. Hardt, and A. Narayanan, “Fairness and Machine Learning: Limitations and Opportunities,” Cambridge, MA: MIT Press, 2020, URL: https://api.semanticscholar.org/CorpusID:113402716

  2. [2]

    On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?,

    E. M. Bender, T. Gebru, and A. McMillan-Major, et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?,” in Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 610–623, doi: 10.1145/3442188.3445922. 21

  3. [3]

    Fairness in Machine Learning: Lessons from Political Philosophy,

    R. Binns, “Fairness in Machine Learning: Lessons from Political Philosophy,” in Proceedings of the 2018 Conference on Fairness, Accountability, and Transparency , 2018, pp. 149–158, URL: https://proceedings.mlr.press/v81/binns18a.html

  4. [4]

    On the Opportunities and Risks of Foundation Models,

    R. Bommasani, D. A. Hudson, E. Adeli, et al., “On the Opportunities and Risks of Foundation Models,” Journal of Artificial Intelligence Research , vol. 74, pp. 65–133, 2022, URL: https://api.semanticscholar.org/CorpusID:237091588

  5. [5]

    Scaling Laws for Neural Language Models,

    T. B. Brown, J. Kaplan, and B. Chess, et al., “Scaling Laws for Neural Language Models,” Journal of Machine Learning Research , vol. 22, no. 1, pp. 1–20, 2020, URL: https://arxiv.org/abs/2001.08361

  6. [6]

    Language Models are Few-Shot Learners,

    T. B. Brown, et al., “Language Models are Few-Shot Learners,” Advances in Neural Information Processing Systems, vol. 33, 2020, URL: https://arxiv.org/abs/2005.14165

  7. [7]

    Datasheets for Datasets,

    T. Gebru, J. Morgenstern, B. Vecchione, et al., “Datasheets for Datasets,” Communications of the ACM , vol. 64, no. 12, pp. 86–92, Dec. 2021, doi: 10.1145/3458723

  8. [8]

    Gender Bias and Stereotypes in Large Language Models,

    H. Kotek, R. Dockum, and D. Q. Sun, “Gender Bias and Stereotypes in Large Language Models,” in Proceedings of the ACM Conference of Collective Intelligence , 2023, pp. 12–21, doi: 10.1145/3701268.3701287

Show all 27 references
  1. [9]

    Model Cards for Model Reporting,

    M. Mitchell, S. Wu, A. Zaldivar, et al., “Model Cards for Model Reporting,” in Proceedings of the Conference on Fairness, Accountability, and Transparency , 2019, pp. 220–229, doi: 10.1145/3287560.3287596

  2. [10]

    Zero-Shot Text-to-Image Generation,

    A. Ramesh, M. Pavlov, and G. Goh, “Zero-Shot Text-to-Image Generation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2021, URL: https://arxiv.org/abs/2102.12092

  3. [11]

    Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints,

    J. Zhao, T. Wang, and M. Yatskar, “Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 2979–2987, doi: 10.18653/v1/D17-1323

  4. [12]

    The State of AI in 2023: Generative AI’s Breakout Year,

    McKinsey & Company, “The State of AI in 2023: Generative AI’s Breakout Year,” McKinsey & Company ,

  5. [13]

    The State of AI in Early 2024: Gen AI Adoption Spikes and Starts to Generate Value,

    McKinsey & Company, “The State of AI in Early 2024: Gen AI Adoption Spikes and Starts to Generate Value,” McKinsey & Company , 2024. [Online]. URL: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai

  6. [14]

    The Top Trends in Tech 2024,

    McKinsey & Company, “The Top Trends in Tech 2024,” McKinsey Technology Trends Outlook , 2024. [Online]. URL: http://www.mckinsey.com/∼/media/mckinsey/business%20functions/mckinsey%20digital/our %20insights/the%20top%20trends%20in%20tech%202024/mckinsey-technology-trends-outloo...

  7. [15]

    The Jobs Most Likely to Be Lost and Created Because of AI,

    World Economic Forum, “The Jobs Most Likely to Be Lost and Created Because of AI,” World Economic Forum, May 4, 2023. [Online]. URL: https://www.weforum.org/stories/2023/05/jobs-lost-created- ai-gpt/#: :text=The%20jobs%20AI%20could%20create,the%20most%20prominent%20emerging%20roles

  8. [16]

    How Might Generative AI Change Creative Jobs?,

    World Economic Forum, “How Might Generative AI Change Creative Jobs?,” World Economic Forum, 2023. [Online]. URL: https://www.weforum.org/stories/2023/05/generative-ai-creative-jobs/. 22 Appendix A Definitions of Technical Terms

  9. [18]

    These models are capable of creating content that mimics human creativity

    Generative AI : AI models designed to generate new content (e.g., text, images, or videos) based on learned patterns from existing data. These models are capable of creating content that mimics human creativity

  10. [19]

    It is capable of generating high-quality text content for a variety of tasks and is the model behind ChatGPT, which is used in interactive AI applications

    GPT-4o: A version of the GPT-4 model optimized for efficiency and lower computational cost. It is capable of generating high-quality text content for a variety of tasks and is the model behind ChatGPT, which is used in interactive AI applications

  11. [20]

    It uses a transformer-based architecture similar to GPT models, but with a focus on visual content

    DALL-E 3 : A generative model by OpenAI capable of creating diverse and complex images from textual descriptions. It uses a transformer-based architecture similar to GPT models, but with a focus on visual content

  12. [21]

    Neural networks consist of layers of interconnected nodes (neurons), each performing a mathematical operation to process input data and pass it through the network

    Neural Networks : A class of machine learning algorithms inspired by the structure and function of biological neural networks. Neural networks consist of layers of interconnected nodes (neurons), each performing a mathematical operation to process input data and pass it throug...

  13. [22]

    This architecture is widely used in NLP tasks (e.g., GPT models) and has proven effective for both text and image generation tasks

    Transformer-based Systems : Deep learning models that rely on the transformer architecture, which uses self-attention mechanisms to process input data in parallel, allowing the model to focus on different parts of the data simultaneously. This architecture is widely used in NL...

  14. [23]

    It is commonly used in text analysis and machine learning to 23 determine the similarity between documents or text sequences based on their vector representations

    Cosine Similarity : A metric used to measure the similarity between two vectors in a multi-dimensional space by calculating the cosine of the angle between them. It is commonly used in text analysis and machine learning to 23 determine the similarity between documents or text ...

  15. [24]

    Perceptual Hashing: A technique used to create a compact digital fingerprint (hash) of an image or multimedia content. Unlike traditional cryptographic hashing, which changes significantly with even the smallest alteration in input data, perceptual hashing generates similar ha...

  16. [25]

    • DALL-E 3 (via ChatGPT) was used for generating visual content to be analyzed in the experiments

    AI Models Used and Purposes: • GPT-4o (via ChatGPT) was used for generating textual content to be analyzed in the experiments. • DALL-E 3 (via ChatGPT) was used for generating visual content to be analyzed in the experiments

  17. [26]

    The building should have gold details around the entrance, showcasing its cultural significance

    Prompts Used for AI-Generated Content (directly shown in this paper): • Figure 3 (Comparison of AI-generated and human-created content): – GPT-4o: ”Write a short article about theaters with highlighting their cultural significance.” – DALL-E 3: ”Create a b+w image of a theater...

  18. [27]

    • I verified the correctness and appropriateness of all generated outputs and take full responsibility for their use in this work

    Generated Content and Responsibility: • All AI-generated content was created solely for the experiments described in this research. • I verified the correctness and appropriateness of all generated outputs and take full responsibility for their use in this work. This declarati...

  19. [2023]

    URL: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in- 2023-generative-ais-breakout-year

    [Online]. URL: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in- 2023-generative-ais-breakout-year

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.