Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Risks of AI-driven product development and strategies for their mitigation

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This perspective paper argues that AI-driven product development, especially by autonomous black-box systems, will produce a specific set of technical and sociotechnical risks, and that eight principles—human control, accountability…

desk verdict A competent, honest perspective that repackages known AI risks for product development; worth engaging, but soften the novelty and certainty claims before publishing. read the letter →

arxiv 2506.00047 v1 pith:B5YKRA3W submitted 2025-05-28 cs.CY cs.AIcs.CE

Jan Göpfert , Jann M. Weinand , Patrick Kuckertz , Noah Pflugradt , Jochen Lin{ss}en This is my paper
classification cs.CYcs.AIcs.CE
keywords AI-drivenproductdevelopmentAIsafetyengineeringdesignriskassessmenthumanoversightexplainabledual-usealignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that as AI systems take over product development—especially opaque, agentic systems that perform design tasks with high autonomy—a recognizable set of technical and sociotechnical risks will emerge, and that these risks can be mitigated by a handful of design principles adopted now. The authors assemble the first systematic risk catalogue for AI-driven product development, spanning design errors amplified by declining human oversight, misalignment with human intent, manipulation and excessive agency, dual-use democratization, labor and market concentration, and strain on the patent system. They propose eight principles as a starting point for norms, standards, and regulation. The paper is a perspective meant to open a consensus-building discussion before the technology matures.

What carries the argument

The central object is the 'error-inflation loop': a coupling between falling AI error rates and declining human oversight, amplified by automation bias and the convincing fluency of LLMs. Around this loop the paper organizes its risk catalogue and maps each risk to mitigation strategies in a table, with the eight principles (Box 1) as the proposed countermeasures. The principles function as constraints on the design system's operating space: humans control the outer process, design results must be explainable and tested, the task and solution space must be narrowly confined and sandboxed, and the system must be aligned with human intentions and evaluated in its holistic context. Each principle is tied to at least one risk it is supposed to mitigate.

What would settle it

A longitudinal field study of engineering teams using AI design tools could settle the central coupling: if, as AI error rates fall, the share of design errors caught by human review does not decline, the error-inflation mechanism on which the technical risk section rests would be unsupported.

Watch

Extended reading notes

Core claim

The central claim is that the shift toward autonomous AI design systems is not a single future event but a trajectory whose risks are already identifiable, and that the dominant danger is the combination of black-box outputs with diminishing human scrutiny. The paper treats 'error inflation' as the key mechanism: as systems improve, humans oversee less, so the probability that any individual error is caught falls, and convincing false statements from LLMs make acceptance of erroneous results more likely. From this technical core, the paper derives a catalogue of sociotechnical risks—dual-use, power concentration, accountability diffusion, weakened worker position, patent-system strain—and argues that eight principles, applied in the design of the systems themselves and in their organizational embedding, can contain these risks. It explicitly frames the result as a basis for discussion rather than a finished regulatory proposal.

Load-bearing premise

The paper's risk catalogue and principles assume a future in which AI systems perform design tasks with high autonomy—a scenario the authors themselves call science fiction—and if actual development remains copilot-like with humans always overseeing, the risks could look very different and the principles could be aimed at the wrong target.

Editorial extensions

If this is right

  • Regulators and standardization bodies get a concrete checklist: human control and accountability, explainable and tested design, constrained and sandboxed operation, and alignment with human values.
  • Product-safety regulation, which today covers unsafe products but not poorly designed or systematically biased ones, would need to extend into the development process itself.
  • Patent law faces pressure to adapt, since lowering the barrier to invention could flood the system with 'obvious' AI-generated designs while dual-use control may require registration of inventions.
  • Engineering education and work design would need to change, with curricula covering the limitations of AI tools and organizations preserving meaningful human work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the real trajectory is a slow extension of copilot-style assistants with humans always in the loop, the risk profile changes: error inflation may be weaker, while accountability diffusion and deskilling may still materialize—so the eight principles would need to be weighted differently.
  • The 'explainable design' principle suggests a testable metric: the rate at which human reviewers detect AI design errors could serve as a measurable proxy for explainability, linking the principle to engineering practice.
  • The mapping in Table 1 is presented as the authors' judgment; converting it into an ordinal risk model (e.g., which principles are necessary, which are merely helpful) would let organizations prioritize interventions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This perspective argues that increasing automation of product development via opaque AI systems will introduce a defined set of technical and sociotechnical risks, and proposes eight principles (Box 1) to mitigate them. The technical risks include error inflation due to declining human oversight, misalignment, manipulation, and excessive agency; the sociotechnical risks include dual-use democratization, labor and market concentration, accountability diffusion, and patent-system strain. After reviewing mitigation strategies, the paper maps risks to principles in Table 1 and calls for an evidence-based debate. The authors repeatedly hedge their claims and explicitly describe the work as theoretical, calling for empirical research.

Significance. The paper is a useful early-stage contribution that consolidates a broad literature into a structured risk catalogue and a compact principle set for a topic with little prior synthesis. Its strengths are transparency about the speculative scenario (high-autonomy AI systems), explicit acknowledgement that the risk-to-principle mapping is the authors' judgment, and a clear call for empirical research. If the risk catalogue holds, the principles could serve as a starting point for norms and regulation, but the absence of empirical validation for a key behavioral premise limits the current support for the error-inflation risk.

major comments (2)
  1. [Risks, Technical concerns] The first technical risk rests on the premise that "With decreasing number of errors, it is likely that also human oversight decreases" (third paragraph of the Risks section). The only supporting evidence cited are automation-bias lab studies (ref 42) and software-security experiments (refs 15–18), none of which is field evidence from physical engineering design. In regulated product domains (medical devices, aviation, EU GPSR) manufacturers are legally required to maintain internal risk-analysis and validation processes, which could keep oversight high even when AI error rates are low. As this coupling is load-bearing for the "error inflation" risk, the manuscript should either provide evidence for it in design settings or explicitly reframe the risk as conditional on a specific organizational and regulatory environment.
  2. [Risks section (general)] The paper uses evidence from software development (refs 14–18) to support claims about physical product development without discussing the differences between the two domains. For example, the automation-bias and overconfidence findings from AI-assisted coding may not transfer to mechanical design, where verification is often more expensive and safety standards are stricter. The manuscript should address domain transferability or label these as analogies rather than direct evidence.
minor comments (5)
  1. [Abstract] The abstract contains a typographical error: "T o this end" should be "To this end."
  2. [Figure 1] The publication counts rely on a Scopus keyword-based query that may overcount documents merely mentioning AI and product development; a brief note on these limitations would improve the figure's credibility.
  3. [Table 1] The table legend distinguishes "strongly" and "moderately" contributions, but the cells use only green shades; adding explicit labels in the cells would improve accessibility and interpretability.
  4. [Introduction] Reference 31 is the authors' own prior work, cited to support the claim that AI is entering product development; citing independent studies in addition would strengthen that claim.
  5. [Conclusion] The paper appropriately admits that its considerations are theoretical and calls for empirical research; it would be even more actionable if it suggested specific study designs, such as longitudinal field studies of human-AI interaction in design teams.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the risk catalogue and principles are self-contained and grounded in external literature; only a minor non-load-bearing self-citation is present.

full rationale

The paper makes no formal derivation with equations and presents no empirical prediction fitted to data. Its central risk catalogue is assembled from external literature: LLM limitations (refs 2-5), alignment (ref 19), automation bias (ref 42), adversarial attacks and excessive agency (refs 46-50), the EU AI Act (ref 40), and product-safety regulation (ref 35). The proposed principles in Box 1 are normative recommendations supported by explicit reasoning (e.g., V1 follows from the need for humans to assess safety; C2 follows from the need to prevent excessive agency), not results derived from the risks by construction. The only self-citation is ref 31, used in the Introduction to support the claim that AI is entering product development; that claim is independently supported by the paper's own Scopus-based publication data, the Stack Overflow survey (ref 14), and WIPO discussions (ref 1), so the self-citation is not load-bearing. The authors also explicitly state in the Conclusion that 'the considerations in this perspective are theoretical' and that empirical research is needed, and Table 1's risk-mitigation mapping is flagged as 'based on the judgment of the authors.' No circular step can be exhibited via quoted equations or fitted parameters renamed as predictions, so the score reflects only minor, non-load-bearing self-citation.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters: the paper fits no numbers; Figure 1 is a descriptive publication count from a documented Scopus query, not a fitted result. The analysis rests on five qualitative assumptions, each stated in the text or standard in the cited AI-safety literature; none is derived in the paper. No invented entities: the eight principles in Box 1 are normative proposals, not technical entities with falsifiable handles, so the invented-entities ledger is empty.

assumptions (5)
  • domain assumption Humanity is moving towards higher levels of automation in product development.
    Introduction: 'we have no doubt that humanity is moving towards higher levels of automation in product development'. The whole risk assessment presupposes this trajectory, which the authors themselves call 'still science fiction' in its end state.
  • domain assumption LLMs are opaque black boxes that cannot be made fully transparent or comprehensible.
    Introduction, citing refs 2-5. The opacity premise motivates the explainable-design (V1) and tested-design (V2) principles; if future systems were fully interpretable, several principles would lose force.
  • domain assumption The relevant scenario is high-autonomy AI that performs the design task and optionally other tasks.
    Introduction: 'we consider the more extreme and distant case of AI systems that perform the design task and optionally other tasks with high autonomy'. The risk catalogue and principles are derived for this scenario only.
  • domain assumption Alignment of AI with human intentions is fundamentally difficult and possibly unsolvable.
    Risks section, citing ref 19 on fundamental limitations of alignment. This underpins the misalignment risk category and principle A1.
  • domain assumption Human oversight declines as AI error rates decrease, and humans are subject to automation bias.
    Risks section: 'With decreasing number of errors, it is likely that also human oversight decreases', supported by ref 42. This coupling drives the technical-risk story but is asserted without quantitative or field evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Risks of AI-driven product development and strategies for their mitigation." pith.science (2026). https://pith.science/paper/B5YKRA3W

@misc{pith2026250600047,
  author       = {Pith},
  title        = {Pith review of: Risks of AI-driven product development and strategies for their mitigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B5YKRA3W}},
  note         = {Machine review of arXiv:2506.00047}
}
read the original abstract

Humanity is progressing towards automated product development, a trend that promises faster creation of better products and thus the acceleration of technological progress. However, increasing reliance on non-human agents for this process introduces many risks. This perspective aims to initiate a discussion on these risks and appropriate mitigation strategies. To this end, we outline a set of principles for safer AI-driven product development which emphasize human oversight, accountability, and explainable design, among others. The risk assessment covers both technical risks which affect product quality and safety, and sociotechnical risks which affect society. While AI-driven product development is still in its early stages, this discussion will help balance its opportunities and risks without delaying essential progress in understanding, norm-setting, and regulation.

Figures

Figures reproduced from arXiv: 2506.00047 by the authors.

Figure 1
Figure 1. Annual documents published on the intersection of AI and engineering with historical mile￾stones and paradigms in AI highlighted. The annual number of publications began to rise sharply around 2017, increasing sevenfold by 2024. See Jurafsky and Martin12 and Gururaja et al.13 for histori￾cal overviews on AI from the perspective of natural language processing. The data is from Scopus us￾ing the following query: TITLE… view at source ↗
Figure 2
Figure 2. Principles for safer AI-driven product development (see Box 1) situated in an abstract development process. Strictly confined operating and solution space. A narrowly constrained solution space and clearly defined objec￾tives (C1) reduce the number of potential errors as well as the attack surface in the event of manipulation. Additionally, it enables more targeted validation. With greater reliance on AI, a more com… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLMs and Agentic AI in Insurance Decision-Making: Opportunities and Challenges For Africa

    cs.CE 2025-08 unverdicted novelty 3.0 of 10

    LLMs and agentic AI are presented as a transformative opportunity for African insurance, with a call for African-led, equitable AI strategies.

Reference graph

Works this paper leans on

93 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Revised Issues Paper on Intellectual Property Policy and Artificial IntelligenceSecond Session

    WIPO Secretariat. Revised Issues Paper on Intellectual Property Policy and Artificial IntelligenceSecond Session. Document code: WIPO/IP/AI/2/GE/20/1 REV (World Intellectual Property Organization, 2020)

    2020

  2. [2]

    & Wallace, B

    Jain, S. & Wallace, B. C. Attention Is Not Explanation in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) NAACL-HLT 2019 (eds Burstein, J., Doran, C. & Solorio, T.) (Association for Computational Linguistics, Minneapolis, Minnes...

    doi:10.18653/v1/n19-1357 2019

  3. [3]

    & Geva, M

    Mosbach, M., Gautam, V ., Vergara Browne, T., Klakow, D. & Geva, M. From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing EMNLP 2024 (eds Al-Onaizan, Y ., Bansal, M. & Chen, Y . -N.) (Association for Computational Linguistics, Miami, F...

    doi:10.18653/v1/2024.emnlp- 2024

  4. [4]

    S., Jenner, E., Casper, S., Sourbut, O., Edelman, B

    Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., Günther, M., Korinek, A., Hernandez-Orallo, J., Hammond, L., Bigelow, E., Pan, A., Langosco, L., Korbak, T., Zhang, H., Zhong, R., hÉigeartaigh, S. Ó., Recchia, G., Corsi, G., Chan, A., Anderljung, M., Edwards...

    arXiv doi:10.48550/arxiv.2404.09932 2024

  5. [5]

    J., Casper, S., Tegmark, M., Saunders, W., Bau, D., Todd, E., Geiger, A., Geva, M., Hoogland, J., Murfet, D

    Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., Biderman, S., Garriga-Alonso, A., Conmy, A., Nanda, N., Rumbelow, J., Wattenberg, M., Schoots, N., Miller, J., Michaud, E. J., Casper, S., Tegmark, M., Saunders, W., Bau, D., Todd, E., Geiger, A., Geva, M., Hoogland, J., Mur...

    arXiv doi:10.48550/arxiv.2501.16496 2025

  6. [6]

    Frontier Review of Multimodal AI eng

    Nan, D. Frontier Review of Multimodal AI eng. in Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 2: Frontier Forum)CCL 2023 (ed Zhang, J.) (Chinese Information Processing Society of China, Harbin, China, 2023), 110–118

    2023

  7. [7]

    S., O’Brien, J., Cai, C

    Park, J. S., O’Brien, J., Cai, C. J., Morris, M. R., Liang, P. & Bernstein, M. S. Generative Agents: Interactive Simulacra of Human Behavior in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (Association for Computing Machinery, New York, NY , USA, 2023), 1–22.ISBN : 9798400701320. doi:10.1145/3586183.3606763

    arXiv 2023

  8. [8]

    V ., Bloom, R., Broadley, T., Du, H., Goodrich, B., Jurkovic, N., Miles, L

    Kwa, T., West, B., Becker, J., Deng, A., Garcia, K., Hasin, M., Jawhar, S., Kinniment, M., Rush, N., Arx, S. V ., Bloom, R., Broadley, T., Du, H., Goodrich, B., Jurkovic, N., Miles, L. H., Nix, S., Lin, T., Parikh, N., Rein, D., Sato, L. J. K., Wijk, H., Ziegler, D. M., Barnes, E. & Chan, L. Measuring AI Ability to Complete Long Tasks. doi:10.48550/arXiv....

    arXiv doi:10.48550/arxiv.2503.14499 2025

Show all 93 references
  1. [9]

    Bresnahan, T. F. & Trajtenberg, M. General Purpose Technologies ‘Engines of Growth’?Journal of Econometrics 65, 83–108. ISSN : 0304-4076. doi:10.1016/0304-4076(94)01598-T (1995)

    1995 doi

  2. [10]

    Artificial Intelligence as a General-Purpose Technology: An Historical Perspective

    Crafts, N. Artificial Intelligence as a General-Purpose Technology: An Historical Perspective. Oxford Review of Economic Policy 37, 521–536. ISSN : 0266-903X. doi:10.1093/oxrep/grab012 (2021)

    2021 doi

  3. [11]

    A., Adeli, E., Altman, R., Arora, S., Arx, S

    Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., Arx, S. von, Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., Donahue, C., Doumbouya, M.,...

    2022 doi arXiv

  4. [12]

    & Martin, J

    Jurafsky, D. & Martin, J. H. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd Edition Online manuscript released January 12, 2025 (2025)

    2025

  5. [13]

    & Strubell, E

    Gururaja, S., Bertsch, A., Na, C., Widder, D. & Strubell, E. To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processingin Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing EMNLP 2023 (eds Boua...

    2023 doi

  6. [14]

    AI | 2024 Stack Overflow Developer Survey https://survey.stackoverflow.co/2024/ ai/ (2025)

    Stack Overflow. AI | 2024 Stack Overflow Developer Survey https://survey.stackoverflow.co/2024/ ai/ (2025)

    2025

  7. [15]

    H., Horstmann, S

    Klemmer, J. H., Horstmann, S. A., Patnaik, N., Ludden, C., Burton, C., Powers, C., Massacci, F., Rahman, A., V otipka, D., Lipford, H. R., Rashid, A., Naiakshina, A. & Fahl, S.Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns i...

    2024

  8. [16]

    & Karri, R

    Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B. & Karri, R. Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions in. 2022 IEEE Symposium on Security and Privacy (SP) (IEEE Computer Society, 2022), 754–768. ISBN : 978-1-66541-316-9. doi:10.1109...

    2022

  9. [17]

    & Boneh, D

    Perry, N., Srivastava, M., Kumar, D. & Boneh, D. Do Users Write More Insecure Code with AI Assistants? in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (Association for Computing Machinery, New York, NY , USA, 2023), 2785–2799.ISBN : 979...

    2023 doi

  10. [18]

    J., Rasheed, S., Moradidakhel, A., Tahir, A

    Majdinasab, V ., Bishop, M. J., Rasheed, S., Moradidakhel, A., Tahir, A. & Khomh, F. Assessing the Security of GitHub Copilot’s Generated Code - A Targeted Replication Studyin. 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (IEEE C...

    2024

  11. [19]

    Wolf, Y ., Wies, N., Avnery, O., Levine, Y . & Shashua, A.Fundamental Limitations of Alignment in Large Language Models in Proceedings of the 41st International Conference on Machine Learning 235 (JMLR.org, Vienna, Austria, 2025), 53079–53112

    2025

  12. [20]

    & Kirk-Giannini, C

    Bales, A., D’Alessandro, W. & Kirk-Giannini, C. D. Artificial Intelligence: Arguments for Catastrophic Risk. Philosophy Compass 19, e12964. ISSN : 1747-9991. doi:10.1111/phc3.12964 (2024)

    2024 doi

  13. [21]

    The Governance of Artificial Intelligence: Interim Report Ninth Report of Session 2022–23 (House of Commons, London, UK, 2023)

    Science, Innovation and Technology Committee. The Governance of Artificial Intelligence: Interim Report Ninth Report of Session 2022–23 (House of Commons, London, UK, 2023)

    2022

  14. [22]

    A., Rimell, L., Isaac, W., Haas, J., Legassick, S., Irving, G

    Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., Biles, C., Brown, S., Kenton, Z., Hawkins, W., Stepleton, T., Birhane, A., Hendricks, L. A., Rimell, L., Isaac, W., Haas, J., Legassick, S., Irving, G....

    2022

  15. [23]

    R., Vidgen, B., Röttger, P

    Kirk, H. R., Vidgen, B., Röttger, P. & Hale, S. A. The Benefits, Risks and Bounds of Personalizing the Alignment of Large Language Models to Individuals. Nature Machine Intelligence 6, 383–392. ISSN : 2522-5839. doi:10.1038/ s42256-024-00820-y (2024)

    2024

  16. [24]

    & Kasirzadeh, A

    Gyevnár, B. & Kasirzadeh, A. AI Safety for Everyone. Nature Machine Intelligence 7, 531–542. ISSN : 2522-5839. doi:10.1038/s42256-025-01020-y (2025)

    2025 doi

  17. [25]

    & Ó hÉigeartaigh, S

    Tzachor, A., Devare, M., King, B., Avin, S. & Ó hÉigeartaigh, S. Responsible Artificial Intelligence in Agriculture Requires Systemic Understanding of Risks and Externalities. Nature Machine Intelligence 4, 104–109. ISSN : 2522-

  18. [26]

    A., Waeiss, Q., Bhatia, R

    Trotsyuk, A. A., Waeiss, Q., Bhatia, R. T., Aponte, B. J., Heffernan, I. M. L., Madgavkar, D., Felder, R. M., Lehmann, L. S., Palmer, M. J., Greely, H., Wald, R., Goetz, L., Trengove, M., Vandersluis, R., Lin, H., Cho, M. K., Altman, R. B., Endy, D., Relman, D. A., Levi, M., S...

    2024 doi

  19. [27]

    & Jayaraman, P

    Etesam, Y ., Cheong, H., Ataei, M. & Jayaraman, P. K. Deep Generative Model for Mechanical System Configuration Design. doi:10.48550/arXiv.2409.06016. arXiv: 2409.06016 [cs] (2024). Preprint

    2024 doi arXiv

  20. [28]

    & Tessier, A

    Ataei, M., Cheong, H., Grandi, D., Wang, Y ., Morris, N. & Tessier, A. Elicitron: An LLM Agent-Based Simulation Framework for Design Requirements Elicitation. doi:10.48550/arXiv.2404.16045. arXiv: 2404.16045 [cs] (2024). Preprint

    2024 doi arXiv

  21. [29]

    H., Srivastava, A

    Picard, C., Regenwetter, L., Nobari, A. H., Srivastava, A. & Ahmed, F. Generative Optimization: A Perspective on AI-Enhanced Problem Solving in Engineering. doi: 10.48550/arXiv.2412.13281 . arXiv: 2412.13281 [cs] (2024). Preprint

    2024 doi

  22. [30]

    C., Grandi, D., Tomich, R., Alam, M

    Doris, A. C., Grandi, D., Tomich, R., Alam, M. F., Ataei, M., Cheong, H. & Ahmed, F. DesignQA: A Multimodal Benchmark for Evaluating Large Language Models’ Understanding of Engineering Documentation. Journal of Computing and Information Science in Engineering 25. ISSN : 1530-9...

    2024 doi

  23. [31]

    M., Kuckertz, P

    Göpfert, J., Weinand, J. M., Kuckertz, P. & Stolten, D. Opportunities for Large Language Models and Discourse in Engineering Design. Energy and AI 17, 100383. ISSN : 2666-5468. doi:10.1016/j.egyai.2024.100383 (2024)

    2024

  24. [32]

    T., Srour, M

    Piller, F. T., Srour, M. & Marion, T. J. Generative AI, Innovation, and Trust. The Journal of Applied Behavioral Science 60, 613–622. ISSN : 0021-8863. doi:10.1177/00218863241285033 (2024)

    2024 doi

  25. [33]

    T., Bouschery, S

    Piller, F. T., Bouschery, S. G. & Blazevic, V . inPDMA Handbook of New Product Development (4th edition) (eds Bstieler, L & Noble, C) 407–425 (John Wiley & Sons, Hoboken, New Jersey, 2023). ISBN : 978-1-119-89021-8

    2023

  26. [34]

    G., Blazevic, V

    Bouschery, S. G., Blazevic, V . & Piller, F. T. Artificial Intelligence-Augmented Brainstorming: How Humans and Ai Beat Humans Alone. doi:10.2139/ssrn.4724068 . Social Science Research Network: 4724068 (2024). Preprint

    2024 doi

  27. [35]

    EU. Regulation (EU) 2023/988 of the European Parliament and of the Council of 10 May 2023 on General Product Safety, Amending Regulation (EU) No 1025/2012 of the European Parliament and of the Council and Directive (EU) 2020/1828 of the European Parliament and the Council, and...

    2025

  28. [36]

    in Philosophy of Technology and Engineering Sciences(ed Meijers, A.) 1103–1146 (North-Holland, Amsterdam, 2009)

    Grunwald, A. in Philosophy of Technology and Engineering Sciences(ed Meijers, A.) 1103–1146 (North-Holland, Amsterdam, 2009). doi:10.1016/B978-0-444-51667-1.50044-6

    2009 doi

  29. [37]

    & Lensman, T

    Acemoglu, D. & Lensman, T. Regulating Transformative Technologies. American Economic Review: Insights 6, 359–376. ISSN : 2640-205X. doi:10.1257/aeri.20230353 (2024)

    2024 doi

  30. [38]

    & Vayena, E

    Jobin, A., Ienca, M. & Vayena, E. The Global Landscape of AI Ethics Guidelines.Nature Machine Intelligence 1, 389–399. ISSN : 2522-5839. doi:10.1038/s42256-019-0088-2 (2019)

    2019 doi

  31. [39]

    & Lundgren, M

    Tallberg, J., Erman, E., Furendal, M., Geith, J., Klamberg, M. & Lundgren, M. The Global Governance of Artificial Intelligence: Next Steps for Empirical and Normative Research. International Studies Review 25, viad040. ISSN : 1521-9488. doi:10.1093/isr/viad040 (2023)

    2023 doi

  32. [40]

    EU. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying down Harmonised Rules on Artificial Intelligence and Amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 a...

    2025

  33. [41]

    Risk Management in the Artificial Intelligence Act

    Schuett, J. Risk Management in the Artificial Intelligence Act. European Journal of Risk Regulation 15, 367–385. ISSN : 1867-299X, 2190-8249. doi:10.1017/err.2023.1 (2024)

    2024 doi

  34. [42]

    J., Mosier, K

    Skitka, L. J., Mosier, K. L. & Burdick, M. A. R. K. Does Automation Bias Decision-Making? International Journal of Human-Computer Studies 51, 991–1006. ISSN : 1071-5819. doi:10.1006/ijhc.1999.0252 (1999)

    1999

  35. [43]

    & Germani, F

    Spitale, G., Biller-Andorno, N. & Germani, F. AI Model GPT-3 (Dis)Informs Us Better than Humans. Science Advances 9, eadh1850. doi:10.1126/sciadv.adh1850 (2023)

    2023 doi

  36. [44]

    Value-Sensitive Design.interactions 3, 16–23

    Friedman, B. Value-Sensitive Design.interactions 3, 16–23. ISSN : 1072-5520. doi:10.1145/242485.242493 (1996)

    1996

  37. [45]

    inThe PDMA Handbook of New Product Development (eds Bstieler, L

    Marion, Tucker J., Moghaddam, Mohsen, Ciuccarelli, Paolo & Wang, Lu. inThe PDMA Handbook of New Product Development (eds Bstieler, L. & Noble, Charles H.) Fourth edition (John Wiley & Sons Inc, Hoboken, New Jersey, 2023). ISBN : 978-1-119-89021-8

    2023

  38. [46]

    Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z. & Fredrikson, M. Universal and Transferable Adversarial Attacks on Aligned Language Models. doi: 10.48550/arXiv.2307.15043 . arXiv: 2307.15043 [cs] (2023). Preprint

    2023 doi arXiv

  39. [47]

    & Henderson, P

    Qi, X., Panda, A., Lyu, K., Ma, X., Roy, S., Beirami, A., Mittal, P. & Henderson, P. Safety Alignment Should Be Made More Than Just a Few Tokens Deep. doi:10.48550/arXiv.2406.05946. arXiv: 2406.05946 [cs] (2024). Preprint

    2024 doi arXiv

  40. [48]

    Skalse, J., Howe, N., Krasheninnikov, D. & Krueger, D.Defining and Characterizing Reward Gaming in Advances in Neural Information Processing Systems NIPS’22: Proceedings of the 36th International Conference on Neural Information Processing Systems. 35. https://proceedings.neur...

    2022

  41. [49]

    OW ASP LLM Project.OWASP Top 10 for LLM & Generative AI Security: LLM06:2025 Excessive Agencyhttps: //genai.owasp.org/llmrisk/llm062025-excessive-agency/ (2025)

    2025

  42. [50]

    Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y ., Carroll, M., Lin, M., Mayhew, A., Collins, K., Molamohammadi, M., Burden, J., Zhao, W., Rismani, S., V oudouris, K., Bhatt, U., Weller, A., Krueger, D. & Maharaj...

    2023

  43. [51]

    Strubell, E., Ganesh, A. & McCallum, A.Energy and Policy Considerations for Deep Learning in NLPin Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics ACL 2019 (eds Korhonen, A., Traum, D. & Màrquez, L.) (Association for Computational Lingui...

    2019

  44. [52]

    & Dean, J

    Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M. & Dean, J. Carbon Emissions and Large Neural Network Training. doi:10.48550/arXiv.2104.10350 . arXiv: 2104.10350 [cs] (2021). Preprint

    2021 doi arXiv

  45. [53]

    Vries, A. de. The Growing Energy Footprint of Artificial Intelligence. Joule 7, 2191–2194. ISSN : 2542-4785, 2542-4351. doi:10.1016/j.joule.2023.09.004 (2023)

    2023 doi

  46. [54]

    Nist, G. M. Managing Misuse Risk for Dual-Use Foundation ModelsNIST AI NIST AI 800-1 ipd (National Institute of Standards and Technology, Gaithersburg, MD, 2024), NIST AI NIST AI 800–1 ipd. doi:10.6028/NIST.AI.800- 1.ipd. 12/15 Preprint

    2024 doi

  47. [55]

    Accountability in a Computerized Society

    Nissenbaum, H. Accountability in a Computerized Society. Science and Engineering Ethics 2, 25–42. ISSN : 1471-

  48. [56]

    F., Moss, E., Laufer, B

    Cooper, A. F., Moss, E., Laufer, B. & Nissenbaum, H. Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing M...

    2022

  49. [57]

    Hackman, J. R. & Oldham, G. R. Motivation through the Design of Work: Test of a Theory. Organizational Behavior and Human Performance 16, 250–279. ISSN : 0030-5073. doi:10.1016/0030-5073(76)90016-7 (1976)

    1976 doi

  50. [58]

    K., Morgeson, F

    Parker, S. K., Morgeson, F. P. & Johns, G. One Hundred Years of Work Design Research: Looking Back and Looking Forward. Journal of Applied Psychology 102, 403–420. ISSN : 1939-1854. doi:10.1037/apl0000106 (2017)

    2017 doi

  51. [59]

    Parker, S. K. & Grote, G. Automation, Algorithms, and Beyond: Why Work Design Matters More Than Ever in a Digital World. Applied Psychology 71, 1171–1204. ISSN : 1464-0597. doi:10.1111/apps.12241 (2022)

    2022 doi

  52. [60]

    Artificial Intelligence, Scientific Discovery, and Product Innovation

    Toner-Rodgers, A. Artificial Intelligence, Scientific Discovery, and Product Innovation. doi:10.48550/arXiv. 2412.17866. arXiv: 2412.17866 [econ] (2024). Preprint

    2024 doi arXiv

  53. [61]

    The Connected Workplace: Characteristics and Social Consequences of Work Surveillance in the Age of Datification, Sensorization, and Artificial Intelligence

    Mettler, T. The Connected Workplace: Characteristics and Social Consequences of Work Surveillance in the Age of Datification, Sensorization, and Artificial Intelligence. Journal of Information Technology 39, 547–567. ISSN : 0268-3962. doi:10.1177/02683962231202535 (2024)

    2024 doi

  54. [62]

    Harms of AI

    Acemoglu, D. Harms of AI. Working Paper Seriesdoi:10.3386/w29247. National Bureau of Economic Research: 29247 (2021). Preprint

    2021 doi

  55. [63]

    J., Pizzinelli, C., Rockall, E

    Cazzaniga, M., Jaumotte, F., Li, L., Melina, G., Panton, A. J., Pizzinelli, C., Rockall, E. & Tavares, M. M. Gen-AI: Artificial Intelligence and the Future of Work.Staff Discussion Notes 2024. doi:10.5089/9798400262548.006 (2024)

    2024 doi

  56. [64]

    & Schiff, D

    Nazareno, L. & Schiff, D. S. The Impact of Automation and Artificial Intelligence on Worker Well-Being.Technology in Society 67, 101679. ISSN : 0160-791X. doi:10.1016/j.techsoc.2021.101679 (2021)

    2021

  57. [65]

    & Stella, L

    Giuntella, O., Koenig, J. & Stella, L. Artificial Intelligence and Workers’ Well-Being. doi: 10 . 2139 / ssrn . 4589223. Social Science Research Network: 4589223 (2023). Preprint

    2023

  58. [66]

    Losing Leverage: Employee Replaceability and Labor Market Power

    Estlund, C. Losing Leverage: Employee Replaceability and Labor Market Power. University of Chicago Law Review

  59. [67]

    & de Streel, A

    Fletcher, A., Crémer, J., Heidhues, P., Kimmelman, G., Monti, G., Podszun, R., Schnitzer, M., Scott Morton, F. & de Streel, A. The Effective Use of Economics in the EU Digital Markets Act. Journal of Competition Law & Economics 20, 1–19. ISSN : 1744-6414. doi:10.1093/joclec/nh...

    2024 doi

  60. [68]

    G., Whittaker, M

    Widder, D. G., Whittaker, M. & West, S. M. Why ‘Open’ AI Systems Are Actually Closed, and Why This Matters. Nature 635, 827–833. ISSN : 1476-4687. doi:10.1038/s41586-024-08141-1 (2024)

    2024 doi

  61. [69]

    & Levine, D

    Boldrin, M. & Levine, D. K. The Case against Patents. Journal of Economic Perspectives 27, 3–22. ISSN : 0895-3309. doi:10.1257/jep.27.1.3 (2013)

    2013 doi

  62. [70]

    Patentability of AI-Generated Inventions: Is a Reform of the Patent System Needed? doi:10.2139/ ssrn.3168703

    Ramalho, A. Patentability of AI-Generated Inventions: Is a Reform of the Patent System Needed? doi:10.2139/ ssrn.3168703. Social Science Research Network: 3168703 (2018). Preprint

    2018

  63. [71]

    De Rassenfosse, G., Jaffe, A. B. & Wasserman, M. F. AI-Generated Inventions: Implications for the Patent System. doi:10.2139/ssrn.4434054. Social Science Research Network: 4434054 (2023). Preprint

    2023 doi

  64. [72]

    & Goldstein, T.A Watermark for Large Language Models in Proceedings of the 40th International Conference on Machine Learning International Conference on Machine Learning

    Kirchenbauer, J., Geiping, J., Wen, Y ., Katz, J., Miers, I. & Goldstein, T.A Watermark for Large Language Models in Proceedings of the 40th International Conference on Machine Learning International Conference on Machine Learning. https : / / proceedings . mlr . press / v202 ...

    2023

  65. [73]

    & Goldstein, T.Tree-Rings Watermarks: Invisible Fingerprints for Diffusion Images in Advances in Neural Information Processing Systems 36 (NeurIPS 2023) 36

    Wen, Y ., Kirchenbauer, J., Geiping, J. & Goldstein, T.Tree-Rings Watermarks: Invisible Fingerprints for Diffusion Images in Advances in Neural Information Processing Systems 36 (NeurIPS 2023) 36. https://proceedings. neurips.cc/paper_files/paper/2023/file/b54d1757c190ba20dbc4...

    2023

  66. [74]

    K., Moroni, G., Vaneker, T., Fadel, G., Campbell, R

    Thompson, M. K., Moroni, G., Vaneker, T., Fadel, G., Campbell, R. I., Gibson, I., Bernard, A., Schulz, J., Graf, P., Ahuja, B. & Martina, F. Design for Additive Manufacturing: Trends, Opportunities, Considerations, and Constraints. CIRP Annals 65, 737–760. ISSN : 0007-8506. do...

    2016 doi

  67. [75]

    A., Simpson, T

    Williams, G., Meisel, N. A., Simpson, T. W. & McComb, C. Design for Artificial Intelligence: Proposing a Conceptual Framework Grounded in Data Wrangling. Journal of Computing and Information Science in Engineering 22. ISSN : 1530-9827. doi:10.1115/1.4055854 (2022). 13/15 Preprint

    2022 doi

  68. [76]

    W., Hooker, S., Leung, J., Trask, A., Bluemke, E., Lebensold, J., O’Keefe, C., Koren, M., Ryffel, T., Rubinovitz, J

    Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., Maharaj, T., Koh, P. W., Hooker, S., Leung, J., Trask, A., Bluemke, E., Lebensold, J., O’Keefe, C., Koren, M., Ryffel, T., Rubinovitz, J. B., Besiroglu, T., Ca...

    2020 doi arXiv

  69. [77]

    J., Mackworth, A

    Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., Jirotka, M., Johnson, H., LaPointe, C., Llorens, A. J., Mackworth, A. K., Maple, C., Pálsson, S. E., Pasquale, F., Winfield, A. & Yeong, Z. K. Governing AI...

    2021 doi

  70. [78]

    Mökander, J., Schuett, J., Kirk, H. R. & Floridi, L. Auditing Large Language Models: A Three-Layered Approach. AI and Ethics 4, 1085–1115. ISSN : 2730-5961. doi:10.1007/s43681-023-00289-2 (2024)

    2024 doi

  71. [79]

    & Clark, J

    Whittlestone, J. & Clark, J. Why and How Governments Should Monitor AI Development. doi:10.48550/arXiv. 2108.12427. arXiv: 2108.12427 [cs] (2021). Preprint

    2021 doi arXiv

  72. [80]

    Naihin, S., Atkinson, D., Green, M., Hamadi, M., Swift, C., Schonholtz, D., Kalai, A. T. & Bau, D. Testing Language Model Agents Safely in the Wild. doi:10.48550/arXiv.2311.10538 . arXiv: 2311.10538 [cs] (2023). Preprint

    2023 doi arXiv

  73. [81]

    & Clark, J

    Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y ., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., Jones, A., Bowman, S., Chen, A., Conerly, T., DasSarma, N., Drain, D., Elhage, N., El-Showk, S., Fort, S., Hatfield-Dodds, Z., Henighan, T., Hernandez, D.,...

    2022 doi arXiv

  74. [82]

    & Papernot, N

    Goodfellow, I., McDaniel, P. & Papernot, N. Making Machine Learning Robust against Adversarial Inputs.Commun. ACM 61, 56–66. ISSN : 0001-0782. doi:10.1145/3134599 (2018)

    2018 doi

  75. [83]

    P., Goodfellow, I., Andersen, D

    Gilmer, J., Adams, R. P., Goodfellow, I., Andersen, D. & Dahl, G. E. Motivating the Rules of the Game for Adversarial Example Research. doi:10.48550/arXiv.1807.06732. arXiv: 1807.06732 [cs] (2018). Preprint

    2018 doi arXiv

  76. [84]

    A., Mateos-Garcia, J., Bergman, S., Kay, J., Griffin, C., Bariach, B., Gabriel, I., Rieser, V

    Weidinger, L., Rauh, M., Marchal, N., Manzini, A., Hendricks, L. A., Mateos-Garcia, J., Bergman, S., Kay, J., Griffin, C., Bariach, B., Gabriel, I., Rieser, V . & Isaac, W. Sociotechnical Safety Evaluation of Generative AI Systems. doi:10.48550/arXiv.2310.11986. arXiv: 2310.11...

    2023 doi arXiv

  77. [85]

    F., Leike, J., Brown, T., Martic, M., Legg, S

    Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S. & Amodei, D.Deep Reinforcement Learning from Human Preferences in Advances in Neural Information Processing Systems 30 (Curran Associates, Inc., 2017)

    2017

  78. [86]

    Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D. & Gebru, T. Model Cards for Model Reporting in Proceedings of the Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery, New York, NY...

    2019

  79. [87]

    W., Wallach, H., III, H

    Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D. & Crawford, K. Datasheets for Datasets. Commun. ACM 64, 86–92. ISSN : 0001-0782. doi:10.1145/3458723 (2021)

    2021 doi

  80. [88]

    Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., Wang, T., Marks, S., Segerie, C.-R., Carroll, M., Peng, A., Christoffersen, P., Damani, M., Slocum, S., Anwar, U., Siththaranjan, A., Nadeau, M., Michau...

    2023 doi arXiv

  81. [89]

    B., Mironov, I., Talwar, K

    Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K. & Zhang, L. Deep Learning with Differential Privacy in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (Association for Computing Machinery, New York, NY , USA, 20...

    2016

  82. [90]

    ISSN : 0041-9494 (2023)

    2023

  83. [93]

    Energy System Design

    AI HLEG. Ethics Guidelines for Trustworthy AI https://digital- strategy.ec.europa.eu/en/ library/ethics-guidelines-trustworthy-ai (High-Level Expert Group on AI (AI HLEG) set up by the European Commission, Brussels, 2019). 14/15 Preprint Acknowledgements The authors would like...

    2019

  84. [5546]

    doi:10.1007/BF02639315 (1996)

    1996 doi

  85. [5839]

    doi:10.1038/s42256-022-00440-4 (2022)

    2022 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.