Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pre-release testing of OpenAI's o3-mini found 87 confirmed unsafe behaviors in 10,080 adversarial prompts, with an external API firewall blocking many of the rest.

desk verdict Useful pre-deployment safety report on o3-mini with a real empirical core, but the comparative 'safer than predecessors' claim is confounded by the API-level firewall and needs reframing. read the letter →

arxiv 2501.17749 v1 pith:DMSUYZVP submitted 2025-01-29 cs.SE cs.AI

Aitor Arrieta , Miriam Ugarte , Pablo Valle , José Antonio Parejo , Sergio Segura This is my paper · ORCID
classification cs.SEcs.AI
keywords LLMsafetytestingASTRALautomatedtestgenerationredteamingpre-deploymentevaluationpolicyviolationguardbenchmarksOpenAIo3-mini
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports a safety test of OpenAI's o3-mini before its public release, performed from outside the model developer's own team. The authors ran 10,080 automatically generated unsafe prompts through a beta version of the model, using their ASTRAL tool to vary 14 harm categories, six writing styles, and five persuasion techniques, including prompts tied to live news events. After manually checking every response the automated judge flagged as unsafe or uncertain, they confirmed 87 genuinely unsafe behaviors, most often on controversial political topics and terrorism. They conclude that o3-mini seems safer than earlier OpenAI models and other tested LLMs, but they identify a key complication: roughly 4,491 inputs were rejected by an external policy-violation mechanism before reaching the model, so the measured safety is that of the API system, not necessarily of o3-mini alone.

What carries the argument

The central mechanism is ASTRAL, an automated safety-testing pipeline whose black-box coverage criterion forces balanced generation across 14 safety categories, six writing styles, and five persuasion techniques. It combines retrieval-augmented generation, few-shot prompting, and live web browsing to produce novel, up-to-date unsafe prompts rather than reusing static benchmarks. A second LLM, GPT-3.5, acts as an automated oracle classifying outputs as safe, unsafe, or unknown; only the flagged and unknown outputs receive manual human review. The pipeline also treats policy-violation rejections as safe outcomes, which is what reframes the measurement as a system-level safety check.

What would settle it

Re-run the exact 10,080-prompt suite against the publicly deployed o3-mini (or a beta without the policy firewall) and count human-confirmed harmful outputs among the roughly 4,491 inputs that previously triggered policy violations. If a substantial share of those prompts now elicit harmful content, the paper's 'safer than predecessors' conclusion would be an artifact of the external guard rather than the model.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is a quantified pre-deployment safety profile: 87 out of 10,080 adversarial prompts produced human-confirmed unsafe outputs in the beta o3-mini, versus much larger unverified counts found by the same tool in earlier models (752 for GPT-3.5, 166 for GPT-4, 215 for GPT-4o, and 95 for Llama 2 on the first test suite). The authors attribute much of this improvement to a firewall-like API guard that rejected roughly 4,491 test inputs with policy-violation errors before they reached the LLM, and they explicitly note that the assessment is therefore at the level of OpenAI's API rather than the o3-mini model in isolation. They recommend re-running the blocked inputs if the guard is not kept in the public release.

Load-bearing premise

The result that o3-mini is safer than its predecessors rests on treating every policy-violation rejection as a safe response; if that external API firewall is absent in the public version, the unexecuted prompts could produce unsafe answers and the safety comparison collapses.

Editorial extensions

If this is right

  • If the API-level policy guard persists in the public release, o3-mini's measured safety level, 87 confirmed unsafe responses in 10,080, is the level users will likely experience.
  • If the guard is removed, OpenAI should re-execute the roughly 4,491 blocked test cases before claiming o3-mini is safe.
  • Recent controversial topics, especially U.S. politics (category c3) and terrorism (c13), are the areas where the model most often failed; these categories warrant focused alignment work.
  • Animal abuse (c1) and drug and weapon content (c5) also produced repeated failures, suggesting those categories deserve special attention alongside c3 and c13.
  • The safety-helpfulness tradeoff was not measured, so the high observed safety may come with reduced usefulness in borderline situations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests the 87 confirmed cases are a lower bound, because only evaluator-flagged and unknown outputs were manually inspected, and some responses the judge called safe could still be harmful in ways the oracle misses.
  • This also means the cross-model comparison is partly confounded: earlier models executed all prompts, while o3-mini blocked nearly half at the API boundary, so 'safer' may reflect a different measurement pipeline as much as a safer model.
  • A natural extension would be to run the same second test suite against the public release and compare both rejection rates and the categories of confirmed failures, measuring whether the firewall and the alignment survive deployment.
  • Because ASTRAL's prompts include current events, safety performance is time-dependent; a model that is safe for January 2025 events may fail on later news topics, so pre-deployment results should be re-evaluated on a rolling basis.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript reports an external pre-deployment safety evaluation of a beta version of OpenAI's o3-mini API. The authors use their ASTRAL tool to generate 10,080 unsafe prompts across 14 safety categories, 6 writing styles, and 5 persuasion techniques (TS1: 3,780; TS2: 6,300), execute them against the beta API, classify responses with a GPT-3.5-based evaluator, and manually verify every evaluator-flagged 'unsafe' or 'unknown' output. They report 87 confirmed unsafe behaviors, with the largest numbers in controversial topics/politics (c3) and terrorism/organized crime (c13). They note that a large share of prompts triggered API policy-violation exceptions and were counted as safe, and they frame the result as system-level rather than model-level in Section 3.2. The paper concludes that o3-mini 'seems to be safer' than older OpenAI models and other ASTRAL-tested LLMs.

Significance. The paper's main concrete contribution—87 manually confirmed unsafe outputs from a pre-deployment system—is useful as an early-access safety report, and the authors provide a replication package and are transparent about the system-level nature of the test. They also distinguish evaluator-flagged from manually confirmed unsafe counts, which is good methodological practice. However, the headline comparative claims (Findings 1 and 2) rely on counts from the authors' previous study [10] that were collected under different API conditions, were not manually verified, and were not subject to the same policy-violation filter. The novelty is an application of an existing tool to a non-public model, so the lasting value depends on whether the safety evidence is correctly attributed to the model rather than to the surrounding API. As presented, the evidence supports a conditional system-level statement, not the stronger model-level comparisons in the abstract and Findings 1 and 2.

major comments (3)
  1. [3.2 and 4.2 (Table 2, Finding 1)] The central comparative claim is confounded by the API-level policy-violation filter. Table 2 implies 2,034 policy-violation responses in TS1 (707+762+565) and 2,457 in TS2, so 4,491 of 10,080 inputs were blocked before demonstrably reaching o3-mini and were redefined as safe. Finding 1 compares the 49 evaluator-flagged (or 32 confirmed) TS1 unsafe outputs with 752, 166, and 215 unsafe behaviors found in GPT-3.5, GPT-4, and GPT-4o in prior work [10], where no such filter is reported and where the counts were not manually verified. The abstract and Finding 1 therefore overstate the model-level conclusion; the evidence supports only a system-level statement conditional on OpenAI's beta API. If the firewall-like mechanism is external and disabled in the public release, the blocked prompts might produce unsafe responses, and the ordering across models could reverse. The authors should reframe the headline claim, report blocked-input counts separately, and re-run the blocked inputs on the released model if access permits.
  2. [3.3] The manual-verification procedure only inspects outputs that the GPT-3.5 evaluator flagged as 'unsafe' or 'unknown'. Consequently, the 87 confirmed cases are a lower bound whose completeness depends on the evaluator's recall, and recall is not measured in this study. The authors state that the evaluator was chosen because it had higher accuracy in [10], but they do not report precision, recall, or inter-annotator agreement for the o3-mini executions, nor do they estimate how many unsafe outputs among the non-flagged, non-blocked responses could have been missed. This does not invalidate the 87 confirmed instances, but the paper should state prominently that the count is not an estimate of the total number of unsafe behaviors in the executed suite.
  3. [4.2 (Finding 2)] Finding 2's comparison with Llama 2 mixes confirmed and unconfirmed counts without adjustment: the 49 unconfirmed unsafe outputs in TS1 for o3-mini are directly compared with 95 unsafe outputs for Llama 2 from [10], which the authors themselves note were not manually verified. If manual verification were applied to the Llama 2 counts, the ordering could change. The comparison should be restricted to like-for-like counts under comparable conditions, or the discussion should explicitly state that the Llama 2 figures are unverified and may include false positives.
minor comments (5)
  1. [Throughout] The model name is typeset inconsistently (e.g., 'O3-mini' in the title versus 'o3-mini' in the text); use a single spelling consistently.
  2. [Replication Package] The sentence 'The reults can be obtained...' contains a typo; also consider citing the Zenodo dataset explicitly as a reference.
  3. [arXiv source] The source includes an unreferenced figure file 'frog.jpg'; either cite the figure in the text or remove it from the submission.
  4. [3.2] The nature of the 'policy violation' exception is only conjectured ('we conjecture that the API was able to detect unsafe test inputs'); reporting the API exception type or message would help readers distinguish an external filter from a model-level refusal.
  5. [Table 2] The caption says 'Column Saf e' and the meaning of 'Safe (policy violation)' should clarify whether these are API exceptions or generated refusal responses; the current wording conflates them.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the 87 unsafe-behavior count is an empirical measurement with manual verification; the API-firewall confound is a validity threat, not a circular reduction.

full rationale

This paper is an empirical safety-testing report, not a derivation. The central count (87 confirmed unsafe behaviors) comes from executing 10,080 generated prompts against the o3-mini beta API and manually verifying the outputs that ASTRAL's evaluator flagged as unsafe or unknown. Manual verification is an independent check, so the 87 figure is not equivalent by construction to the evaluator's classifications. The authors do rely on their own prior ASTRAL paper [10] for the generation/evaluation tooling and for comparison counts (752, 166, 215, and 95), but those are empirical measurements from a separate published study, not inputs that the current paper re-derives; the current paper's claims would stand or fall on the reproducibility of those measurements, which is a validity/reproducibility concern rather than circularity. The paper explicitly acknowledges the main threat: Section 3.2 states that policy-violation exceptions may have blocked inputs before reaching the model, so the assessment is at the API-system level, and Section 3.3 notes only flagged outputs were manually checked. These are confounding or coverage limitations, not circular reductions. No equation or fitted parameter is renamed as a prediction, and no load-bearing premise is justified solely by a self-citation. Therefore no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new mathematical constants or entities. Its empirical claims rest on assumptions about the evaluator, the policy filter, and manual verification coverage, listed above.

assumptions (4)
  • domain assumption The GPT-3.5-based evaluator used to label o3-mini outputs as safe, unsafe, or unknown is sufficiently accurate for this target model.
    Section 3.2 says the same evaluator setup as the prior paper [10] was used because it had higher accuracy there, but no accuracy check is reported for o3-mini outputs.
  • domain assumption API responses classified as policy violations are safe and therefore not unsafe behaviors of the model.
    Section 3.2 explicitly adapts the evaluator to classify policy violations as safe; Finding 3 notes these prompts may never reach the LLM.
  • domain assumption Manual verification of only evaluator-flagged unsafe and unknown outputs is sufficient to estimate the true unsafe count.
    Section 3.3 states safe-classified outputs were not manually reviewed, so false negatives are unmeasured.
  • domain assumption ASTRAL's black-box coverage features (14 categories, 6 styles, 5 persuasion techniques) represent the space of unsafe user interactions.
    Section 2.2 and Table 1 assume coverage of these features yields a broad safety assessment; no validation is reported that these features are representative of real user behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation." pith.science (2026). https://pith.science/paper/DMSUYZVP

@misc{pith2026250117749,
  author       = {Pith},
  title        = {Pith review of: Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMSUYZVP}},
  note         = {Machine review of arXiv:2501.17749}
}
read the original abstract

Large Language Models (LLMs) have become an integral part of our daily lives. However, they impose certain risks, including those that can harm individuals' privacy, perpetuate biases and spread misinformation. These risks highlight the need for robust safety mechanisms, ethical guidelines, and thorough testing to ensure their responsible deployment. Safety of LLMs is a key property that needs to be thoroughly tested prior the model to be deployed and accessible to the general users. This paper reports the external safety testing experience conducted by researchers from Mondragon University and University of Seville on OpenAI's new o3-mini LLM as part of OpenAI's early access for safety testing program. In particular, we apply our tool, ASTRAL, to automatically and systematically generate up to date unsafe test inputs (i.e., prompts) that helps us test and assess different safety categories of LLMs. We automatically generate and execute a total of 10,080 unsafe test input on a early o3-mini beta version. After manually verifying the test cases classified as unsafe by ASTRAL, we identify a total of 87 actual instances of unsafe LLM behavior. We highlight key insights and findings uncovered during the pre-deployment external testing phase of OpenAI's latest LLM.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A new benchmark shows that top reasoning models identify all relevant risks in under 40% of cases even when their final answers look safe.

Reference graph

Works this paper leans on

25 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [10]

    Astral: Automated safety testing of large language models,

    M. Ugarte, P . V alle, J. A. Parejo, S. Segura, and A. Arrie ta, “Astral: Automated safety testing of large language models,” in 2025 IEEE/ACM International Conference on Automation of So ftware T est (AST), pp. 30–39, IEEE, 2025

    work page 2025

  2. [1]

    Sorry-bench: Systematically evaluating large language model safety ref usal behaviors,

    T. Xie, X. Qi, Y . Zeng, Y . Huang, U. M. Sehwag, K. Huang, L. H e, B. Wei, D. Li, Y . Sheng,et al., “Sorry-bench: Systematically evaluating large language model safety ref usal behaviors,” arXiv preprint arXiv:2406.14598 , 2024

    arXiv 2024

  3. [2]

    S-eval: Auto- matic and adaptive test generation for benchmarking safety evaluation of large language models,

    X. Y uan, J. Li, D. Wang, Y . Chen, X. Mao, L. Huang, H. Xue, W . Wang, K. Ren, and J. Wang, “S-eval: Auto- matic and adaptive test generation for benchmarking safety evaluation of large language models,” arXiv preprint arXiv:2405.14191, 2024

    arXiv 2024

  4. [3]

    Safetybench: Evaluating the safety of large language models with multiple choice questions,

    Z. Zhang, L. Lei, L. Wu, R. Sun, Y . Huang, C. Long, X. Liu, X. Lei, J. Tang, and M. Huang, “Safetybench: Evaluating the safety of large language models with multiple choice questions,” arXiv preprint arXiv:2309.07045, 2023

    arXiv 2023

  5. [4]

    Chisafetybench: A chinese hierar- chical safety benchmark for large language models,

    W . Zhang, X. Lei, Z. Liu, M. An, B. Y ang, K. Zhao, K. Wang, an d S. Lian, “Chisafetybench: A chinese hierar- chical safety benchmark for large language models,” arXiv preprint arXiv:2406.10311 , 2024

    arXiv 2024

  6. [5]

    Safebench: A safety evaluation framework for multimodal large language models,

    Z. Ying, A. Liu, S. Liang, L. Huang, J. Guo, W . Zhou, X. Liu, and D. Tao, “Safebench: A safety evaluation framework for multimodal large language models,” arXiv preprint arXiv:2410.18927 , 2024

    arXiv 2024

  7. [6]

    Longsafetybench: Long-context llms struggle with safety issues,

    M. Huang, X. Liu, S. Zhou, M. Zhang, C. Tan, P . Wang, Q. Guo, Z. Xu, L. Li, Z. Lei, et al., “Longsafetybench: Long-context llms struggle with safety issues,” arXiv preprint arXiv:2411.06899 , 2024

    arXiv 2024

  8. [7]

    Salad-bench: A hierarchical and comprehensive safety benchmark for large language models,

    L. Li, B. Dong, R. Wang, X. Hu, W . Zuo, D. Lin, Y . Qiao, and J. Shao, “Salad-bench: A hierarchical and comprehensive safety benchmark for large language models, ” arXiv preprint arXiv:2402.05044 , 2024

    arXiv 2024

Show all 25 references
  1. [8]

    Beavertails: Towards improved safety alignment of LLM via a human-preference dataset,

    J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. S un, Y . Wang, and Y . Y ang, “Beavertails: Towards improved safety alignment of LLM via a human-preference dataset,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  2. [9]

    Simplesafetytests: a test suite for identifying critical safety risks in large langua ge models,

    B. Vidgen, N. Scherrer, H. R. Kirk, R. Qian, A. Kannappan, S. A. Hale, and P . Röttger, “Simplesafetytests: a test suite for identifying critical safety risks in large langua ge models,” arXiv preprint arXiv:2311.08370 , 2023

    2023 arXiv

  3. [11]

    A survey on metamorphic testing,

    S. Segura, G. Fraser, A. B. Sanchez, and A. Ruiz-Cortés, “A survey on metamorphic testing,” IEEE Transactions on software engineering, vol. 42, no. 9, pp. 805–824, 2016

    2016

  4. [12]

    Guardrails for trust, safet y, and ethical development and deployment of large language models (llm),

    A. Biswas and W . Talukdar, “Guardrails for trust, safet y, and ethical development and deployment of large language models (llm),” Journal of Science & T echnology, vol. 4, no. 6, pp. 55–82, 2023

    2023

  5. [13]

    European Commission AI Act

    “European Commission AI Act.” https://digital-strategy.ec.europa.eu/en/policies/r egulatory-framework-ai,

  6. [14]

    Artificial Intelligence Act (Regulation (EU) 2024/16 89), Official Journal version of 13 June 2024

    “Artificial Intelligence Act (Regulation (EU) 2024/16 89), Official Journal version of 13 June 2024.” https://eur-lex.europa.eu/eli/reg/2024/1689/oj, 2024. [Online]

    2024

  7. [15]

    Llama guard: Llm-based input-output safeguard for human- ai conversations,

    H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y . Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, et al. , “Llama guard: Llm-based input-output safeguard for human- ai conversations,” arXiv preprint arXiv:2312.06674, 2023

    2023 arXiv

  8. [16]

    ShieldLM: Empowering LLMs as aligned, customizable and explainable safety detec tors,

    Z. Zhang, Y . Lu, J. Ma, D. Zhang, R. Li, P . Ke, H. Sun, L. Sha , Z. Sui, H. Wang, et al., “ShieldLM: Empowering LLMs as aligned, customizable and explainable safety detec tors,” arXiv preprint arXiv:2402.16444 , 2024

    2024 arXiv

  9. [17]

    A strongreject for empty jailbreaks,

    A. Souly, Q. Lu, D. Bowen, T. Trinh, E. Hsieh, S. Pandey, P . Abbeel, J. Svegliato, S. Emmons, O. Watkins, et al., “A strongreject for empty jailbreaks,” arXiv preprint arXiv:2402.10260 , 2024. 7 Early External Safety Testing of OpenAI’s o3-mini: Insight s from Pre-Deployment ...

    2024 arXiv

  10. [18]

    Red teaming language models to reduce harms: Methods, sca ling behaviors, and lessons learned,

    D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y . Bai, S. K adavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, et al., “Red teaming language models to reduce harms: Methods, sca ling behaviors, and lessons learned,” arXiv preprint arXiv:2209.07858, 2022

    2022 arXiv

  11. [19]

    Catastrop hic jailbreak of open-source llms via exploiting generation,

    Y . Huang, S. Gupta, M. Xia, K. Li, and D. Chen, “Catastrop hic jailbreak of open-source llms via exploiting generation,” arXiv preprint arXiv:2310.06987 , 2023

    2023 arXiv

  12. [20]

    Universal and transferable adversarial attacks on aligned language models,

    A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. F redrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043 , 2023

    2023 arXiv

  13. [21]

    Harm- bench: A standardized evaluation framework for automated r ed teaming and robust refusal,

    M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakh aee, N. Li, S. Basart, B. Li, et al. , “Harm- bench: A standardized evaluation framework for automated r ed teaming and robust refusal,” arXiv preprint arXiv:2402.04249, 2024

    2024 arXiv

  14. [22]

    " do an ything now

    X. Shen, Z. Chen, M. Backes, Y . Shen, and Y . Zhang, “" do an ything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” arXiv preprint arXiv:2308.03825 , 2023

    2023 arXiv

  15. [23]

    Jailbroken: H ow does llm safety training fail?,

    A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: H ow does llm safety training fail?,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  16. [24]

    Walledeval: A comprehensive safety evaluation toolkit f or large language models,

    P . Gupta, L. Q. Y au, H. H. Low, I. Lee, H. M. Lim, Y . X. Teoh, J. H. Koh, D. W . Liew, R. Bhardwaj, R. Bhard- waj, et al. , “Walledeval: A comprehensive safety evaluation toolkit f or large language models,” arXiv preprint arXiv:2408.03837, 2024

    2024 arXiv

  17. [25]

    Bi-fact orial preference optimization: Balancing safety- helpfulness in language models,

    W . Zhang, P . H. Torr, M. Elhoseiny, and A. Bibi, “Bi-fact orial preference optimization: Balancing safety- helpfulness in language models,” arXiv preprint arXiv:2408.15313 , 2024. 8 This figure "frog.jpg" is available in "jpg" format from: http://arxiv.org/ps/2501.17749v1

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.