REVIEW 5 cited by
Jailbreaking Large Language Models with Symbolic Mathematics
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advancements in AI safety have led to increased efforts in training and red-teaming large language models (LLMs) to mitigate unsafe content generation. However, these safety mechanisms may not be comprehensive, leaving potential vulnerabilities unexplored. This paper introduces MathPrompt, a novel jailbreaking technique that exploits LLMs' advanced capabilities in symbolic mathematics to bypass their safety mechanisms. By encoding harmful natural language prompts into mathematical problems, we demonstrate a critical vulnerability in current AI safety measures. Our experiments across 13 state-of-the-art LLMs reveal an average attack success rate of 73.6\%, highlighting the inability of existing safety training mechanisms to generalize to mathematically encoded inputs. Analysis of embedding vectors shows a substantial semantic shift between original and encoded prompts, helping explain the attack's success. This work emphasizes the importance of a holistic approach to AI safety, calling for expanded red-teaming efforts to develop robust safeguards across all potential input types and their associated risks.
Forward citations
Cited by 5 Pith papers
-
Whose Refusal Is It? The Unmeasured Contribution of Black-Box Multimodal Guardrails
The reported benefit of a multimodal guardrail is a sum of guard blocks and the target model's own refusals, and evaluation protocol alone can move the guardrail's measured share from 0% to 99%.
-
Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks
Pairing a decoy image with an encoded jailbreak prompt can sharply amplify the safety effect of an image-aware defense (ECSO), at the cost of more benign refusals unless gated by a detector.
-
Should LLM Safety Be More Than Refusing Harmful Instructions?
LLMs that can decrypt common ciphers show safety failures split across two dimensions, refusing too much or generating unsafe output, and current defenses fix one side while breaking the other.
-
Don't Command, Cultivate: An Exploratory Study of System-2 Alignment
Encouraging LLMs to analyze user requests step-by-step (System-2 Alignment) modestly improves safety on open-source models, but with trade-offs and limited evidence.
-
Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation
An iterative LLM-based prompt evaluator blocked 100% of the Best-of-N jailbreaking paper's released successful prompts and 99.8% of a fresh replication, with false-positive rates near zero.
Discussion (0). Continue with ORCID to comment.