Pith.

Cryptography and Security · arXiv:2611.25435 v1 · submitted 19 Nov 2026 · 61 references

VisualJevBench: A Benchmark and Black-Box Attacks for Visual Typed Decision Models

Li Zeng, Yeqi Zeng, Hengkai Liang, Shiguo Wang, Meng Xie, Zeyu Ye, Jinheng Zhang, Xianlong Wang

show 2 more authors

Ziqi Zhou, Hangtao Zhang

The paper says

“Typed decision models (TDMs) return structured decisions instead of generating free-text responses, enabling fast and cheap decisions in real-world applications. Recent support for image inputs extends Jev, a representative TDM, to visual decision-making.”

more of the abstract

We refer to this setting as Visual Jev. However, red teaming of Visual Jev leaves two gaps: a systematic evaluation of direct image manipulations is lacking, and universal visual attacks remain underexplored. To address these gaps, we introduce VisualJevBench, a benchmark for evaluating the adversarial robustness of Visual Jev. It comprises 800 images and 3,200 questions across eight scenarios and four visual attack surfaces: typographic manipulation, visual evidence spoofing, target selection interference, and geometric transformation. We also introduce Adaptive Black-Box Parameter Search (ABPS), a lightweight framework that uses typed-output feedback to find effective image manipulations. ABPS supports both image-specific attacks and universal attacks that transfer to unseen images and tasks in new scenarios without further optimization. Extensive experiments on jev-1.14.0 show that image-specific attacks achieve an attack success rate (ASR) of up to 69.6%, while universal attacks achieve an average ASR of 33.15% on unseen images. Geometric transformations achieve an average ASR of 40.09% in cross-scenario transfer.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 20 Nov 2026 · read the full text (73,825 characters) · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A solid benchmark and black-box attack framework for visual typed decision models, with internally consistent numbers and a careful validity protocol—though the load-bearing answer-preservation audit is in a missing appendix. This line is wrong

9 minor · 2 open questions

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

The paper, first character to last

The reviewer was given the whole text, 73,825 characters. Not checked: figures, code, what the 61 references say.

9 minor objections

The paper says, at Abstract

Pith says: minor objection 1 of 9

The abstract in the manuscript differs slightly from the arXiv metadata. The manuscript abstract includes the sentence 'Our project page is available at...' which is not in the arXiv metadata. This is not a problem, but the authors should ensure consistency.

The paper says, at Sec. 3.1, Visual Jev

Pith says: minor objection 2 of 9

The paper uses the term 'state' to refer to the input to Jev, but the relationship between 'state', 'question instructions', and 'answer criteria' is not fully explained. A concrete example of a request would help readers understand the input format.

show 7 more

The paper says, at Sec. 4.1, Benchmark Construction

Pith says: minor objection 3 of 9

The paper states that images and annotations are drawn from COCO, CV-Bench, GQA, Visual Genome, and BLINK, but does not specify how many images come from each source or how they are distributed across scenarios. This information would be useful for understanding potential biases in the benchmark.

The paper says, at Sec. 5.2, Image-Specific Search

Pith says: minor objection 4 of 9

The description of the 'local mutation' and 'joint mutation' operations is vague. The paper says 'Changes one parameter' and 'Changes a predefined parameter pair', but does not specify how the new value is chosen (e.g., random neighbor, gradient-based). More detail would improve reproducibility.

The paper says, at Sec. 6.4, Defense Evaluation

Pith says: minor objection 5 of 9

The defense evaluation reports interception rates on successful attacks and false positive rates on clean-correct inputs, but does not specify how the defenses are applied (e.g., before or after the attack manipulation). The paper also does not report the impact of defenses on benign accuracy for incorrect inputs (i.e., how many clean-incorrect decisions become correct after defense).

The paper says, at Table 1

Pith says: minor objection 6 of 9

The table caption says 'A dash indicates unsupported or unavailable results.' However, the table does not contain any dashes; instead, some cells are empty (e.g., for SubQ Snap 1.2 and Clef-Flash 9B under BoxType). The authors should clarify whether these are unsupported or simply not reported.

The paper says, at Sec. 6.5, Ablation Study

Pith says: minor objection 7 of 9

The ablation study compares ABPS with variants that remove feedback, joint mutation/recombination, and exploration. However, it does not include a comparison with random search under the same query budget, which would be a more natural baseline. The paper mentions that such a comparison is in Appendix C, but Appendix C was not found in the supplied text.

The paper says, at Sec. 7, Conclusion

Pith says: minor objection 8 of 9

The conclusion states 'Existing image-level defenses offer only partial protection.' This is supported by the defense evaluation, but the paper does not discuss the implications of the defense results or suggest directions for improving defenses. A brief discussion would strengthen the conclusion.

The paper says, at Throughout

Pith says: minor objection 9 of 9

The paper uses many custom color definitions and formatting commands that are not essential for understanding. Some of these may cause issues in different rendering environments. The authors should ensure that the paper is readable in plain black-and-white format.

Open questions (2)

Pith raised these and could not confirm them against the text of the paper it read. They are questions for the authors, not findings.

  1. Sec. 4.3, Attack Validity Can the authors provide the audit protocol, sample counts, and results from Appendix B to verify that the reported ASR values are not inflated by manipulations that change the correct answer?
  2. Sec. 6.1, Experimental Setup Can the authors provide the exact benign flip rates and the number of clean-correct instances per question type to help interpret the ASR values?

What would settle it

If an independent audit of the stratified sample of successful attacks (Appendix B, not provided in the supplied text) finds that more than a small fraction of retained manipulations change the correct answer or remove required visual evidence, the reported ASR values would be inflated and the central claim of answer-preserving attacks would be undermined.

from Pith's summary of the paper

How the review went

5 passes by deepseek-v4-flash · 20 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic concern not confirmed: listed as an open question 9 s · 24,803 tokens read, 602 written
  3. Referee 9 minor 20 s · 25,061 tokens read, 1,961 written
  4. Check against the paper kept 0 of 5 objections 51 s · 51,778 tokens read, 5,345 written
  5. Editor wrote the verdict above 7 s · 25,559 tokens read, 452 written

Not run on this paper: the simulated authors' defence.

Pith's passes raised 5 objections. A check against the paper's text kept 0, withdrew 2 (2 Pith's reviewer misread the paper) and left 3 as questions.

The review in full

The review in brief, machine-written

This paper introduces VisualJevBench, a benchmark of 800 images and 3,200 questions across four attack surfaces (typographic, evidence spoofing, target interference, geometric) for evaluating visual typed decision models like Jev. It also proposes ABPS, a black-box parameter search that uses typed-output feedback to find image-specific and universal manipulations. The numbers are internally consistent: the reported universal mean of 33.15% matches the per-surface values, the ablation averages check out, and the protocol counts an attack only when a clean-correct decision flips, with benign flip rates stated as a control. The paper does what it sets out to do: it gives red-teamers a reusable benchmark and a query-efficient search method, and it shows that fixed manipulations transfer across images and scenarios to a non-trivial degree.

The soft spots are real but bounded. The central validity check—that manipulations preserve the correct answer—is verified only on a stratified sample, with the audit protocol, sample counts, and results promised in Appendix B, which is not in the supplied text. The ASR numbers come without error bars, and the cross-scenario 40.09% figure sits in a figure I could not see. These are the items to resolve before the headline vulnerability numbers are relied upon. The paper acknowledges its own limitations (predefined manipulation families, non-adaptive defense testing), which is appropriate.

Who this is for: researchers working on adversarial robustness of structured-output models, and practitioners deploying typed decision models in visual settings. The paper deserves a serious referee. The benchmark and the ABPS method are new relative to prior work on Jev (JevAdvBench, FlipJev) and on visual attacks against vision-language models. The claims are defensible given what is shown, but the missing appendix and lack of error bars should be addressed in revision. I would send this to peer review.

The stress test's concern

Not confirmed. Pith could not check this concern against the text of the paper it was given; it is listed as an open question, not a finding.

The referee's report

Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 4 of its 4 major objections did not survive that check.

Pith's reading of the paper (a machine's summary, not the authors')

The paper introduces VisualJevBench, a benchmark for evaluating the adversarial robustness of visual typed decision models (TDMs) like Jev, which return structured decisions (binary, choice, score, bounding box) rather than free text. The authors also propose Adaptive Black-Box Parameter Search (ABPS), a lightweight framework that uses typed-output feedback to find effective image manipulations. Experiments on Jev-1.14.0 and eight other models show that image-specific attacks achieve up to 69.6% attack success rate (ASR), universal attacks frozen on training images achieve 33.15% average ASR on unseen images, and geometric transformations transfer across scenarios with 40.09% average ASR. The paper argues that visual TDMs are vulnerable to direct image manipulations that preserve the correct semantic answer, and that these vulnerabilities generalize across images and scenarios.

What carries the argument. The central object is the Adaptive Black-Box Parameter Search (ABPS) framework, which uses typed-output feedback (probabilities for Noul/Choice, score deviation for Score, IoU for BoxType) to guide mutation, recombination, and exploration of manipulation parameters. The benchmark VisualJevBench provides the testbed with 800 images and 3,200 questions across eight scenarios and four attack surfaces. The attack validity constraint ensures manipulations preserve the correct semantic answer, verified by auditing a stratified sample of successful attacks.

Core claim. The paper claims that visual typed decision models like Jev are vulnerable to direct image manipulations that preserve the correct semantic answer but flip correct decisions to incorrect ones. Using the proposed ABPS framework, image-specific attacks achieve up to 69.6% ASR on Jev-1.14.0, and universal manipulations frozen on training images achieve 33.15% average ASR on unseen images without further optimization. Geometric transformations achieve 40.09% average ASR in cross-scenario transfer. The benchmark covers four attack surfaces (typographic manipulation, visual evidence spoofing, target selection interference, geometric transformation) across four question types (Noul, Choice, Score, BoxType) on 800 images and 3,200 questions.

The premise it rests on. The attack success rate numbers depend on the audit of a stratified sample of successful attacks to verify that manipulations preserve the correct semantic answer; if a non-negligible share of retained manipulations actually changes the question-relevant content, the ASR overstates true vulnerability.

If the paper is right (Pith's inferences, not the authors')

If the paper's claims hold, visual typed decision models used in content moderation, agent control, and other applications can be reliably flipped by image manipulations that leave the correct answer unchanged, undermining trust in their structured outputs.

Universal attacks that transfer to unseen images without per-image optimization suggest that a single manipulation can be effective across many inputs, reducing the cost of adversarial exploitation.

Geometric transformations transferring across scenarios (40.09% average ASR) implies that spatial distortions are a broadly effective attack vector regardless of domain content.

Existing image-level defenses (CLIP filtering, JPEG compression, median filtering) intercept fewer than 9% of attacks on average, indicating that current protections are insufficient.

The ABPS framework's query efficiency (7.2 queries per image on average) makes black-box attacks practical even with limited API budgets.

The benchmark's reliance on manual annotation and review of reference answers (from COCO, GQA, Visual Genome, BLINK) means its quality depends on human judgment; independent replication of the audit protocol would strengthen confidence in the ASR numbers.

The paper evaluates only predefined manipulation families and non-adaptive defenses; adaptive defenses that learn from attack patterns could potentially reduce ASR, but the paper does not test this.

The universal attack success rates (33.15% average) suggest that visual TDMs may have systematic weaknesses that are independent of image content, which could be exploited by adversaries with limited resources.

The paper's focus on Jev and similar models leaves open whether other typed decision models with different architectures or training data would show similar vulnerabilities.

Check this review

What was read

The passes were given all 73,825 characters of the paper's LaTeX source.

Fingerprint of that text (SHA-256): 25e5d9b8c836d71bca87ff544d902e69ed0fefa73c1698e4417d9aaf4959f1aa

Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:

PYTHONPATH=. python scripts/review_input_check.py 2611.25435

The passes, in the order they ran

  1. Reader deepseek-v4-flash · medium effort · $0.0067 · instructions of the Reader, fingerprint f9ead7300382…
  2. Skeptic deepseek-v4-flash · medium effort · $0.0024 · instructions of the Skeptic, fingerprint 8f88c0f61488…
  3. Referee deepseek-v4-flash · medium effort · $0.0026 · instructions of the Referee, fingerprint a852d99aea0d…
  4. Check against the paper deepseek-v4-flash · medium effort · $0.0038 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
  5. Editor deepseek-v4-flash · medium effort · $0.0024 · instructions of the Editor, fingerprint e7211f6a3d27…
  6. Circularity check deepseek-v4-flash · medium effort · $0.0023 · instructions of the Circularity check, fingerprint e9946e39d67b…
  7. Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0027 · instructions of the Summary for the page, fingerprint e4f56bf568e7…

The passes cost $0.023 in model calls together.

Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00

Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.

curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum

prints the fingerprint of the reader's instructions. The same works for every pass.

The receipt

The review's fingerprint was taken on 20 Nov 2026. The review you are reading still matches it.

Fingerprint of the review (SHA-256): 8d0eb71dc1e81ad65e8379ea95dd8ea0268ab1a12430451025d6e4e3e705b0e2

curl -s 'https://pith.science/api/v1/verdicts/2611.25435?form=canonical' | sha256sum

prints the fingerprint of the review as it stands now. The rule, in four steps.

A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, from the paper's LaTeX source.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Typed decision models (TDMs) return structured decisions instead of generating free-text responses, enabling fast and cheap decisions in real-world applications. Recent support for image inputs extends Jev, a representative TDM, to visual decision-making. We refer to this setting as Visual Jev. However, red teaming of Visual Jev leaves two gaps: a systematic evaluation of direct image manipulations is lacking, and universal visual attacks remain underexplored. To address these gaps, we introduce VisualJevBench, a benchmark for evaluating the adversarial robustness of Visual Jev. It comprises 800 images and 3,200 questions across eight scenarios and four visual attack surfaces: typographic manipulation, visual evidence spoofing, target selection interference, and geometric transformation. We also introduce Adaptive Black-Box Parameter Search (ABPS), a lightweight framework that uses typed-output feedback to find effective image manipulations. ABPS supports both image-specific attacks and universal attacks that transfer to unseen images and tasks in new scenarios without further optimization. Extensive experiments on jev-1.14.0 show that image-specific attacks achieve an attack success rate (ASR) of up to 69.6%, while universal attacks achieve an average ASR of 33.15% on unseen images. Geometric transformations achieve an average ASR of 40.09% in cross-scenario transfer.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-20

The authors' main claim, as the classifier picked it out 0.99

“Extensive experiments on classifier-1.14.0 show that image-specific attacks achieve an attack success rate (ASR) of up to 69.6%, while universal attacks achieve an average ASR of 33.15% on unseen images.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.21 · human participants 0.04 · controlled comparison 0.13 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.13 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Typed decision models (TDMs) return structured decisions instead of generating free-text responses, enabling fast and cheap decisions in real-world applications. background 1.00 0.05 0.02 asserted 1.00
Recent support for image inputs extends Jev, a representative TDM, to visual decision-making. background 1.00 0.08 0.03 asserted 1.00
We refer to this setting as Visual Jev. background 0.90 0.32 0.02 asserted 0.95
However, red teaming of Visual Jev leaves two gaps: a systematic evaluation of direct image manipulations is lacking, and universal visual attacks remain underexplored. gap 1.00 0.08 0.03 asserted 1.00
To address these gaps, we introduce VisualJevBench, a benchmark for evaluating the adversarial robustness of Visual Jev. contribution 1.00 0.89 0.06 asserted 0.99
It comprises 800 images and 3,200 questions across eight scenarios and four visual attack surfaces: typographic manipulation, visual evidence spoofing, target selection interference, and geometric transformation. method 0.71 0.85 0.04 asserted 1.00
We also introduce Adaptive Black-Box Parameter Search (ABPS), a lightweight framework that uses typed-output feedback to find effective image manipulations. method 0.64 0.93 0.04 asserted 1.00
ABPS supports both image-specific attacks and universal attacks that transfer to unseen images and tasks in new scenarios without further optimization. method 0.81 0.86 0.09 asserted 1.00
Extensive experiments on jev-1.14.0 show that image-specific attacks achieve an attack success rate (ASR) of up to 69.6%, while universal attacks achieve an average ASR of 33.15% on unseen images. result 1.00 0.96 0.48 asserted 1.00
Geometric transformations achieve an average ASR of 40.09% in cross-scenario transfer. result 1.00 0.94 0.12 asserted 1.00

What kind of paper the classifier reports this is: dataset or benchmark 0.98 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-20: 1 request, 7,571 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (5)
Figure 1 of the paper
Figure 1. Overview of VISUALJEVBENCH across the four question types (columns), with task prompts, ground-truth answers, and attack targets shown at the bottom. Rows show benign images and variants under typographic manipulation, visual evidence spoofing, target selection interference, and geometric transformation. In the BOX column, boxes are overlaid for visualization only: the red box (Benign) marks the ground-truth answer, while blue boxes (attacked) show misled predictions. Abstract Typed decision models (TDMs) return structured decisions instead of generating free-text responses,…
Figure 2 of the paper
Figure 2. Overview of VISUALJEVBENCH. the manipulated image as x ′ = Tm(x; θ), where Tm ap￾plies manipulation operator m to image x with parameters θ (e.g., text placement or rotation angle). The operators be￾long to the four attack surfaces summarized in
Figure 3 of the paper
Figure 3. Overview of Adaptive Black-Box Parameter Search (A
Figure 4 of the paper
Figure 4. Cross-scenario ASR (%) across four attack surfaces with
Figure 5 of the paper
Figure 5. Six defenses on Visual Jev: (a) interception rate on suc

Figures are the authors', reproduced from the paper.

References (61)
  1. [1] Maksym Andriushchenko, Francesco Croce, Nicolas Flam- marion, and Matthias Hein. Square attack: A query-efficient black-box adversarial attack via random search. InPro- ceedings of the European Conference on Computer Vision (ECCV), pages 484–501, 2020. 3 on Pith
  2. [2] Seyedarmin Azizi, Erfan Baghaei Potraghloo, and Massoud Pedram. One word opens the gate: The option-channel attack on typed decision models as agent guardrails.arXiv preprint arXiv:2610.12292, 2026. 3 on PithPith's reviewarXiv
  3. [3] Nitin Aravind Birur, Divyanshu Kumar, Sahil Agarwal, and Prashanth Harshangi. JevBreak: Your model, their decision. arXiv preprint arXiv:2611.03759, 2026. 3 on PithPith's reviewarXiv
  4. [4] Bynn Intelligence. DecisionNode: Model documentation. https://bynn.com/docs, 2026. Official documen- tation. Evaluated model: decisionnode-main-0.9.1-rc1. Ac- cessed November 16, 2026. 6, 7 on Pith
  5. [5] Yue Cao, Yun Xing, Jie Zhang, Di Lin, Tianwei Zhang, Ivor Tsang, Yang Liu, and Qing Guo. SceneTAP: Scene- coherent typographic adversarial planner against vision- language models in real-world environments. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Patt... on Pith
  6. [6] Hao Cheng, Erjia Xiao, Jindong Gu, Le Yang, Jinhao Duan, Jize Zhang, Jiahang Cao, Kaidi Xu, and Renjing Xu. Unveil- ing typographic deceptions: Insights of the typographic vul- nerability in large vision-language models. InProceedings of the European Conference on Computer Vis... on Pith
  7. [7] Cloudflare. Clef-Flash: Model card.https : / / huggingface . co / Cloudflare / clef - flash, on Pith
  8. [8] CompVis. Stable diffusion safety checker.https:// huggingface.co/CompVis/stable-diffusion- safety-checker, 2022. Hugging Face model repository. Accessed November 16, 2026. 8 on Pith
  9. [9] Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. BLINK: Multimodal large language models can see but not perceive. InEuropean Conference on Com- puter Vision, 2024. 4 on Pith
  10. [10] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using input transformations. InInternational Conference on Learning Representations, 2018. 8 on Pith
  11. [11] Aleksandr Gushchin, Khaled Abud, Georgii Bychkov, Eka- terina Shumitskaya, Anna Chistyakova, Sergey Lavrushkin, Bader Rasheed, Kirill Malyshev, Dmitriy S. Vatolin, and Anastasia Antsiferova. Guardians of image quality: Bench- marking defenses against adversarial attacks on ima... on Pith
  12. [12] H2O.ai. H2O-Lightning-4B.https://huggingface. co/h2oai/h2o- lightning- 4b, 2026. Model card. Accessed November 17, 2026. 6, 7 on Pith
  13. [13] Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, and Leo Yu Zhang. JevAd- vBench: A benchmark and black-box attacks for reinforce- ment learning for calibrated decisions models.arXiv preprint arXiv:2609.31142, 2026. 3, 6 on PithPith's reviewarXiv
  14. [14] Drew A. Hudson and Christopher D. Manning. GQA: A new dataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 6700–6709, 2019. 4 on Pith
  15. [15] Yerin Hwang, Dongryeol Lee, Kyungmin Min, Taegwan Kang, Yongil Kim, and Kyomin Jung. Fooling the LVLM judges: Visual biases in LVLM-based evaluation. InPro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23186–23205, 2025. 3 on Pith
  16. [16] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations.I... on Pith
  17. [17] Keyi Li, Yihao He, and Quanyi Li. Beyond calibration: Do a typed-decision model’s probabilities obey the probability axioms?arXiv preprint arXiv:2609.33209, 2026. 3 on PithPith's reviewarXiv
  18. [18] Yubo Li, Yidi Miao, Ramayya Krishnan, and Rema Padman. JEV-as-a-Judge: Accept when confident, escalate when un- sure.arXiv preprint arXiv:2609.26550, 2026. 2 9 on PithPith's reviewarXiv
  19. [19] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In European Conference on Computer Vision, pages 740–755, on Pith
  20. [20] Xiaogeng Liu, Xinyan Wang, and Chaowei Xiao. Flip- Jev: Fast black-box red teaming finds universal adversar- ial suffixes for typed decision models.arXiv preprint arXiv:2610.38895, 2026. 3 on PithPith's reviewarXiv
show 41 more
  1. [21] Fabian Lukassen, Christoph Weisser, and Alexander Silbers- dorff. Cheap, consistent, calibrated, but biased? typed de- cision models as judges.arXiv preprint arXiv:2610.18122, on PithPith's reviewarXiv
  2. [22] Federico Mazzoni and Andrea Failla. Benchmarking sys- tem one models in online moderation.arXiv preprint arXiv:2610.07953, 2026. 2, 3 on PithPith's reviewarXiv
  3. [23] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturba- tions. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1765–1773, on Pith
  4. [24] Zaid Mukaddam. Amos 1 Large. Hugging Face model card, on Pith
  5. [25] Nace.AI. Drex models: Drex Max v1.1.https : / / console . nace . ai / docs / drex / reference / models, 2026. Official documentation. Accessed Novem- ber 17, 2026. 6, 7 on Pith
  6. [26] Accessed: 2026-11-17. 6, 7 on Pith
  7. [27] Maan Qraitem, Piotr Teterwak, Kate Saenko, and Bryan A. Plummer. Web artifact attacks disrupt vision language mod- els. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 1048–1057, 2025. 3 on Pith
  8. [28] Maan Qraitem, Nazia Tasnim, Piotr Teterwak, Kate Saenko, and Bryan A. Plummer. Vision-llms can fool themselves with self-generated text. InNeurIPS 2024 Workshop on Mul- timodal Algorithmic Reasoning, 2024. 3 on Pith
  9. [29] Rahul Sharma, Andrew B. Duncan, Ga ´etan Marceau Caron, and Sebastian J. V ollmer. TypedBench: A benchmark for cal- ibration, framing sensitivity, and cost in system one decision models.arXiv preprint arXiv:2610.11392, 2026. 2 on PithPith's reviewarXiv
  10. [30] Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and in turn reflecting on inappropriate content? InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. ACM, 2022. 8 on Pith
  11. [31] Yufei Song, Ziqi Zhou, Qi Lu, Hangtao Zhang, Yifan Hu, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. Segtrans: Transferable adversarial examples for segmenta- tion models.IEEE Transactions on Multimedia, 2026. 3 on Pith
  12. [32] Yufei Song, Ziqi Zhou, Minghui Li, Xianlong Wang, Hang- tao Zhang, Menghao Deng, Wei Wan, Shengshan Hu, and Leo Yu Zhang. Pb-uap: Hybride universal adversarial at- tack for image segmentation. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Pro... on Pith
  13. [33] Lijuan Tang and Yuemeng Zheng. Typed decision mod- els: An early evidence audit and evaluation checklist.arXiv preprint arXiv:2609.32160, 2026. 2 on PithPith's reviewarXiv
  14. [34] Subquadratic. SubQ Snap 1.2: Order-robust decisions and higher long-context accuracy. Official release announce- ment, 2026. Accessed November 16, 2026. 6, 7 on Pith
  15. [35] TypeSafe AI. TypeSafe Documentation.https://docs. typesafe.ai/. Accessed November 11, 2026. 2, 3, 6 on Pith
  16. [36] Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian- 1: A fully open, vision-centric exploration of multimodal LLMs. InAd... on Pith
  17. [37] Valen Team. Valen: Models, datasets, and demos. Official documentation, 2026. Accessed November 16, 2026. 6, 7 on Pith
  18. [38] TypeSafe AI. Jev: For the builders AND dreamers. Official release announcement, 2026. Jev version 1.14.0. Accessed: 2026-11-17. 7 on Pith
  19. [39] Junhui Wang, Hangtao Zhang, Zhirun Zheng, Li Zeng, Jiejun Xiao, Xi Luo, Lihua Yin, and Saiqin Long. Pvdetector: Detecting prompt injection attacks on purpose-specific llm agents through policy-violation concept analysis. InPro- ceedings of the 34th ACM International Conference... on Pith
  20. [40] vLLM Semantic Router. d3-ultra: Model card. Hugging Face, 2026. Accessed November 16, 2026. 6, 7 on Pith
  21. [41] Xianlong Wang, Hewen Pan, Hangtao Zhang, Minghui Li, Shengshan Hu, Ziqi Zhou, Lulu Xue, Aishan Liu, Yunpeng Jiang, Leo Yu Zhang, et al. Trojanrobot: Physical-world backdoor attacks against vlm-based robotic manipulation. arXiv preprint arXiv:2411.11683, 2024. 11 on PitharXiv
  22. [42] Xianlong Wang, Minghui Li, Wei Liu, Hangtao Zhang, Shengshan Hu, Yechao Zhang, Ziqi Zhou, and Hai Jin. Un- learnable 3d point clouds: Class-wise transformation is all you need.Advances in Neural Information Processing Sys- tems, 37:99404–99432, 2024. 11 on Pith
  23. [43] Yichen Wang, Yuxuan Chou, Ziqi Zhou, Hangtao Zhang, Wei Wan, Shengshan Hu, and Minghui Li. Breaking barriers in physical-world adversarial examples: Improving robust- ness and transferability via robust feature. InProceedings of the AAAI Conference on Artificial Intelligence, ... on Pith
  24. [44] Xianlong Wang, Hangtao Zhang, Wenbo Pan, Ziqi Zhou, Changsong Jiang, Li Zeng, and Xiaohua Jia. Dual- branch robust unlearnable examples.arXiv preprint arXiv:2605.01718, 2026. 11 on PithPith's reviewarXiv
  25. [45] Tiantong Wu and Wei Yang Bryan Lim. Decision hijacking: Prompt injection attacks on Jev’s typed probabilistic deci- sions.arXiv preprint arXiv:2609.28613, 2026. 3 on PithPith's reviewarXiv
  26. [46] Yichen Wang, Hangtao Zhang, Hewen Pan, Ziqi Zhou, Xian- long Wang, Peijin Guo, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. Advedm: Fine-grained adversarial at- tack against vlm-based embodied agents.Advances in Neu- ral Information Processing Systems, 38:136551–136575, on Pith
  27. [47] Meng Xie, Li Zeng, Hangtao Zhang, Xianlong Wang, Ziqi Zhou, Pengpeng Qiao, and Zhetao Li. Typo: Instruction- dense visual jailbreaks against commercial closed-source image-generation models.arXiv preprint arXiv:2607.24897, on PithPith's reviewarXiv
  28. [48] Tiantong Wu and Wei Yang Bryan Lim. REFLEX with Jev for efficient selective control in LLM agents.arXiv preprint arXiv:2609.26532, 2026. 2, 3 10 on PithPith's reviewarXiv
  29. [49] Zixiang Xu, Zirui Song, Chiyu Zhang, Xiuying Chen, Xi Liu, Xiyang Hu, and Yue Zhao. JevOut: Natural context can flip decision models.arXiv preprint arXiv:2609.30243, on PithPith's reviewarXiv
  30. [50] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Proceedings of the Network and Distributed System Security Symposium. Internet Society, 2018. 8 on Pith
  31. [51] Zeyu Ye, Yanchun Li, Sibei He, Meng Xie, Hangtao Zhang, Xianlong Wang, Li Zeng, Jiahao Chen, Yichen Wang, Junhui Wang, et al. False claims, credible images: A red-teaming benchmark for commercial image generators.arXiv preprint arXiv:2610.11112, 2026. 2 on PithPith's reviewarXiv
  32. [52] Zeming Yao, Hangtao Zhang, Yicheng Guo, Xin Tian, Wei Peng, Yi Zou, Leo Yu Zhang, and Chao Chen. Reverse back- door distillation: Towards online backdoor attack detection for deep neural network models.IEEE Transactions on De- pendable and Secure Computing, 21(6):5098–5111, 2024. 11 on Pith
  33. [53] Li Zeng, Zeyu Ye, Meng Xie, Hangtao Zhang, Xianlong Wang, Yanchun Li, and Zhetao Li. Ghostprompt: Cross- image adversarial prompt for vision-language models. In Proceedings of the 34th ACM International Conference on Multimedia, 2026. 3 on Pith
  34. [54] Li Zeng, Xiaojun Mo, Meng Xie, Hangtao Zhang, Yixiang Liu, Yezhuo Peng, and Yanchun Li. Psfd: Proactive spatial- frequency defense against malicious exemplar-guided image editing. In2025 IEEE International Conference on Multime- dia and Expo (ICME), pages 1–6. IEEE, 2025. 11 on Pith
  35. [55] Hangtao Zhang, Yichen Wang, Shihui Yan, Chenyu Zhu, Ziqi Zhou, Linshan Hou, Shengshan Hu, Minghui Li, Yan- jun Zhang, and Leo Yu Zhang. Test-time backdoor detection for object detection models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 2437... on Pith
  36. [56] Hangtao Zhang, Shengshan Hu, Yichen Wang, Leo Yu Zhang, Ziqi Zhou, Xianlong Wang, Yanjun Zhang, and Chao Chen. Detector collapse: Backdooring object detection to catastrophic overload or blindness in the physical world. In Proceedings of the Thirty-Third International Joint Co... on Pith
  37. [57] Hangtao Zhang, Yucheng Zhao, Sishun Liu, Ziqi Zhou, Zeyu Ye, Wei Wan, Minghui Li, Shengshan Hu, Yanjun Zhang, Yi Liu, and Leo Yu Zhang. Defending jailbreak attacks on large language models via manifold trajectory kinetics. In35th USENIX Security Symposium (USENIX Security 26),... on Pith
  38. [58] Hangtao Zhang, Chenyu Zhu, Xianlong Wang, Ziqi Zhou, Changgan Yin, Minghui Li, Lulu Xue, Yichen Wang, Sheng- shan Hu, Aishan Liu, et al. Badrobot: Jailbreaking embod- ied llm agents in the physical world. InProceedings of the Thirteenth International Conference on Learning Rep... on Pith
  39. [60] Ziqi Zhou, Menghao Deng, Yufei Song, Hangtao Zhang, Wei Wan, Shengshan Hu, Minghui Li, Leo Yu Zhang, and Dezhong Yao. Darkhash: A data-free backdoor attack against deep hashing.IEEE Transactions on Information Forensics and Security, 2025. 11 on Pith
  40. [61] Additional Related Work Adversarial robustness of visual models.Prior work has investigated the robustness and security of visual models under a variety of adversarial settings. Existing studies cover defenses against malicious image manipulation [52], adversarial vulnerabilit... on Pith
  41. [2026] Hugging Face. Accessed November 16, 2026. 6, 7 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment