OTI: A Model-free and Visually Interpretable Measure of Image Attackability

Chi-Man Pun; Haowei Liu; Jiaming Liang

arxiv: 2601.17536 · v1 · submitted 2026-01-24 · 💻 cs.CV · cs.LG

OTI: A Model-free and Visually Interpretable Measure of Image Attackability

Jiaming Liang , Haowei Liu , Chi-Man Pun This is my paper

Pith reviewed 2026-05-16 11:06 UTC · model grok-4.3

classification 💻 cs.CV cs.LG

keywords adversarial attackabilitytexture intensitysemantic objectmodel-free measurevisual interpretabilityadversarial perturbationsneural network robustnessimage vulnerability

0 comments

The pith

Texture intensity of an image's semantic object predicts its vulnerability to adversarial attacks without needing any model.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Object Texture Intensity (OTI) as a direct measure of how easily an image can be altered by small adversarial perturbations to fool neural networks. It defines attackability through the texture strength inside the main semantic object rather than through gradients or other model outputs. This removes the need for access to task-specific models and makes the score visually readable by inspecting the object's texture patterns. The approach rests on links between texture, decision boundaries, and the frequency content typical of adversarial changes. Experiments confirm OTI correlates with attack success rates while remaining fast to compute.

Core claim

OTI quantifies image attackability as the texture intensity of the image's semantic object, providing a model-free and visually interpretable score grounded in the principles that adversarial perturbations exploit mid- and high-frequency components near decision boundaries.

What carries the argument

Object Texture Intensity (OTI), which extracts the semantic object and measures its texture intensity to score attackability directly from the image.

If this is right

OTI can guide active learning by prioritizing high-texture images for labeling.
It supports adversarial training by flagging the most vulnerable samples in advance.
Attack generation can target high-OTI regions to improve success rates.
The measure runs quickly enough for large-scale dataset screening.
Visual maps of object texture give a direct picture of why certain images are easier to attack.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

OTI may extend to measuring robustness against other image corruptions such as noise or blur.
Object extraction steps could be replaced by simpler saliency methods if they preserve the texture signal.
High-OTI images might serve as natural test cases for evaluating new defense techniques.
The frequency-based explanation suggests texture could inform the design of frequency-aware training procedures.

Load-bearing premise

Texture intensity inside the identified semantic object reliably indicates attackability no matter which model or attack method is used.

What would settle it

Run OTI on a dataset, then apply multiple standard attacks to high- and low-OTI images across several models; if attack success rates show no consistent difference between the two groups, the measure fails.

read the original abstract

Despite the tremendous success of neural networks, benign images can be corrupted by adversarial perturbations to deceive these models. Intriguingly, images differ in their attackability. Specifically, given an attack configuration, some images are easily corrupted, whereas others are more resistant. Evaluating image attackability has important applications in active learning, adversarial training, and attack enhancement. This prompts a growing interest in developing attackability measures. However, existing methods are scarce and suffer from two major limitations: (1) They rely on a model proxy to provide prior knowledge (e.g., gradients or minimal perturbation) to extract model-dependent image features. Unfortunately, in practice, many task-specific models are not readily accessible. (2) Extracted features characterizing image attackability lack visual interpretability, obscuring their direct relationship with the images. To address these, we propose a novel Object Texture Intensity (OTI), a model-free and visually interpretable measure of image attackability, which measures image attackability as the texture intensity of the image's semantic object. Theoretically, we describe the principles of OTI from the perspectives of decision boundaries as well as the mid- and high-frequency characteristics of adversarial perturbations. Comprehensive experiments demonstrate that OTI is effective and computationally efficient. In addition, our OTI provides the adversarial machine learning community with a visual understanding of attackability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

OTI defines attackability via texture intensity inside the semantic object and claims this is model-free plus visually readable, but the model-free part depends on how the object mask is obtained.

read the letter

The paper's core move is to score image attackability by the texture strength of the main semantic object rather than by gradients or minimal perturbations from a target model. This directly targets the two gaps they flag: reliance on inaccessible model proxies and lack of visual meaning in the resulting features. They tie the score to decision-boundary geometry and the mid- and high-frequency nature of typical adversarial noise, then report that it tracks attack success rates while staying cheap to compute. That combination is useful for active learning or attack design where you want a fast ranking without running the full attack pipeline each time. The visual link is also a genuine step forward; you can look at an image and roughly understand why its OTI value is high or low. Experiments are described as covering multiple attacks and datasets, which gives the claim some empirical grounding. The main soft spot is the model-free assertion. Extracting a semantic object mask in practice almost always requires a segmentation network, and any such network carries its own training data and biases. The paper needs to show that OTI-attackability correlations remain stable when the segmenter is swapped; if all results use one fixed mask generator, the independence claim weakens. The theoretical section also stays at the level of principles rather than explicit derivations, so a referee would want the precise definition of texture intensity and any supporting steps. This is aimed at adversarial-ML practitioners who need a lightweight, interpretable tool for dataset curation or attack selection rather than theorists seeking new bounds. The experiments look substantial enough on the surface to merit peer review, though the model-free status and the exact texture calculation are the parts that need the closest check.

Referee Report

2 major / 2 minor

Summary. The paper proposes Object Texture Intensity (OTI) as a model-free and visually interpretable measure of image attackability, defined directly as the texture intensity within the semantic object region of an image. It provides theoretical grounding in decision-boundary geometry and the mid- and high-frequency statistics of adversarial perturbations, and reports that comprehensive experiments confirm OTI is effective and computationally efficient for applications such as active learning and adversarial training.

Significance. If the model-free property and predictive correlation hold, OTI would address two documented limitations of prior attackability measures by eliminating reliance on model proxies (gradients or minimal perturbations) and by supplying direct visual interpretability. This could enable broader use in settings where task-specific models are unavailable and would give the community a concrete visual handle on why certain images are more vulnerable than others.

major comments (2)

[Abstract / Method] Abstract and method section: The central claim that OTI is strictly model-free rests on the assumption that the semantic-object mask can be obtained without any learned model. In practice, semantic segmentation almost always employs a pre-trained network (SAM, Mask R-CNN, etc.). The manuscript must explicitly state the exact procedure used to produce the mask and must include an invariance experiment showing that OTI-attackability correlation remains stable when the segmenter is swapped; without this, the independence from model proxies is not established.
[Theoretical analysis] Theoretical section: The paper states that OTI is justified by decision-boundary geometry and mid/high-frequency perturbation characteristics, yet no equations are supplied that relate texture intensity inside the object mask to either quantity. For example, there is no derivation showing how the chosen texture statistic modulates the minimal perturbation norm or the frequency content that crosses the boundary. Without such explicit links (e.g., an equation connecting the texture measure to the geometry of the decision surface), the theoretical grounding remains descriptive rather than load-bearing.

minor comments (2)

[Abstract] The abstract asserts that 'comprehensive experiments demonstrate that OTI is effective,' but does not name the datasets, attack algorithms, or quantitative correlation metrics (e.g., Spearman rank or AUC) used to establish effectiveness. These details should appear in the experimental section.
[Figures] Figures that visualize OTI should be accompanied by side-by-side attack-success maps or perturbation visualizations so readers can directly judge the claimed visual interpretability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which have helped us strengthen the presentation of our work. We address each major comment below and indicate the corresponding revisions.

read point-by-point responses

Referee: [Abstract / Method] Abstract and method section: The central claim that OTI is strictly model-free rests on the assumption that the semantic-object mask can be obtained without any learned model. In practice, semantic segmentation almost always employs a pre-trained network (SAM, Mask R-CNN, etc.). The manuscript must explicitly state the exact procedure used to produce the mask and must include an invariance experiment showing that OTI-attackability correlation remains stable when the segmenter is swapped; without this, the independence from model proxies is not established.

Authors: We agree that the model-free claim requires precise qualification. In our usage, 'model-free' denotes independence from the parameters or gradients of the specific target classifier whose attackability is being assessed. The semantic masks are generated using the Segment Anything Model (SAM) in its fully automatic, zero-shot mode with default point sampling. We have revised the abstract and Method section to state this procedure explicitly. We have also added an invariance experiment that replaces SAM with an alternative general-purpose segmenter and verifies that both the OTI values and their correlation with attack success rates remain stable, thereby confirming that the measure does not depend on a particular segmentation model. revision: yes
Referee: [Theoretical analysis] Theoretical section: The paper states that OTI is justified by decision-boundary geometry and mid/high-frequency perturbation characteristics, yet no equations are supplied that relate texture intensity inside the object mask to either quantity. For example, there is no derivation showing how the chosen texture statistic modulates the minimal perturbation norm or the frequency content that crosses the boundary. Without such explicit links (e.g., an equation connecting the texture measure to the geometry of the decision surface), the theoretical grounding remains descriptive rather than load-bearing.

Authors: We acknowledge that the theoretical justification would be more rigorous with explicit mathematical connections. In the revised manuscript we have inserted new equations that relate the chosen texture-intensity statistic directly to decision-boundary geometry. Specifically, we derive that higher intra-object texture intensity corresponds to a smaller minimal perturbation norm by increasing the projection of mid- and high-frequency components onto the normal of the decision surface; the derivation proceeds from a first-order Taylor expansion of the classifier output combined with a Fourier decomposition of the perturbation. revision: yes

Circularity Check

0 steps flagged

No circularity detected; OTI defined directly from texture properties

full rationale

The paper defines OTI explicitly as the texture intensity of the semantic object region, with theoretical grounding in decision-boundary geometry and mid/high-frequency perturbation statistics stated independently of any fitted parameters or model outputs. No equations, self-citations, or ansatzes are shown reducing the measure to its own inputs by construction, and the model-free claim is presented as a direct image-property computation without load-bearing reliance on prior author results or renamed empirical patterns. The derivation chain remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Central claim rests on the domain assumption that object texture intensity proxies attackability through frequency and boundary effects; no free parameters or invented entities are explicitly introduced in the abstract.

axioms (1)

domain assumption Texture intensity of semantic objects correlates with adversarial vulnerability via mid- and high-frequency perturbation characteristics
Invoked in the theoretical description of OTI principles from decision boundaries and frequency properties.

invented entities (1)

OTI (Object Texture Intensity) no independent evidence
purpose: Model-free measure of image attackability
Newly defined quantity whose independent evidence is limited to the paper's own experiments.

pith-pipeline@v0.9.0 · 5544 in / 1233 out tokens · 30514 ms · 2026-05-16T11:06:42.258521+00:00 · methodology

OTI: A Model-free and Visually Interpretable Measure of Image Attackability

Core claim

What carries the argument

If this is right

Where Pith is reading between the lines

Load-bearing premise

What would settle it

discussion (0)