{"id":"f1a8eb56-05ca-41f8-a399-851f4e591685","arxiv_id":"1908.02435","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper introduces a bounded 'tent' activation function and reports improved adversarial robustness over PGD adversarial training on MNIST and CIFAR-10.","lead":"A new activation function, the tent, replaces ReLU to make image classifiers more robust to adversarial attacks. Tests on MNIST and CIFAR-10 report large accuracy gains over adversarial training, but the evaluation omits adaptive attacks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"High white-box accuracies may reflect zero-gradient saturation of tents rather than genuine robustness; an adaptive attack bypassing the zero gradient for |x|>δ is needed before the SOTA claim can be accepted.","rationale":"The reader correctly flags the open-space-risk explanation as unproven and notes the absence of adaptive attacks. My concern is slightly sharper: the tent activation's zero-gradient region is not just a threat to the explanation but to the measured accuracies themselves. The reported numbers come from default ART attacks that differentiate through the true network; with zero gradients across a large fraction of units, those attacks can get stuck and report failure where an adversary using a surrogate gradient would succeed. This is the single most load-bearing issue because if adaptive attacks break the method, the central claim of state-of-the-art robustness collapses; if they do not, the conditional accept can be upgraded. The paper does have genuine positive features: a simple, cheap activation, a public code base, and a plausible hypothesis. It also honestly describes the saturation risk and reports one weak result (CW-l2 on CIFAR-10), which is consistent with partially obfuscated gradients rather than uniformly strong robustness. No formal verification or error bars are provided, so the empirical evaluation is the linchpin. I recommend keeping the reader's CONDITIONAL verdict: the specific condition is that an adaptive, gradient-masking-robust evaluation must confirm the reported accuracies before the claims are accepted at face value. If such an evaluation is not run or fails, the verdict should move toward REJECT. Since the reader already set CONDITIONAL, no verdict change is needed from this stress-test pass.","tokens_in":17946,"tokens_out":4275,"duration_ms":43494,"concrete_test":"Re-evaluate WRN-28-10 tent (0.004) on CIFAR-10 under PGD (ε=0.032, step 0.008, 100 iterations, 20 random restarts) using the Backward Pass Differentiable Approximation (BPDA): keep the forward tent activation, but on the backward pass replace its gradient by the identity (or ReLU) wherever |x|>δ. Also run AutoAttack (APGD-CE, APGD-DLR, FAB, Square) at the same ε. If the accuracy on the adaptive PGD set drops materially (e.g., by more than 10 points relative to the reported 83.3%), the headline robustness is inflated by gradient masking. Repeat for MNIST-Net tent (0.12) with ε=0.3.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that tent activations provide state-of-the-art white-box robustness (91.8% MNIST, 73.5% CIFAR-10). This claim rests on accuracies against six gradient-based attacks (FGSM, BIM, PGD, DeepFool, CW-l2, CW-l∞) reported in Tables 1 and 2. However, the tent activation f(x;δ)=max(0,δ−|x|) has zero derivative for all |x|>δ (Eq. 2). For a white-box adversary using the network's true gradient, any saturated unit contributes no gradient signal; attacks such as PGD and BIM can terminate at points where the local gradient is zero, producing a false impression of robustness. The paper itself notes that tents 'can easily become inactive' for out-of-range inputs, and Appendix B shows learned δ values (0.058–0.234 on MNIST) that leave many pre-activations in the saturated regime. The three high-accuracy CIFAR results (FGSM 84.6, BIM 83.3, PGD 83.3) are exactly the attacks most susceptible to gradient masking, while CW-l2, which uses a different optimization formulation, drops to 23.7. Until an adaptive attack is run, the reported gap over PGD adversarial training cannot be attributed to true decision-boundary robustness, and the supporting open-space-risk explanation is likewise unproven.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the tent activation function f(x;δ)=max(0,δ−|x|), argues that monotonic activations such as ReLU have unbounded open space risk while tents have bounded open space risk, and reports that replacing ReLUs with tents yields substantially improved white-box adversarial robustness. On MNIST, the tent network achieves an average accuracy of 91.8% across six white-box attacks versus 76.8% for PGD adversarial training; on CIFAR-10 the corresponding numbers are 73.5% versus 41.8%. The experiments also include black-box transfer results, activation-input distribution analyses, and a formal section connecting the tent to the open-set-risk framework of Scheirer et al.","tokens_in":18203,"tokens_out":8211,"duration_ms":91246,"significance":"If the empirical claim is correct, this is an important result: it suggests that a simple bounded activation function trained at no extra computational cost can match or beat PGD adversarial training on standard benchmarks, and it offers a conceptual link between open space risk and adversarial vulnerability. The paper is also commendable for evaluating six attack types, reporting several weight-decay settings, and including an analysis of activation distributions under attack. However, the central robustness claim is not yet established because the reported white-box accuracies may reflect zero-gradient saturation of the tent activation rather than genuine decision-boundary robustness, and the open-space-risk explanation is not isolated from confounding changes in the training protocol.","major_comments":[{"comment":"The white-box accuracies that support the state-of-the-art claim are not trustworthy without an adaptive attack. The tent activation in Eq. (1) has zero derivative for |x|>δ (Eq. 2), and Appendix B reports learned δ values (e.g., 0.058–0.234 on MNIST) that leave a large fraction of activation inputs in the saturated regime. Under a standard PGD/BIM attack, gradients vanish in those regions, so the attack can terminate at a point of zero local gradient; the reported pattern, with FGSM/BIM/PGD accuracies above 83% but CW-l2 accuracy only 23.65 for WRN-28-10 tent (0.004), is exactly the signature of gradient masking. Please evaluate with an adaptive attack that handles the zero-gradient regions (e.g., BPDA with a straight-through surrogate, or random/ensemble PGD restarts), or provide a certified robustness bound, before the claim of significantly improved adversarial robustness compared to the state of the art can be accepted.","section":"§4.2, Tables 1–2; Eq. (2)"},{"comment":"The causal claim that reducing open space risk is the reason for the observed robustness is not identified by the experiments. The tent networks differ from the ReLU baselines in several confounded ways: for MNIST, the primary comparison includes the batch-normalization addition, and for CIFAR-10 the tent models use Adam while the ReLU and adversarial-training models use SGD; in all tent models, weight-decay is applied to the δ parameters rather than to the network weights. Bounding the output range alone, or changing the optimization trajectory, could explain the results without the open-space-risk mechanism. To support the attribution, the authors should compare against other bounded activations (e.g., clipped ReLU, hard tanh, tanh, sigmoid) under a matched training protocol, and ideally vary the bound while holding everything else fixed.","section":"§4.1, Tables 1–2; §5"},{"comment":"The theoretical analysis is at the level of a scalar activation function, but the robustness claim concerns classifiers over input images. Theorem 2 establishes that a tent activation has finite open space risk when viewed as a local recognition function over its own input; it does not establish a relationship between that quantity and the success of adversarial perturbations in image space. The conclusion's assertion that the open space risk model explains many issues for adversarial example generation is therefore stronger than what is proved. Please state the connection as a hypothesis and test it directly, for example by measuring the change in feature-space open space risk under attacks for tent versus ReLU networks.","section":"§3, Theorems 1–2; §5"}],"minor_comments":[{"comment":"The state-of-the-art claim is supported only by a comparison to Madry et al. [6]; please either broaden the comparison to other defenses available at the time of submission or temper the wording.","section":"Abstract and §5"},{"comment":"No error bars or multiple-seed results are reported; given that the paper makes a state-of-the-art claim, reporting the mean and variance over at least three runs would strengthen the conclusions.","section":"§4.1, Tables 1–2"},{"comment":"The proof of Theorem 1 is described as a sketch, although the contributions section says the paper proves the result; please provide a complete proof with explicit assumptions on the activation function and training distribution.","section":"§3, Theorem 1"},{"comment":"The appendix states that quantitative evaluation of adversarial robustness solely based upon accuracies may not be sufficient; this undercuts the accuracy-only evaluation in the main body and should be reconciled in the text.","section":"Appendix A"},{"comment":"Figure 3's caption has a typo, 'class indeces' should be 'class indices', and both captions could clarify that the displayed images are adversarial examples generated on the source network and then evaluated on the target network.","section":"Figures 2 and 3 captions"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, the tent activation f(x;δ)=max(0,δ−|x|) is a genuinely simple and original proposal, and the paper's formal point is correct: standard monotonic activations, including bounded ones like tanh, have unbounded open-space risk in the Scheirer–Boult sense because they don't decay to zero, while the tent does. Second, the headline result — state-of-the-art white-box robustness without adversarial training — is not supported by the evidence as presented. The high accuracies look like an artifact of zero gradients, not improved decision boundaries.\n\nThe paper deserves credit for a clean theoretical observation and for running six white-box and six black-box attacks on two architectures. The activation-by-activation distribution plots in Appendix B are a nice piece of diagnostic work. The proof sketch that ReLU-family activations have infinite open-space risk is straightforward and valid, and the tent's compact support is the right way to make the integral finite.\n\nThe soft spots are significant. Most of the model's units are saturated for |x|>δ, and Appendix B shows learned δ values around 0.1–0.2 on MNIST. FGSM, BIM, and PGD are exactly the attacks that get stuck when local gradients vanish. The clearest tell is Table 2: on CIFAR-10, WRN-28-10 tent(0.003) reports 84.6% FGSM, 83.3% BIM, 83.3% PGD, 83.6% DeepFool, but 24.1% against CW-l2. A robust classifier should not show that gap. The paper needs an adaptive attack — e.g., using a straight-through gradient estimator, or attacking a surrogate with similar activations, or running PGD with random restarts in the active regime — before any SOTA claim is credible. There are also no error bars, no multiple seeds, and no comparison against existing bounded activations like hard tanh. The open-space-risk explanation is offered as a hypothesis, which is honest, but experiments that are consistent with it don't prove it.\n\nWho should read this? Anyone working on adversarial defenses, mainly as a cautionary example of gradient masking. The idea is worth a serious referee, because if the robustness survives adaptive attacks it would be a real low-cost defense, and if it doesn't, the paper still demonstrates how easily gradient-based evaluations can be fooled. I'd send it to review, but the authors need to run the adaptive attacks and the hard-tanh baseline before the central claim is publishable.","headline":"The tent activation is a clean idea, but the reported robustness numbers almost certainly owe as much to zero-gradient saturation as to any real boundary robustness.","tokens_in":18722,"tokens_out":5054,"would_cite":false,"duration_ms":51306,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing monotonic activations with bounded tent activations makes classifiers robust to six white-box attacks, beating PGD adversarial training on both MNIST and CIFAR-10.","keywords":["tent activation function","open space risk","adversarial robustness","adversarial examples","monotonic activation function","PGD adversarial training","MNIST","CIFAR-10"],"falsifier":"Train the same architecture with a bounded activation that is nonzero across the same interval but has no flat zero-gradient region, such as a smoothed tent, and attack it with the same PGD and Carlini-Wagner budgets; if accuracy collapses relative to the hard tent, the flat tails, not the bounded output, carried the defense.","tokens_in":17731,"feed_emoji":"⛺","tokens_out":6746,"duration_ms":68562,"temperature":0.7,"pith_summary":"This paper argues that adversarial examples succeed because conventional activation functions like ReLU have unbounded \"open space risk\": an adversary can keep pushing an activation input in the direction that increases its response, and with monotonic activations that response grows without limit. The authors introduce the tent activation $f(x;\\delta)=\\max(0,\\delta-|x|)$, a bounded, learnable activation, and prove that standard monotonic activations have unbounded open space risk while tents have bounded risk. On six white-box attacks, tent networks average 91.8% accuracy on MNIST and 73.5% on CIFAR-10, compared with 76.8% and 41.8% for PGD adversarial training, with no additional training cost. If correct, simply changing the activation function offers a cheaper path to robustness.","feed_headline":"Tent activations beat PGD training on six attacks","feed_subtitle":"Swapping ReLU for a bounded activation lifts average attack accuracy to 91.8% and 73.5% with no extra training cost.","key_machinery":"The load-bearing object is the tent activation function $f(x;\\delta)=\\max(0,\\delta-|x|)$, a learnable bump that is zero outside $[-\\delta,\\delta]$, reaches height $\\delta$ at the origin, and has derivative $-\\operatorname{sgn}(x)$ inside the interval and zero outside. A normalized tent $\\hat{f}(x;\\delta)=f(x;\\delta)/\\delta^2$ acts as a compact probability abating model in the open-set recognition sense, which is what makes its open set risk finite; Theorem 1 of the paper shows no monotonic activation can be bounded this way. In the trained networks, every ReLU is replaced by batch normalization plus a tent, with tents shared across channels, initialized at $\\delta=1$, and squeezed by weight decay so that adversarial perturbations pushing activation inputs beyond the tent produce zero output instead of larger responses.","core_discovery":"The paper's central claim is that replacing monotonic activation functions with tent activations yields significantly improved adversarial robustness compared to the state of the art. It proves that every standard monotonic activation function acts as a weak classifier with unbounded open space risk, then shows that the tent activation $f(x;\\delta)=\\max(0,\\delta-|x|)$ has bounded open space risk. Replacing each ReLU with a tent fed by batch normalization and trained with weight decay on the learnable width $\\delta$ produces classifiers whose average accuracy across six white-box attacks reaches 91.8% on MNIST and 73.5% on CIFAR-10, versus 76.8% and 41.8% for PGD adversarial training. The paper presents this as support for its hypothesis that open space risk is a major contributing factor to the success of adversarial attacks.","pith_inferences":["The paper does not test combining tents with adversarial training; since tents cap activation output and adversarial training augments the input distribution, the two defenses target different failure modes and may stack, a testable extension.","Because the same bounded activations that resist attacks also limit the range of internal features, tent networks are a natural candidate for out-of-distribution rejection, an experiment the paper's open-space framing invites but does not run.","The uniform weight decay applied to all $\\delta$ parameters is admittedly crude; per-layer schedules or a learned budget on total open space risk could push the reported accuracies higher.","The tent's flat zero-gradient tails could in principle be exploited by adaptive attacks; a smoothed bounded activation would separate the effect of bounded range from the effect of zero gradients."],"forward_implications":["Adversarial robustness can be improved at no extra training cost: tents replace ReLUs without the extra forward and backward passes that PGD adversarial training requires.","Robustness is more even across attack types: tent classifiers are strong against DeepFool and the Carlini-Wagner attacks, the two families where PGD adversarial training is weakest.","The trade-off between clean accuracy and adversarial accuracy is gentler on smaller models, as the narrow WRN-28-1 keeps near-82% clean accuracy with tents while adversarial training drops it below 74%.","The learned tent widths $\\delta$ act as a robustness dial: shrinking them with weight decay improves adversarial accuracy until the tents become too small and clean accuracy falls."],"supporting_citations":[{"why":"Defines open space risk and compact probability abating models, the formal framework used by the paper's two theorems.","marker":"[8]"},{"why":"Supplies the PGD adversarial-training baseline that the tent classifiers are compared against and beat.","marker":"[6]"},{"why":"ReLU, the standard activation whose unbounded open space risk the paper proves and then replaces.","marker":"[25]"},{"why":"Batch normalization placed before each tent to keep activation inputs inside the active range during training.","marker":"[26]"},{"why":"Fast gradient sign method, one of the six white-box attacks in the evaluation.","marker":"[2]"},{"why":"Basic iterative method, one of the six white-box attacks in the evaluation.","marker":"[5]"},{"why":"DeepFool, the attack that exposes a weakness of PGD adversarial training and where tents show large gains.","marker":"[3]"},{"why":"Carlini-Wagner l2 and l-infinity attacks, among the six adversaries used for evaluation.","marker":"[4]"},{"why":"Toolbox used to generate the adversarial attacks and to run the PGD adversarial-training procedure.","marker":"[36]"}],"fun_headline_variants":["No extra training: tent activations beat PGD on six attacks","Tent activations outperform PGD training on six attacks","Bounded tent activations yield stronger adversarial robustness","Open space risk is key: tents fix adversarial attacks","Tent activations: higher accuracy on six white-box attacks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The explanation assumes that adversarial attacks succeed chiefly because monotonic activations can be pushed to arbitrarily large responses, so capping the response is what makes networks robust; if the accuracy gains actually come from the zero gradients on the tent's flat tails, the explanatory claim would be unsupported even with the accuracy numbers intact.","fun_headline_variants_meta":{"raw":{"variants":["No extra training: tent activations beat PGD on six attacks","Tent activations outperform PGD training on six attacks","Bounded tent activations yield stronger adversarial robustness","Open space risk is key: tents fix adversarial attacks","Tent activations: higher accuracy on six white-box attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00076,"raw_usage":{"total_tokens":3362,"prompt_tokens":920,"completion_tokens":2442,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":2361}},"tokens_in":536,"tokens_out":2442,"duration_ms":18239,"temperature":1.0,"reasoning_tokens":2361,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:43:20.470968+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture with a bounded activation that is nonzero across the same interval but has no flat zero-gradient region, such as a smoothed tent, and attack it with the same PGD and Carlini-Wagner budgets; if accuracy collapses relative to the hard tent, the flat tails, not the bounded output, carried the defense.","supporting_citations":[{"cited_title":"Scheirer, L.P","cited_arxiv_id":null,"evidence_quote":"Defines open space risk and compact probability abating models, the formal framework used by the paper's two theorems."},{"cited_title":"To- wards deep learning models resistant to adversarial attacks","cited_arxiv_id":null,"evidence_quote":"Supplies the PGD adversarial-training baseline that the tent classifiers are compared against and beat."},{"cited_title":"Rectified linear units improve restricted Boltzmann machines","cited_arxiv_id":null,"evidence_quote":"ReLU, the standard activation whose unbounded open space risk the paper proves and then replaces."},{"cited_title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","cited_arxiv_id":null,"evidence_quote":"Batch normalization placed before each tent to keep activation inputs inside the active range during training."},{"cited_title":"Goodfellow, Jonathon Shlens, and Christian Szegedy","cited_arxiv_id":null,"evidence_quote":"Fast gradient sign method, one of the six white-box attacks in the evaluation."},{"cited_title":"Goodfellow, and Samy Bengio","cited_arxiv_id":null,"evidence_quote":"Basic iterative method, one of the six white-box attacks in the evaluation."},{"cited_title":"Deepfool: a simple and accurate method to fool deep neural networks","cited_arxiv_id":null,"evidence_quote":"DeepFool, the attack that exposes a weakness of PGD adversarial training and where tents show large gains."},{"cited_title":"Towards evaluating the robustness of neural networks","cited_arxiv_id":null,"evidence_quote":"Carlini-Wagner l2 and l-infinity attacks, among the six adversaries used for evaluation."}],"review_version":1}