{"id":"d1d661ff-6479-414c-bbf4-88861238816f","arxiv_id":"2411.12071","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"TARL replaces Triangle Attack's fixed angle-update rule with a Q-learning policy, reporting comparable attack success on ImageNet and CIFAR-10 with 500 queries instead of 1,000.","lead":"TARL modifies the Triangle Attack adversarial method by using Q-learning to pick the search angle, and the authors report similar attack success rates at half the query budget on several image classifiers. The paper also critiques Triangle Attack's angle-update rule, though the critique is illustrative rather than proven.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'half as many queries' claim lacks a matched-query control: TARL@500 is never compared against TA@500, so the RL component's contribution to the reported ASR is unmeasured.","rationale":"The reader's weakest assumption focuses on whether the Q-learning agent converges within 500 queries, given the discrete Q-table versus continuous action space and lack of convergence analysis. I agree that this is a serious implementation-level concern, but I find a more load-bearing issue in the experimental design: the query-efficiency claim is never tested against a matched-query baseline. Comparing TARL@500 to TA@1000 does not isolate the effect of RL; it only shows that a budget reduction plus RL stays close to the full-budget original. If TA@500 already performs similarly to TA@1000, the central contribution disappears, regardless of whether the Q-learning converges. The paper's own tables support this worry: at tight perturbation budgets TARL is systematically worse than TA@1000, and the only regime where TARL wins is a near-saturated C=0.5 budget where differences are fractions of a percent. The internal inconsistencies in Algorithms 1, 3, and 4, plus the absence of code, seeds, and error bars, make the reported numbers impossible to attribute to a specific mechanism. I do not recommend changing the reader's CONDITIONAL verdict; the paper could be accepted if the authors supply a matched-query comparison, release code, and soften the abstract. But the missing control is the single most load-bearing gap because it directly undermines the stated query-efficiency advantage.","tokens_in":13232,"tokens_out":5028,"duration_ms":51457,"concrete_test":"Compute TA and TARL under identical query budgets (500 and 1000) on the same 200 ImageNet images and 1000 CIFAR-10 images, across multiple random seeds, reporting mean and standard deviation of ASR at RMSE C=0.01, 0.05, 0.1, and 0.5. If TARL@500 does not significantly exceed TA@500 (or if TA@500 already matches TA@1000), the 'half as many queries' claim is not supported. Additionally, release code and hyperparameters so the Q-table update in Algorithm 4 can be checked for consistency with the text.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that replacing TA's fixed alpha-update with online Q-learning yields similar attack success with half the queries. The experiments only compare TARL with 500 queries to TA with 1000 queries (Tables 1-5). This design confounds the query-budget reduction with the RL modification. It never reports TA at 500 queries, so we cannot know whether TARL's 'similar' ASR comes from the learned policy or from TA's ASR already saturating by 500 queries on these models. The paper's own results show TARL is consistently worse than TA@1000 on nearly all ImageNet models at RMSE C=0.01, 0.05, and 0.1; only at the very loose C=0.5 budget (Table 5) does TARL beat TA, by +0.2% to +0.9% near 100% saturation. If TA@500 is within noise of TA@1000, TARL provides no query-efficiency gain and the abstract overstates the result. The algorithm description also contains internal inconsistencies that prevent attributing the numbers to Q-learning: Algorithm 1 line 7 checks 'if q-table is empty' though Algorithm 2 initializes a zero-filled Q-table; Algorithm 4 sets reward=-l2 when adversarial while Section 2.3 says reward=1/l2; Algorithm 3 uses an undefined probability p and undefined action list; no hyperparameters, seeds, or error bars are given. Section 2.2's 'theoretical correction' is only illustrative figures, not a formal refutation of Proposition 1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes TARL, a decision-based black-box attack that replaces Triangle Attack's fixed alpha-update rule with an online Q-learning policy, and reports experiments comparing TARL at 500 queries to TA at 1,000 queries across ImageNet and CIFAR-10 models, including a diffusion-based defense. The paper also presents illustrative counterexamples to Triangle Attack's Proposition 1 about the learned angle alpha and claims that TARL attains similar or better attack success rates with half the queries. The empirical tables show that TARL is consistently below TA on tight perturbation budgets and only surpasses TA at the largest tested budget (RMSE C = 0.5).","tokens_in":13545,"tokens_out":4602,"duration_ms":45896,"significance":"If the central claim were established, TARL would be a useful query-efficiency improvement over a competitive decision-based attack, and the extension to nine ImageNet/CIFAR-10 models plus a diffusion defense is a broader evaluation than the original Triangle Attack paper. I see no circularity problem in comparing TARL to external attacks, since the agent learns online from the attack's own queries rather than from benchmark labels. However, the claimed advantage is not currently supported because the experimental design lacks a matched-query baseline and the algorithm description contains unresolved inconsistencies; the significance is therefore conditional on substantial revision.","major_comments":[{"comment":"The abstract's 'similar, if not better' claim is not supported for tight budgets: across ImageNet models, TARL at 500 queries is 1.9–7.3 percentage points below TA at 1,000 queries for C=0.01, 1.2–4.7 points for C=0.05, and up to 4.5 points for C=0.1, with TARL exceeding TA only at C=0.5 in Table 5. Because the query budgets differ in every comparison, the contribution of the RL component is confounded with the query-budget reduction; please report TA at 500 queries under matched conditions, since without that control the claimed 'half as many queries' advantage is not measurable.","section":"§4.2, Tables 2–4"},{"comment":"The algorithm description is internally inconsistent: Algorithm 1 line 7 checks 'if q-table is empty' although Algorithm 2 initializes a zero-filled Q-table; Algorithm 4 assigns reward = -l2 for adversarial outcomes while Section 2.3 states reward = 1/l2; Algorithm 3 uses an undefined probability p and an action list that is never constructed; and the text claims Q-learning handles continuous action spaces while Algorithm 2 builds a discrete state-action table. As written, the reported numbers cannot be attributed to the described learning procedure, so please provide consistent pseudocode, define every symbol, and state the discretization of the action space.","section":"§2.3 and Algorithms 1–4"},{"comment":"The claimed 'theoretical corrections' are illustrated with 2D figures rather than formal arguments; no theorem or counterexample is stated, and the conclusion that the alpha-update algorithm 'shows there is potential for improvement' is not derived from the figures alone. If the paper's title promises corrections, please state the assumptions and provide a formal counterexample to Proposition 1 of [37], or alternatively reframe the section as an empirical limitation.","section":"§2.2"},{"comment":"No hyperparameter values, exploration schedules, random seeds, or variance estimates are reported, yet several ASR differences in Tables 4 and 5 are 0.1–0.9 percentage points, which is within plausible run-to-run noise for the reported sample sizes. Please provide the Q-learning hyperparameters (learning rate, discount factor, epsilon schedule, alpha step size, alpha range), the number of independent runs, and standard deviations or confidence intervals for every table.","section":"§4.1 and Tables 1–5"}],"minor_comments":[{"comment":"There are typos such as 'threating' for 'threatening' and 'sucessful' for 'successful'; please proofread the manuscript.","section":"§1.1"},{"comment":"The text refers to 'Figure 1 Part (a)' and 'Figure 2 Part (b)', but the figures are not included in the manuscript; please ensure all figures and their sub-panels are present and labeled.","section":"§2.2 and Figures 1–2"},{"comment":"The sign convention of the 'Diff.' column should be stated in each table caption; the text says a positive difference means TARL generated more adversarial examples, but the captions do not say this.","section":"Tables 2–5"},{"comment":"The conclusion says the evaluation was extended to 'the entire ImageNet dataset', but the experiments use 1,000 selected images per model; please rephrase to avoid overstating the evaluation scope.","section":"§4.2"},{"comment":"Table 5 is described as 'RMSE < 0.5', while Eq. (3) defines RMSE with a constant C; please specify that Table 5 uses C = 0.5 and clarify the relationship between C and the RMSE threshold.","section":"§4.1, Eq. (3)"},{"comment":"Reference [15] is cited for Inception-v3 but points to Ioffe's batch normalization paper; please replace it with the correct Inception-v3 citation.","section":"References"},{"comment":"The contribution list mentions '10 victim models and 1 defense model', but Tables 1–5 include 11 entries including the diffusion defense; please reconcile the count.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is reasonable and the missing matched-query control is in principle easy to run, so I view this as a major-revision situation rather than a rejection. I would also ask the editor to verify the provenance of the experimental numbers, because the algorithm pseudocode cannot currently reproduce the reported results; that is a reproducibility risk independent of the scientific claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a reasonable incremental variant of Triangle Attack with a broader experimental scope, but the central claim—same accuracy with half the queries—is not supported by the experiments as designed. The stress-test concern is correct and lands. The paper compares TARL at 500 queries against TA at 1,000 queries, never TA at 500. Given how quickly TA saturates on these benchmarks, the RL policy may be contributing little or nothing. For the tight budgets where the comparison matters (RMSE C=0.01, 0.05), TARL is consistently 2–7% worse than TA@1000 on ImageNet. The only setting where TARL wins, C=0.5, is near saturation, so those small gains are not meaningful.\n\nWhat is genuinely new: replacing TA's fixed alpha-update with a Q-learning-based adaptive policy, and extending evaluation to more models (ViTs, Mamba), a diffusion-based defense, and 1,000-image sets. The authors also deserve credit for reporting the negative differences in Tables 2–4 rather than hiding them. Section 2.2's critique of TA is illustrative, not a formal refutation, and the paper does not claim more than that.\n\nThe soft spots are several. The missing matched-query baseline is the load-bearing one. There is no code, no hyperparameters, no seeds, and no error bars, so the numbers cannot be reproduced or even interpreted. The algorithm description is internally inconsistent: Algorithm 1 checks whether the q-table is empty although Algorithm 2 initializes it to zeros; Algorithm 4 uses reward = -l2 while the text says reward = 1/l2; Algorithm 3 uses an undefined probability p and undefined action list. The Q-learning setup is also framed as handling continuous actions while using a discrete Q-table. And newer decision-based attacks cited in the introduction (DEAL, CGBA, BounceAttack, QE-DBA) are not benchmarked, which weakens the claim that TARL is state-of-the-art.\n\nWho is this for? Someone working specifically on query-efficient decision-based attacks might find the idea worth a quick look, especially the attempt to learn alpha adaptively. It is not a conceptual advance, and as written it is not a publishable evaluation of that idea. I would send it to peer review only if the authors provide code, hyperparameters, error bars, and—critically—a TA@500 control. Without those, the empirical core is not yet there. But it is not a waste of a referee's time; the idea is plausible and the experimental breadth is useful.","headline":"A modest, honest incremental attack paper whose headline query-efficiency claim is not actually demonstrated: TARL@500 is never compared against TA@500, so the RL contribution is unmeasured.","tokens_in":722,"tokens_out":1513,"would_cite":false,"duration_ms":30848,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that Triangle Attack's fixed rule for updating its learned angle can be replaced by an online Q-learning policy, giving an attack that matches the original's success rate with half as many queries.","keywords":["adversarial examples","decision-based black-box attack","query efficiency","Triangle Attack","reinforcement learning","Q-learning","frequency-space perturbation","image classification"],"falsifier":"Run TARL on the same 1,000 ImageNet images and models with the Q-table update disabled, so each alpha action is chosen randomly from the same epsilon-greedy distribution; if attack success rates at 500 queries stay within noise of TARL's reported numbers, then the learned policy is not what produces the query savings.","tokens_in":12984,"feed_emoji":"🎯","tokens_out":8062,"duration_ms":71033,"temperature":0.7,"pith_summary":"The paper sets out to show that Triangle Attack (TA), a state-of-the-art decision-based black-box attack, is limited by its hand-coded rule for adjusting the learned angle $\\alpha$, and that replacing that rule with reinforcement learning produces a more query-efficient attack. The proposed method, TARL, keeps TA's triangle geometry and frequency-space subspace search but uses a Q-learning agent to decide whether $\\alpha$ should increase or decrease on each query, based on the $\\alpha$ values, $\\ell_2$ distances, and success labels observed so far. The claim is that TARL achieves similar or better attack success rates than TA with 500 queries where TA needs 1,000, which matters because query count is the scarce resource in decision-based attacks. The experimental comparison is mixed but follows a clear pattern: TA retains an edge at very tight perturbation budgets, the methods are close at RMSE C=0.1, and TARL overtakes TA when the perturbation limit is relaxed.","feed_headline":"RL agent halves queries for triangle adversarial attacks","feed_subtitle":"TARL matches Triangle Attack's success rate with half the queries on ImageNet and CIFAR-10.","key_machinery":"The carrying object is the learned angle $\\alpha$ in Triangle Attack's triangle construction, where the benign image, the current adversarial candidate, and the next candidate form a triangle in a 2D subspace of the image's frequency space. $\\alpha$ controls the trade-off between finding any adversarial candidate and minimizing the perturbation, and TARL's machinery is a Q-table with discretized $\\alpha$ states and two actions (increase or decrease $\\alpha$), updated by epsilon-greedy Q-learning with reward $-\\ell_2$ for successful adversarial candidates and 0 for failures. This learned update replaces TA's fixed multiplicative $\\alpha$ rule, and it is the component the paper credits for making the attack adaptive to different decision-boundary shapes and more query-efficient.","core_discovery":"In the paper's own framing, the discovery is that Triangle Attack's $\\alpha$-update proposition does not hold for every decision-boundary shape, and that this failure can be repaired by learning. TARL replaces TA's deterministic rule (increase $\\alpha$ on adversarial queries, decrease it on failures, at fixed rates) with a Q-learning agent whose state is the current $\\alpha$ value plus the observed $\\ell_2$ distance and success label, whose actions are increase or decrease $\\alpha$, and whose reward is the negative $\\ell_2$ norm for adversarial candidates and zero otherwise. The agent is updated online after every query using the standard Q-learning rule, with epsilon-greedy exploration and clipping of $\\alpha$ to a predefined range. The reported conclusion is that, across ImageNet and CIFAR-10 classifiers including vision transformers, a state-space model, and a diffusion-based defense, TARL at a 500-query budget performs within a few percent of TA at a 1,000-query budget, and outperforms TA on every ImageNet model when the perturbation limit is set to RMSE < 0.5.","pith_inferences":["A direct test the paper leaves implicit is to train the Q-agent offline on a set of images and freeze it before attacking new images; the current per-image online learning makes it hard to separate genuine learned boundary geometry from the effect of random exploration within the first 500 queries.","The crossover pattern in the tables suggests an adaptive attack that switches between TA's fixed rule and TARL's learned rule depending on the perturbation budget could dominate either method alone, since each dominates in a different RMSE regime.","The Q-table discretizes alpha even though the paper describes alpha as continuous; if a finer alpha grid or a continuous function approximator closes the remaining small gaps at RMSE C=0.05, that would indicate discretization, not learning, is the current bottleneck.","A cross-image transfer test would clarify whether the agent learns reusable boundary geometry or only per-image quirks; TARL's stated per-image training leaves this question open."],"forward_implications":["If TARL's learning works as claimed, a 500-query budget is enough to reach attack success rates that previously required 1,000 queries, effectively doubling the number of attacks an adversary can attempt under query-limited conditions.","The gain is budget-dependent: at RMSE C=0.01, TARL trails TA by 1.9 to 7.3 percentage points on ImageNet models, so tight-perturbation settings still favor TA's fixed rule.","TARL stays within about 2 percentage points of TA against a diffusion-based adversarial training defense while using half the queries, extending query-efficient attacks to at least one state-of-the-art defense.","Because TARL only changes the alpha-update step, its Q-learning policy can be layered onto any future improvement to TA's subspace selection or beta search without changing the rest of the attack."],"supporting_citations":[{"why":"Defines Triangle Attack, its alpha-update rule, and the 1,000-query results that TARL is measured against.","marker":"[37]"},{"why":"Supplies the Q-learning update rule and table structure that TARL uses to replace TA's fixed alpha update.","marker":"[39]"},{"why":"Provides the QEBA baseline, the binary-search initialization, and the RMSE metric used in the comparison.","marker":"[21]"},{"why":"HopSkipJumpAttack is one of the decision-based baselines in the main comparison table.","marker":"[7]"},{"why":"Surfree is a query-efficient baseline that TARL is compared against in the same table.","marker":"[26]"},{"why":"GeoDA is a geometric decision-based baseline used to position TARL among boundary-search attacks.","marker":"[30]"},{"why":"Supplies the diffusion-model adversarial training defense used to test TARL against a state-of-the-art defense.","marker":"[38]"}],"fun_headline_variants":["RL teaches triangle attack to match accuracy with half the queries","TARL: Q-learning fixes triangle attack flaw, cuts queries in half","Learning the right move: RL-optimized triangle attack, 50% queries","Triangle attack gets a Q-learning boost: half the queries, same punch"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack's gains rest on the assumption that a Q-learning agent, learning from scratch for each image, can converge to a useful alpha-adjustment policy within the 500-query budget even though the decision boundary and the best alpha keep changing as the attack progresses.","fun_headline_variants_meta":{"raw":{"variants":["RL teaches triangle attack to match accuracy with half the queries","TARL: Q-learning fixes triangle attack flaw, cuts queries in half","Learning the right move: RL-optimized triangle attack, 50% queries","Triangle attack gets a Q-learning boost: half the queries, same punch"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1351,"prompt_tokens":903,"completion_tokens":448,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":519,"tokens_out":448,"duration_ms":12919,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:56:19.330008+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TARL on the same 1,000 ImageNet images and models with the Q-table update disabled, so each alpha action is chosen randomly from the same epsilon-greedy distribution; if attack success rates at 500 queries stay within noise of TARL's reported numbers, then the learned policy is not what produces the query savings.","supporting_citations":[{"cited_title":"Better diffusion models further improve adversarial traini ng","cited_arxiv_id":null,"evidence_quote":"Supplies the diffusion-model adversarial training defense used to test TARL against a state-of-the-art defense."},{"cited_title":"Triangle attack: A query-efficient decision- based adversarial attack","cited_arxiv_id":null,"evidence_quote":"Defines Triangle Attack, its alpha-update rule, and the 1,000-query results that TARL is measured against."},{"cited_title":"Q-learning","cited_arxiv_id":null,"evidence_quote":"Supplies the Q-learning update rule and table structure that TARL uses to replace TA's fixed alpha update."},{"cited_title":"Qeba: Query- efficient boundary-based blackbox attack","cited_arxiv_id":null,"evidence_quote":"Provides the QEBA baseline, the binary-search initialization, and the RMSE metric used in the comparison."},{"cited_title":"H opskipjumpattack: A query-efficient decision-based attack","cited_arxiv_id":null,"evidence_quote":"HopSkipJumpAttack is one of the decision-based baselines in the main comparison table."},{"cited_title":"Surfre e: a fast surrogate- free black-box attack","cited_arxiv_id":null,"evidence_quote":"Surfree is a query-efficient baseline that TARL is compared against in the same table."},{"cited_title":"Geoda: a geometric framework for black-box adversarial att acks","cited_arxiv_id":null,"evidence_quote":"GeoDA is a geometric decision-based baseline used to position TARL among boundary-search attacks."}],"review_version":1}