{"id":"462198d0-6a5c-410f-8b67-787312c2bfc4","arxiv_id":"2507.02021","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"REDUS prunes low-weight, correctly classified samples from each training epoch, reporting 72.6% time savings at 1.62% accuracy loss on the CICIoT2023 IoT attack dataset.","lead":"The paper proposes REDUS, an AdaBoost-style resampling method that drops easy-to-classify training samples each epoch to cut deep-learning training time in centralized and federated IoT settings. On CICIoT2023 attack data it reports up to 72.6% faster training with about 1.6% accuracy loss, but the stated algorithm may not produce that saving because it still needs full forward passes to update sample weights.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"REDUS's claimed speedup omits the full-dataset forward pass required each epoch in Algorithm 1; Eq. (6) counts only backprop on the subset, so the 72.6% reduction is not supported by the presented algorithm.","rationale":"The reader's weakest assumption identifies exactly the load-bearing gap in the paper's argument. The paper defines training time in Eq. (6) solely in terms of backpropagation on the selected subset, but Algorithm 1's per-epoch bookkeeping (lines 8-13) requires a full forward pass over all n samples to compute f(x_i) for the error and weight updates. This forward pass is not counted in the time model, and no measurements or code are provided to show that it is negligible in practice. Because vanilla training also performs forward passes, including this cost changes the speedup calculation substantially; for typical forward-to-backward time ratios, the claimed 72.6% reduction is either impossible or requires an implausibly tiny subset. The concern is therefore not merely a matter of presentation but a direct challenge to the central efficiency result. The post hoc threshold selection and lack of error bars are additional weaknesses, but they are secondary to the unresolved computational-cost accounting. If the timing data are ever released and confirm the 213s figure under a faithful implementation, the headline result might survive; until then, the paper does not substantiate its main claim.","tokens_in":9546,"tokens_out":6513,"duration_ms":73080,"concrete_test":"Re-implement Algorithm 1 exactly on the CICIoT2023 setup (336K samples, 10 epochs, θ=1.5e-6) and measure wall-clock time per epoch, separating (a) the full-dataset forward pass for ε_t and weight updates (lines 8-13) from (b) forward+backward on D_hat. Compare the measured speedup against the claimed 72.6% reduction. Alternatively, profile t_f and t_b on the target hardware and compute the theoretical maximum speedup; if the maximum falls below 72.6% for the subset sizes implied by Table I, the claim is impossible. A reproduction that matches the reported 213s would indicate the overhead is already included and the concern is only with the paper's time model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim rests on Eq. (6), which sets total training time to |D|·τ·E and ignores the forward pass needed to compute f(x_i) for all n samples in Algorithm 1 lines 8-13. Each epoch after the first, ε_t and the weight update (Eqs. 9-11) require evaluating the model on the full dataset D, including samples excluded from D_hat. This full-dataset forward pass costs roughly n·t_f per epoch, where t_f is forward time per sample. If we include it, per-epoch cost is n·t_f + |D_hat|·(t_f+t_b), while vanilla is n·(t_f+t_b). The maximum possible time reduction when |D_hat|=0 is t_b/(t_f+t_b), which exceeds 72.6% only if t_f < 0.377·t_b. With typical t_b/t_f ratios of 2-3, the maximum reduction is 50-67%, and it is lower for positive |D_hat|. The paper does not report forward/backward time ratios or wall-clock profiles; without them, the claimed 72.6% reduction cannot be derived from the algorithm as written. The measurements in Table I are uncorroborated by code or timing logs.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes REDUS, an AdaBoost-inspired resampling technique that maintains per-sample weights during deep learning training and, at each epoch, trains only on the subset of samples whose weights exceed a threshold θ. The authors claim that this reduces the number of training samples per epoch, thereby reducing training time and energy consumption in both centralized and federated settings. The method is evaluated on the CICIoT2023 dataset for IoT attack detection in a federated learning setup, reporting a training time reduction of up to 72.6% at θ=1.5e-6 with an accuracy reduction of 1.62% relative to a baseline ANN. The paper also sketches an extension of REDUS to federated learning with FedAvg-style aggregation.","tokens_in":9780,"tokens_out":7902,"duration_ms":101051,"significance":"If the efficiency claim were rigorously supported, REDUS would be a simple and practical tool for reducing DL training budgets on resource-constrained IoT/edge devices, and the federated extension would be a natural fit for the paper's target scenario. The algorithmic idea is clearly stated in Algorithm 1, the equations are straightforward, and the use of a standard public dataset (CICIoT2023) is a strength. However, the manuscript does not provide code, timing logs, or a validation-based threshold selection procedure, and the central time-reduction argument omits an operation that the algorithm itself requires. As it stands, the headline result is not established, and the experimental protocol is circular in its selection of the reported operating point.","major_comments":[{"comment":"The total-training-time model omits the full-dataset forward pass required by the algorithm. After the first epoch, computing ε_t in Eq. (9) and updating every w_i in Eqs. (10)-(11) requires evaluating f(x_i) for all i ∈ D, including samples excluded from the subset D_hat. Since Eq. (6) counts only |D|·τ·E, with τ defined as the back-propagation time per sample, the derived reduction is not justified. Including the forward pass, the per-epoch cost is n·t_f + |D_hat|·(t_f + t_b) for REDUS versus n·(t_f + t_b) for vanilla training; the maximum reduction when |D_hat|=0 is t_b/(t_f + t_b), which is only about 50-67% for typical t_b/t_f ratios of 2-3 and is lower for positive |D_hat|. The paper itself states two paragraphs below Eq. (11) that \"the time complexity of the REDUS method is O(E · n)\", which concedes the same asymptotic order as vanilla training. The measured times in Table I therefore need to be supported by an implementation description or timing profiles that account for the forward pass, or the algorithm and claims need to be revised.","section":"Section IV-A, Eq. (6), and Algorithm 1 lines 8-13"},{"comment":"The thresholds θ = 1.3×10^-6 and θ = 1.5×10^-6 are selected as \"optimized\" after evaluating test-set accuracy and time over the ten scanned threshold values. Because the same test evaluation is used both to select θ and to report the resulting accuracy, the advertised 1.62% accuracy reduction is a post-hoc selected value rather than an unbiased estimate of the method's generalization performance. The authors should use a validation split for threshold selection and then report test performance at the selected threshold, along with the variance across the five experimental repetitions.","section":"Section V, Table I"},{"comment":"The weight-update rule is undefined when ε_t = 0 or ε_t = 1 because α_t = 0.5·log((1-ε_t)/ε_t) involves a division by zero and a logarithm of zero. Since REDUS progressively excludes correctly classified samples and concentrates on harder samples, a state with ε_t = 0 can plausibly occur after several epochs; the algorithm needs a fallback (for example, skip the weight update or set α_t = 0) to remain well-defined over the full training run.","section":"Section IV-A, Eq. (10)"}],"minor_comments":[{"comment":"The denominator is typeset as \"P i = 1nwi\" and should be Σ_{i=1}^n w_i; also, \"wether\" should be \"whether\".","section":"Section III, Eq. (1)"},{"comment":"The symbols \"LLT\" and \"LT T\" appear to be typographical errors for the total training time L_TT.","section":"Section IV-A, Eq. (6)"},{"comment":"The summation index is written as K=1 to K; it should be i=1 to K.","section":"Section IV-B, Eq. (12)"},{"comment":"The text says the threshold ranges from 0 to 1/n, but the experimental sweep goes from 0 to 2/3 × 1/n; please clarify the allowed range and the number of sampled steps.","section":"Section V"},{"comment":"No standard deviations or confidence intervals are reported for the five repetitions; because the accuracy differences of interest are small (e.g., 92.71% vs. 93.65%), variance information is needed to assess whether the reported accuracy drops are significant.","section":"Section V, Table I"},{"comment":"The text following the conclusion contains undecodable character sequences; the figure and its caption need to be regenerated in the manuscript.","section":"After Section VI (Fig. 4)"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: REDUS is a modest but real variant of AdaBoost-style sample weighting applied to subset selection in a single deep network, with re-inclusion and FedAvg integration. The paper is clearly written and the idea is easy to follow. But the central efficiency claim is not supported on the page: Eq. (6) counts only backprop on the included subset and ignores the full-dataset forward pass that Algorithm 1 requires every epoch to update the weights of all samples (lines 8-13). As the stress-test note points out, if you include that forward pass, the maximum possible time reduction is bounded by t_b/(t_f+t_b) — typically 50-67% with common forward/backward ratios, not 72.6%. So the headline number needs either a corrected complexity model or real wall-clock profiles with code.\n\nThe post hoc selection of theta from the same test set compounds the problem. The two 'optimized' thresholds are picked from Table I after seeing accuracy and time; that's fitted, not predicted. No error bars despite five runs, no baseline beyond vanilla ANN, no comparison to AdaBoost-CNN or LoAdaBoost, and the energy claims have no measurements behind them.\n\nThat said, the algorithm itself is coherent. The re-inclusion mechanism is a nice touch, and the FL integration is straightforward. The paper does not misrepresent its novelty relative to the cited works: AdaBoost-CNN is ensemble-based and LoAdaBoost operates at client level. So the combination is new.\n\nIf the authors can produce code, timing logs, and a corrected cost model that includes the forward pass, the central claim might survive. As written, it doesn't. I'd send it to review — the idea is worth a serious referee, but the expectation should be major revision and re-evaluation with real measurements. Not something I'd cite as-is.","headline":"A plausible but unproven efficiency claim: the time model omits the full-dataset forward pass, so the 72.6% speedup is not supported by the algorithm as written.","tokens_in":10336,"tokens_out":2795,"would_cite":false,"duration_ms":33596,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an AdaBoost-inspired per-epoch resampling scheme can cut deep-learning training time by 72.6% on the CICIoT2023 IoT attack dataset while reducing accuracy by only 1.62 percentage points.","keywords":["resampling","federated learning","deep learning","software-defined networking","IoT attack detection","CICIoT2023 dataset","training time reduction","AdaBoost"],"falsifier":"Measure the wall-clock time of one REDUS epoch at $\\theta = 1.5\\times10^{-6}$ on CICIoT2023 with the same hardware and split it into the subset back-propagation time and the time spent computing $\\varepsilon_t$ and updating all $n$ weights (including the full forward pass over excluded samples). If the latter is non-negligible, the observed 72.6% reduction relative to vanilla training will shrink correspondingly; a direct comparison of per-epoch times with and without the full-dataset forward pass would settle whether the claimed savings are attainable.","tokens_in":9314,"feed_emoji":"⚡","tokens_out":4613,"duration_ms":42812,"temperature":0.7,"pith_summary":"The paper proposes REDUS, an AdaBoost-inspired resampling method for deep learning training that each epoch keeps only samples whose running weight exceeds a threshold, so the model trains on a shrinking, hard-example-focused subset. On the CICIoT2023 IoT attack dataset, REDUS with threshold $\\theta = 1.5\\times 10^{-6}$ cuts average training time per round from 777 seconds to 213 seconds (72.60% reduction) while accuracy drops only 1.62 percentage points, from 94.33% to 92.71%. The authors argue that such targeted resampling balances computational demands and model performance, making deep learning feasible on resource-limited edge devices in centralized and federated settings. If true, REDUS offers a practical way to reduce training budgets in SDN-managed IoT networks without retraining models from scratch.","feed_headline":"Resampling cuts DL training time 72.6%","feed_subtitle":"An AdaBoost-style weight filter drops redundant samples each epoch, costing just 1.62% accuracy on IoT attack detection.","key_machinery":"The central object is the sample-weight update and thresholded inclusion mechanism in Algorithm 1. Each sample starts at weight $w_i = 1/n$; at every epoch after the first, the weighted misclassification rate $\\varepsilon_t$ is computed, and every sample's weight is updated by an AdaBoost-style factor, increasing for misclassified samples and decreasing for correctly classified ones, then normalized by $Z_t$. Samples whose updated weight falls below $\\theta$ are excluded from the next epoch's training set $\\hat{D}$, but their weights keep being updated so they can re-enter if they become hard again. This mechanism converts an AdaBoost ensemble-building procedure into a per-epoch sample selector for a single deep network, defining the claimed trade-off between training time and accuracy.","core_discovery":"The central claim is that a single strong deep-learning model can be trained efficiently by repeatedly reweighting samples with an AdaBoost-style update and discarding, each epoch, the samples whose weights fall below a fixed threshold $\\theta$; the model is then trained only on the surviving subset $\\hat{D}$. The weight of a misclassified sample is multiplied by $e^{\\alpha_t}/Z_t$ and that of a correctly classified sample by $e^{-\\alpha_t}/Z_t$, with $\\alpha_t = \\frac{1}{2}\\log\\frac{1-\\varepsilon_t}{\\varepsilon_t}$ and $Z_t = 2\\sqrt{\\varepsilon_t(1-\\varepsilon_t)}$, so hard samples dominate the next epoch while already-learned samples drop out. Since the gradient updates are applied only to samples in $\\hat{D}$, the per-epoch back-propagation cost falls as training proceeds, which the authors express through the time model $L_{TT} = |D| \\cdot \\tau \\cdot E$ and the reported wall-clock savings. Evaluated on CICIoT2023 within a five-client federated setup, REDUS at the optimized thresholds achieves 59.84% and 72.60% training-time reductions with 0.68% and 1.62% accuracy reductions respectively, which the paper presents as evidence that the method preserves model performance while substantially cutting computational load.","pith_inferences":["The reported time savings assume the cost of computing $\\varepsilon_t$ and updating weights for all $n$ samples each epoch is negligible; if the full-dataset forward pass required for those updates is included, the worst-case per-epoch cost remains linear in $n$, and the savings depend on forward passes being much cheaper than back-propagation.","REDUS is essentially a hard-example mining schedule; one could test it against simpler heuristics like keeping a fixed top-$k$ fraction per epoch to see whether the AdaBoost weighting, rather than the subset selection, drives the observed gains.","The accuracy recovery at $\\theta = 1.1\\times10^{-6}$ (94.33%, same as baseline, with 0.52% time reduction) suggests non-monotonic behavior worth investigating; the re-inclusion dynamics may occasionally improve generalization.","In a federated setting, the threshold could be adapted per client based on local data difficulty or resource budget, but the paper does not explore this."],"forward_implications":["At $\\theta = 1.3\\times 10^{-6}$, training time drops 59.84% with only 0.68% accuracy loss, showing a favorable operating point.","At $\\theta = 1.5\\times 10^{-6}$, training time drops 72.60% with only 1.62% accuracy loss.","Because REDUS runs locally on each client before FedAvg aggregation, it reduces the per-client training time $L_{TT_i}$ and thus the round time in federated learning without changing communication rounds.","The threshold $\\theta$ provides a tunable knob between computational savings and accuracy, with high thresholds ($1.7\\times10^{-6}$ and above) cutting time by about 78% but losing 2.82 to 3.48% accuracy.","REDUS preserves the model's final accuracy within a couple of points on the tested dataset, supporting its use in resource-limited IoT and edge settings."],"supporting_citations":[{"why":"Supplies the CICIoT2023 dataset that all experiments and reported time/accuracy numbers are based on.","marker":"[13]"},{"why":"Provides the preprocessing and IID partitioning procedure that defines the five-client federated training setup.","marker":"[22]"},{"why":"One of the AdaBoost references that inspires the sample-weight update and thresholding mechanism in REDUS.","marker":"[11]"},{"why":"The other AdaBoost-related reference used to ground the boosting-inspired resampling idea.","marker":"[12]"}],"fun_headline_variants":["AdaBoost-style resampling cuts IoT training time by 72.6%","REDUS drops redundant samples, trains 72.6% faster on IoT","Resampling filter cuts federated IoT training time 72.6%","Weighted resampling trims IoT training 72.6%","Federated IoT training 72.6% faster via AdaBoost-style resampling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's claimed savings depend on the assumption that recomputing weights for all $n$ samples each epoch, including the excluded ones, is so cheap that total time is dominated by back-propagation on the surviving subset; if that full-dataset pass is counted, worst-case per-epoch cost remains linear in $n$ and the 72.6% savings do not follow as written.","fun_headline_variants_meta":{"raw":{"variants":["AdaBoost-style resampling cuts IoT training time by 72.6%","REDUS drops redundant samples, trains 72.6% faster on IoT","Resampling filter cuts federated IoT training time 72.6%","Weighted resampling trims IoT training 72.6%","Federated IoT training 72.6% faster via AdaBoost-style resampling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001152,"raw_usage":{"total_tokens":4843,"prompt_tokens":1079,"completion_tokens":3764,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":3664}},"tokens_in":695,"tokens_out":3764,"duration_ms":29709,"temperature":1.0,"reasoning_tokens":3664,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:43:39.904796+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the wall-clock time of one REDUS epoch at $\\theta = 1.5\\times10^{-6}$ on CICIoT2023 with the same hardware and split it into the subset back-propagation time and the time spent computing $\\varepsilon_t$ and updating all $n$ weights (including the full forward pass over excluded samples). If the latter is non-negligible, the observed 72.6% reduction relative to vanilla training will shrink correspondingly; a direct comparison of per-epoch times with and without the full-dataset forward pass would settle whether the claimed savings are attainable.","supporting_citations":[{"cited_title":"CICIoT2023: A real-time dataset and benchmark for large- scale attacks in IoT environment,","cited_arxiv_id":null,"evidence_quote":"Supplies the CICIoT2023 dataset that all experiments and reported time/accuracy numbers are based on."},{"cited_title":"A robust federated learning approach for combating attacks against IoT systems under non-IID challenges,","cited_arxiv_id":null,"evidence_quote":"Provides the preprocessing and IID partitioning procedure that defines the five-client federated training setup."},{"cited_title":"Adaboost-based security level classification of mobile intelligent terminals,","cited_arxiv_id":null,"evidence_quote":"One of the AdaBoost references that inspires the sample-weight update and thresholding mechanism in REDUS."},{"cited_title":"Joint provisioning of QoS and se- curity in IoD networks: Classical optimization meets AI,","cited_arxiv_id":null,"evidence_quote":"The other AdaBoost-related reference used to ground the boosting-inspired resampling idea."}],"review_version":1}