{"id":"90e4506a-1b65-4841-80b8-bdc57c161d90","arxiv_id":"2605.02987","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"LiteShield uses hybrid feature selection with common classifiers to reach 98% binary and 80-85% multiclass accuracy on UNSW-NB15 while favoring Random Forest for lower model size and inference cost in IoT environments.","lead":"LiteShield applies mutual information and recursive feature elimination to select features for six standard machine learning classifiers, then reports their accuracy on the UNSW-NB15 dataset for binary and multiclass intrusion detection in IoT settings. A smart generalist might read it to see how feature selection can make detection feasible on devices with tight compute and memory limits.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No evaluation on actual IoT hardware or IoT-specific traffic leaves the central suitability claim untested.","rationale":"The reader's weakest_assumption already isolates the exact generalization gap. Because the full manuscript follows the same UNSW-NB15-only protocol described in the abstract, the concern remains load-bearing and the UNVERDICTED status is appropriate; no stronger internal inconsistency or formal error was located.","tokens_in":1839,"tokens_out":344,"duration_ms":36981,"concrete_test":"Port the final Random Forest model (with the reported feature subset) to a representative IoT device such as Raspberry Pi Zero W or ESP32, run it on a replay of IoT-flavored traffic, and record end-to-end inference time, peak memory, and accuracy; if latency exceeds 10 ms or accuracy drops >3 points relative to the UNSW-NB15 numbers, the deployment claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's core claim is that hybrid feature selection plus lightweight classifiers yields accurate, computationally feasible intrusion detection for resource-constrained IoT networks. All reported results (KNN 98.26% binary / 85.22% multiclass; RF 98.01% / 80.39%) come from the UNSW-NB15 corpus on conventional hardware; no measurements of inference latency, RAM/ROM footprint, or power draw appear on embedded platforms, and no IoT-protocol traffic (MQTT, CoAP, etc.) is used. The assumption that dataset accuracy and model-size numbers will translate directly to live IoT conditions is therefore the least-secured link in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents LiteShield, a hybrid feature selection-driven lightweight IDS for resource-constrained IoT networks. Using the UNSW-NB15 dataset, it applies preprocessing, imbalance handling, and a two-stage feature selection (MI and RFECV), then evaluates six classifiers (DT, RF, KNN, LR, NB, SVM) for binary and multiclass attack detection. The key findings are that KNN attains the highest accuracy (98.26% binary, 85.22% multiclass) while RF offers the best practical trade-off with 98.01% binary and 80.39% multiclass accuracy along with lower model size and inference cost.","tokens_in":1979,"tokens_out":510,"duration_ms":82551,"significance":"Should the performance and efficiency claims hold under real IoT conditions, this work would provide a useful empirical demonstration of how hybrid feature selection can enable accurate yet deployable IDS in constrained environments. It builds on standard datasets and highlights classifier trade-offs, which is valuable for the field. However, the current evaluation on conventional hardware limits its significance for the stated IoT application.","major_comments":[{"comment":"The claim that Random Forest delivers 'substantially lower model size and inference cost than KNN' (abstract) is not accompanied by specific quantitative measurements or the hardware platform on which these costs were evaluated, which is central to the lightweight IoT suitability argument.","section":null},{"comment":"Details on the train-test split, the exact number of features retained after RFECV, hyperparameter optimization procedure, and any statistical significance testing for the accuracy figures are not provided (abstract and results), hindering verification of the reported performance (98.26% and 85.22% for KNN).","section":null},{"comment":"No experiments are conducted on actual resource-constrained IoT hardware or using IoT-specific traffic protocols (evaluation section); all results are from the UNSW-NB15 dataset on standard computing resources, leaving the core claim of feasibility for IoT networks untested.","section":null}],"minor_comments":[{"comment":"The description of class imbalance mitigation is high-level (abstract); more specifics on the techniques used would improve clarity.","section":null},{"comment":"Consider adding a dedicated section on limitations and future work to discuss the gap between dataset results and real-world IoT deployment.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper fits the cs.CR scope as it addresses security in IoT, but the lack of hardware validation is a key concern that should be addressed."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed feedback on our manuscript. We address each major comment point by point below, indicating where we will revise the paper to improve clarity, reproducibility, and discussion of limitations.","responses":[{"response":"We agree that the abstract and results would be strengthened by explicit quantitative data. In the revised manuscript we will add a table reporting model sizes (in KB) and per-sample inference times for all six classifiers, with direct comparison between Random Forest and KNN. We will also state that all timing and size measurements were obtained on a standard Intel Core i7 CPU and briefly discuss why these metrics remain relevant as a proxy for IoT constraints.","revision_made":"yes","referee_comment":"The claim that Random Forest delivers 'substantially lower model size and inference cost than KNN' (abstract) is not accompanied by specific quantitative measurements or the hardware platform on which these costs were evaluated, which is central to the lightweight IoT suitability argument."},{"response":"We acknowledge that these implementation details are necessary for reproducibility. The revised version will explicitly state the train-test split ratio, the precise number of features retained after the RFECV stage, the hyperparameter search method (including any cross-validation used), and whether statistical significance tests were applied to the accuracy figures. These additions will appear in the Experimental Setup and Results sections.","revision_made":"yes","referee_comment":"Details on the train-test split, the exact number of features retained after RFECV, hyperparameter optimization procedure, and any statistical significance testing for the accuracy figures are not provided (abstract and results), hindering verification of the reported performance (98.26% and 85.22% for KNN)."},{"response":"This is a valid observation about the scope of the evaluation. While UNSW-NB15 is a standard benchmark in the IoT IDS literature, we did not run tests on physical IoT devices or with protocols such as MQTT. In the revision we will add a dedicated Limitations and Future Work subsection that openly discusses this gap, justifies the use of the benchmark dataset and conventional hardware as an initial demonstration, and outlines planned follow-on work on constrained platforms. We cannot, however, add new hardware experiments within the current revision cycle.","revision_made":"partial","referee_comment":"No experiments are conducted on actual resource-constrained IoT hardware or using IoT-specific traffic protocols (evaluation section); all results are from the UNSW-NB15 dataset on standard computing resources, leaving the core claim of feasibility for IoT networks untested."}],"tokens_in":1506,"tokens_out":603,"duration_ms":70245,"standing_objections":["The absence of direct experiments on actual resource-constrained IoT hardware or IoT-specific traffic protocols cannot be fully resolved without new empirical work beyond the scope of this revision."]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main move is to take mutual information followed by RFECV, feed the reduced features into Decision Tree, Random Forest, KNN, Logistic Regression, Naive Bayes, and SVM, and measure binary and multiclass accuracy on the long-used UNSW-NB15 set. KNN reaches 98.26 % binary and 85.22 % multiclass; Random Forest sits at 98.01 % and 80.39 % while using less memory and inference time. They also run a quick check on minority classes and note that imbalance hurts multiclass results. That comparison of raw accuracy against model size is the only concrete thing the work adds, and it is done cleanly enough to be usable as a baseline recipe.","headline":"LiteShield runs standard MI-plus-RFECV selection and six textbook classifiers on UNSW-NB15, reports the usual accuracy numbers, but never tests anything on actual IoT hardware or IoT traffic.","tokens_in":2513,"tokens_out":233,"would_cite":false,"duration_ms":32947,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"claude-opus-4-7","evidence":[{"relation":"unclear","rs_module":"N/A — applied ML pipeline, no overlap with Foundation/Cost/Constants modules","rs_theorem":null,"paper_passage":"employs a two-stage feature selection pipeline based on Mutual Information (MI) and Recursive Feature Elimination with Cross-Validation (RFECV). Six lightweight classifiers are evaluated...: Decision Tree, Random Forest, K-Nearest Neighbors (KNN), Logistic Regression, Naïve Bayes, and Support Vector Machine."},{"relation":"unclear","rs_module":"N/A — empirical accuracy on UNSW-NB15 is not an RS-shaped quantity (no J-cost reading, no φ-ladder rung, no 8-tick clock).","rs_theorem":null,"paper_passage":"KNN achieved the highest raw predictive performance, reaching 98.26% accuracy for binary classification and 85.22% accuracy for multiclass classification."}],"headline":"Applied ML intrusion detection benchmark on UNSW-NB15; no contact with RS forcing chain, J-cost, φ, or 8-tick structure.","alignment":"orthogonal","rationale":"LiteShield is an empirical cybersecurity paper: it combines Mutual Information + RFECV feature selection with six standard sklearn-grade classifiers (DT, RF, KNN, LR, NB, SVM) on the UNSW-NB15 network intrusion dataset, reporting accuracy/F1 and model size. There is no cost functional, no ratio-symmetric reciprocal cost J(x) = ½(x + x⁻¹) − 1, no golden-ratio or φ-ladder structure, no 8-tick periodicity, no parameter-free derivation of constants, and no claim that touches spacetime, c, ℏ, or G. The \"lightweight\" framing concerns RAM/ROM/latency budgets on IoT hardware, not RS-style recognition cost minimization. RS makes no prediction about machine-learning classifier performance on network traffic corpora, and nothing in the paper contradicts an RS theorem. The domain is simply outside RS's scope, so the correct judgment is orthogonal with high confidence.","tokens_in":9705,"confidence":"high","tokens_out":908,"duration_ms":16661,"cache_read_input_tokens":62009,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Hybrid feature selection with lightweight classifiers enables accurate intrusion detection under tight IoT resource limits.","keywords":["IoT security","intrusion detection","feature selection","lightweight machine learning","network attack detection","binary classification","multiclass classification","resource constraints"],"falsifier":"Running the Random Forest model on physical IoT devices against real network traffic and observing whether binary accuracy stays near 98 percent while inference time and memory use remain within device limits.","tokens_in":2715,"feed_emoji":"🛡️","tokens_out":620,"duration_ms":33616,"temperature":0.7,"pith_summary":"The paper seeks to close the gap between computationally heavy intrusion detection systems and the limited processing power of IoT devices. It introduces a two-stage feature selection method followed by evaluation of six simple classifiers on network traffic data for both binary and multiclass attack identification. Results indicate that one model reaches the highest accuracy while another maintains nearly the same detection quality at far lower model size and inference cost. A sympathetic reader would care because this points to a practical route for embedding security directly on edge hardware instead of relying on external servers.","feed_headline":"Random Forest balances accuracy and efficiency for IoT intrusion detection","feed_subtitle":"It reaches 98.01 percent binary accuracy and 80.39 percent multiclass with far lower model size and inference cost than the top performer.","key_machinery":"The two-stage hybrid feature selection pipeline that first ranks features by Mutual Information then refines them via Recursive Feature Elimination with Cross-Validation to produce a compact input set for the classifiers.","core_discovery":"LiteShield applies Mutual Information and Recursive Feature Elimination with Cross-Validation to shrink the feature space, then shows that Random Forest reaches 98.01 percent binary accuracy and 80.39 percent multiclass accuracy with substantially smaller model size and lower inference cost than the highest-accuracy alternative, while ablation tests confirm that class imbalance reduces multiclass performance.","pith_inferences":["Direct hardware tests could show whether the reported accuracy holds when traffic patterns differ from the training corpus.","The same selection-plus-classifier pattern might apply to other edge security tasks such as anomaly detection in sensor streams.","Further trimming of the feature list could yield even smaller models for the most memory-limited microcontrollers."],"forward_implications":["Random Forest supplies the strongest practical trade-off, delivering near-maximum detection quality at reduced computational expense.","Class imbalance in attack types directly lowers multiclass accuracy and requires targeted handling.","The reduced feature set supports both binary and multiclass tasks without exceeding typical IoT budgets.","Lightweight models lower the barrier to on-device monitoring instead of offloading to the cloud."],"fun_headline_variants":["Random Forest achieves 98.01% binary accuracy with low IoT inference cost","LiteShield framework delivers efficient IoT attack detection with Random Forest","RFECV and MI reduce features for lightweight IoT intrusion detection","Random Forest provides 80.39% multiclass accuracy at reduced IoT cost"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Performance measured on the benchmark network dataset will translate directly to accurate low-cost detection when the same models run on actual resource-constrained IoT hardware with live traffic.","fun_headline_variants_meta":{"raw":{"variants":["Random Forest achieves 98.01% binary accuracy with low IoT inference cost","LiteShield framework delivers efficient IoT attack detection with Random Forest","RFECV and MI reduce features for lightweight IoT intrusion detection","Random Forest provides 80.39% multiclass accuracy at reduced IoT cost"]},"model":"grok-4.3","cost_usd":0.008267,"raw_usage":{"total_tokens":3776,"prompt_tokens":723,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":82674500,"prompt_tokens_details":{"text_tokens":723,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2975,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":723,"tokens_out":78,"duration_ms":26280,"temperature":1.0,"reasoning_tokens":2975,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-08T18:48:09.834283+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the Random Forest model on physical IoT devices against real network traffic and observing whether binary accuracy stays near 98 percent while inference time and memory use remain within device limits.","supporting_citations":[],"review_version":1}