{"id":"b79a1e74-7895-4981-815d-8bcad60c3dfc","arxiv_id":"2411.11603","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FSNID uses neural estimates of transfer entropy to remove uninformative network features without hurting intrusion detection accuracy, and an LSTM variant exploits time patterns to shrink the feature set further.","lead":"This paper presents FSNID, a feature selection method for network intrusion detection that uses a neural network to estimate how much information each network feature transfers to the attack label, then drops features that transfer almost none. A version with a recurrent network also uses time patterns, letting it keep even fewer features while matching the accuracy of models using all features.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The implemented FSNID decision rule replaces the exact condition Φ=0 with a normality-based 95% threshold on noisy neural estimates, and the paper never validates that threshold; if it removes many small-but-nonzero contributions, the claimed information-preservation and performance-maintenance…","rationale":"The empirical evaluation is extensive and the pattern across five datasets is internally consistent: feature counts drop sharply while accuracy and F1 remain near the full-feature baselines. That is real evidence in favor of the central claim. The theoretical guarantee in Theorem 1 would supply the missing rigor, but as implemented the guarantee is not directly applicable: Section 4.7 intervenes between the ideal condition Φ=0 and the actual per-feature decision. The reader identified this as the weakest assumption, and I agree. A normality failure alone would not be fatal if the empirical threshold produced similar selections; the proposed test would show whether it does. If the two thresholds diverge, the paper would need either a calibrated nonparametric rule or a conservative bound on information loss, and the conditional verdict would remain the right one. No ad hominem is intended; the concern is about an unvalidated statistical bridge between a clean theorem and a noisy estimator, not about the authors' conduct.","tokens_in":25259,"tokens_out":6912,"duration_ms":75934,"concrete_test":"On UNSW-NB15 (or the smallest dataset for tractability), run FSNID's complete pipeline at least 20 times with independent seeds, recording Φ estimates for real features and for 50 independently drawn random features inserted as null variables. Replace the normal-theory Neyman-Pearson cutoff with the empirical 95th percentile of the null Φ distribution, and compare the selected feature sets and downstream classifier F1/accuracy against the normal-theory selection. If the empirical-threshold feature set differs in size or F1 beyond a pre-specified equivalence margin (e.g., F1 drop > 0.01), the normality assumption is load-bearing; if the two selections and F1s coincide, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 and Theorem 1 are sound only when Φ is evaluated exactly: under exact evaluation, each removed feature leaves H(Y|current set) unchanged, so H(Y|X*)=H(Y|X) follows by induction. The implemented method never evaluates Φ exactly. It estimates Φ as the difference of two noisy neural MINE estimates (Eqs. 4 and 6), then replaces the condition Φ=0 with the Section 4.7 rule: a feature is kept only if its estimated Φ is outside the null range at 95% under an assumed normal distribution for both the null model and the feature estimates. The paper provides no normality check, no bias/variance analysis of the MINE-based difference, and no account of how the rule behaves when many features have small but nonzero Φ. In that regime, the per-feature test can remove each feature individually while the cumulative loss of conditional information is large, so the theorem's conclusion H(Y|X)=H(Y|X*) need not hold for the selected set. The empirical 'maintaining NID performance' claim is therefore only as reliable as the threshold calibration, which is unvalidated; reported feature-retention fractions also lack confidence intervals and no equivalence test is run against the full-feature classifiers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FSNID, a filter-based feature selection method for network intrusion detection. The method defines a transfer-entropy-like measure Φ_{X_i;X→Y} = H(Y|X\\{X_i}) − H(Y|X), estimates it by subtracting two neural mutual-information estimates (based on MINE), and removes features for which this estimate is judged statistically indistinguishable from zero using a null model and a 95% Neyman-Pearson rule. A sequential variant (Algorithm 1) is designed to handle redundant features, and a temporal version replaces the feedforward estimator with an LSTM, GRU, or TCN. The paper includes a theoretical guarantee (Appendix B) that exact application of Algorithm 1 preserves H(Y|X) = H(Y|X*), and an empirical study on five public NID datasets comparing against PI, LASSO, UMFI, MIFA, CLM, and a random baseline. The experiments report reduced feature counts with accuracy, F1, and false-positive rates close to the full-feature classifiers, plus a scalability experiment showing near-linear runtime in the number of features.","tokens_in":25441,"tokens_out":5487,"duration_ms":53025,"significance":"If the information-preservation claim held for the implemented estimation procedure, FSNID would be a practically valuable contribution: it is a filter method with linear complexity, no preset number of features k, a native treatment of redundancy, and an optional recurrent extension that exploits temporal dependencies. The empirical scope is a strength: five public datasets, six baselines, a random-selection control, and 95% confidence intervals for classification metrics. However, the central theoretical guarantee applies only to exact computation of Φ, while the deployed decision rule replaces the exact condition Φ=0 with an unvalidated statistical threshold on noisy neural estimates. The experimental claim of 'maintaining NID performance' is supported only by point comparisons and overlapping confidence intervals, not by equivalence testing, and the feature-retention results are reported without confidence intervals. These gaps leave the main claims weaker than the abstract suggests.","major_comments":[{"comment":"The implemented FSNID rule is not the exact condition Φ_{X_i;X→Y}=0 that Theorem 1 analyzes. Section 4.7 replaces that condition with a 95% Neyman-Pearson test that assumes both the null model NM and the feature Φ estimates are normally distributed. The paper provides no normality check, no bias or variance analysis of the MINE-based estimate in Eq. (6), and no account of how the rule behaves when many features have small but nonzero Φ. Under such a rule, the per-feature test can remove individually negligible features whose cumulative conditional information is substantial, so the conclusion H(Y|X)=H(Y|X*) of Theorem 1 does not transfer to the implemented algorithm. This directly affects the central 'maintaining NID performance' claim, and the threshold needs validation (e.g., calibration plots, bootstrap intervals, or a sensitivity analysis) before the information-preservation claim can be attributed to FSNID as deployed.","section":"4.7, Appendix B"},{"comment":"The top row of Figure 2 reports the fraction of features retained by each method, but these are point estimates only. The paper states that Φ and classification results are obtained over 5 runs, yet no confidence intervals, standard deviations, or significance tests are reported for the feature-count reduction. The abstract's claim that FSNID 'selects a significantly reduced feature set' therefore lacks statistical support; the word 'significantly' is not backed by any test. The authors should report the distribution of selected feature counts across runs and test whether the reduction relative to each baseline is meaningful given the variability of the MINE estimates.","section":"5.3.1, Figure 2"},{"comment":"The claim that FSNID 'maintains NID performance' is supported by showing that accuracy, F1, and false-positive values fall within overlapping confidence intervals with the full-feature classifier. Overlapping intervals are not evidence of equivalence. An equivalence test (e.g., two one-sided tests) or at least a non-inferiority analysis is needed to substantiate the performance-maintenance claim for RQ1. Without this, the empirical conclusion is only that no large degradation was observed on these datasets, which is weaker than the abstract's claim.","section":"5.3.2"},{"comment":"The proof of Theorem 1 contains a direction error in the inequality chain in Eq. (18). From monotonicity of conditional entropy, for a superset relation one obtains H(Y|X) ≤ H(Y|X\\X_j) ≤ H(Y|X_i), not H(Y|X\\X_j) ≤ H(Y|X_i) ≤ H(Y|X) as written. The conclusion can likely be repaired in the specific perfect-redundancy case because the definition gives H(Y|X)=H(Y|X_i), but the proof as printed is incorrect. In addition, Lemma 3's proof relies on an ill-defined 'monotonicity of conditional entropy' applied to the intersection of sets; the relationship between Lemma 3 (which says perfectly redundant variables cannot be removed) and Theorem 1 (which says the sequential algorithm removes them without loss) should be clarified, since as written Lemma 3 appears to contradict the theorem's conclusion.","section":"Appendix B, Theorem 1 proof"}],"minor_comments":[{"comment":"The word 'portrey' in Section 1 should be 'portray'.","section":"1"},{"comment":"Reference [16], attributed to Dorothy Denning, appears to have an incorrect title ('Algorithmic Enumeration of Ideal Classes for Quaternion Orders'); the intended citation is likely Denning's 'An Intrusion-Detection Model' (1987).","section":"References"},{"comment":"The stopping rule for MIFA is under-specified: the paper says features are added from k=5 in increments of 5 until the next set does not give a statistically significant improvement in MI, but it does not state which statistical test is used or how multiple comparisons are handled, which is important for a fair baseline comparison.","section":"Appendix F"},{"comment":"The top-row y-axis label reads '% Features Retained' but the axis values run from 0 to 1 (or 0.0 to 0.8), which suggests the plotted quantity is a fraction, not a percentage; the axis label and units should be made consistent.","section":"Figure 2"},{"comment":"In the Theorem 1 proof, the measure is written as Φ_{X_j;X→A} in several places, while the main text uses Φ_{X_i;X→Y}; the symbol A is not defined in that context (elsewhere A(X) denotes the set of realizations of X), so this is a notational inconsistency that should be corrected.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and a broad empirical evaluation, but the central theoretical guarantee is for exact Φ while the implemented method uses a statistical threshold whose validity is not established. The proof of Theorem 1 also contains a repairable technical error. I believe the issues are fixable within the scope of a revision, so I recommend major revision rather than rejection. The authors should be encouraged to add a normality check or distribution-free alternative for the null-model threshold, report confidence intervals on selected feature counts, and perform an equivalence test for the performance-maintenance claim. I would also point out that the current version does not actually release code, contrary to the abstract's promise."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: FSNID is a sensible feature-selection method for NID, and the empirical story is decent, but the paper's theoretical guarantee does not actually cover the implemented algorithm. The exact condition Phi=0 in Algorithm 1 is replaced by a normality-based 95% threshold on noisy neural estimates, and that substitution is neither analyzed nor validated.\n\nWhat's new: using the difference of two MINE estimates as a transfer-entropy score, sequentially eliminating features, and adding an LSTM variant to fold temporal dependencies into the selection step. None of the baselines do that. The experiments cover five public datasets, six baselines, and the results are consistent: large feature reductions with small accuracy changes. That is worth taking seriously.\n\nThe load-bearing flaw is the gap between Theorem 1 and the implementation. Theorem 1 assumes exact evaluation of Phi; the method estimates Phi as a difference of two noisy neural estimators and then applies the Section 4.7 rule assuming both the null model and the feature estimates are normal. No normality check, no bias/variance analysis, and no handling of the case where many features have small but nonzero Phi. The theorem's conclusion H(Y|X)=H(Y|X*) may not hold for the selected set. The stress-test note is right: the per-feature test can remove individually negligible features while the cumulative loss is large.\n\nThe appendix proofs are also shaky. Lemma 1's equivalence over the powerset is asserted rather than proved, and the Jensen argument in Appendix C has a suspicious step. These are fixable, but they need to be fixed before the claims are rigorous. Empirically, retained feature fractions are reported without confidence intervals, and there is no equivalence test against full-feature classifiers. Code is promised but not available, which makes the threshold behavior harder to audit.\n\nWho this is for: network security researchers and ML practitioners who want a practical, filter-based feature selection method that handles correlation and temporal structure. With revision, this could be a solid contribution. As it stands, it's a promising but over-claimed version.\n\nRecommendation: send to peer review with a strong request for revision. Add threshold validation, correct the proofs, report error bars and equivalence tests. The idea deserves referee time.","headline":"FSNID is a promising feature-selection method with a decent empirical story, but the theoretical guarantee doesn't cover the implemented threshold, and the appendix proofs are shaky.","tokens_in":26041,"tokens_out":2762,"would_cite":false,"duration_ms":26328,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that features with zero estimated transfer entropy to the attack label can be removed one by one, leaving a subset that preserves the same information-theoretic accuracy bound as the full feature set.","keywords":["feature selection","network intrusion detection","transfer entropy","mutual information neural estimation","recurrent neural network","information theory","redundancy and synergy","security classification"],"falsifier":"Run the $\\Phi$ estimator many times on a dataset with a known-irrelevant shuffled feature and plot the distribution of estimated values; a visibly non-normal or heavy-tailed distribution means the 95% threshold will select a different set than intended. A direct test of the theorem is to estimate $H(Y|X^*)$ and $H(Y|X)$ on held-out data and check whether they are equal or whether the selected subset loses information.","tokens_in":24966,"feed_emoji":"🛡️","tokens_out":7106,"duration_ms":64181,"temperature":0.7,"pith_summary":"This paper presents FSNID, a filter method that chooses which network features to keep for intrusion detection by measuring how much each feature reduces uncertainty about whether traffic is an attack. The central claim is that features adding zero conditional mutual information can be removed one at a time, and the selected subset preserves the same information-theoretic bound on classification accuracy as the full feature set. The authors prove this preservation claim as Theorem 1 and support it experimentally on five public NID datasets, where the selected subset is much smaller while accuracy, F1, and false-positive rates stay close to full-feature classifiers. A recurrent version of the estimator also folds temporal dependencies into the selection step, so time-correlated attack patterns are not lost when features are dropped.","feed_headline":"A transfer-entropy filter keeps only attack-relevant features","feed_subtitle":"Fewer monitored features means cheaper and stealthier intrusion detection without losing accuracy","key_machinery":"The central object is the transfer-entropy measure $\\Phi_{X_i;\\,X\\to Y}=H(Y|X\\setminus X_i)-H(Y|X)$, estimated as the difference of two neural mutual-information estimates built on the Donsker-Varadhan representation. The vanishing condition $\\Phi=0$ identifies features that do not reduce uncertainty about $Y$; Algorithm 1 evaluates each feature once, removes it if $\\Phi=0$, and thus handles correlated and redundant features while setting the final feature count automatically. For temporal data, the estimator takes sequences of length $s$ and uses an LSTM, so the importance score reflects information carried across time rather than within a single snapshot.","core_discovery":"The paper's central discovery is that the drop in conditional entropy of the attack label when a feature is added back to the remaining set, written $\\Phi_{X_i;\\,X\\to Y}=H(Y|X\\setminus X_i)-H(Y|X)$, is a sound basis for greedy feature removal in NID. Features with $\\Phi=0$ are classified as irrelevant, redundant, or perfectly redundant, and the paper argues that sequential removal handles the perfectly redundant case so that the final set $X^*$ satisfies $H(Y|X)=H(Y|X^*)$. Consequently, by known information-theoretic bounds on classification, a classifier trained on $X^*$ has the same theoretical accuracy ceiling as one trained on all features. In experiments, FSNID retains far fewer features than baselines, and the LSTM version retains fewer still, with comparable classification performance.","pith_inferences":["A direct extension is to apply the same sequential $\\Phi=0$ rule to other high-dimensional classification problems, but the normality assumption on estimated scores would need to be re-validated in each new domain.","If the neural estimates are biased so that the ordering of small positive $\\Phi$ values is unstable, the selected set could silently drop weakly informative features; running selection over multiple random seeds and measuring the stability of $X^*$ would test this.","The identity $H(Y|X)=H(Y|X^*)$ is an information-theoretic statement, not a guarantee for a specific classifier, so in practice finite-sample behaviour will also depend on the model's inductive bias.","The per-feature information value could be reused for active data acquisition, deciding which network measurements to collect next rather than only which to keep."],"forward_implications":["If $H(Y|X)=H(Y|X^*)$ holds, any classifier trained only on the selected features inherits the full-feature-set accuracy bound, so the selection step does not cap detection performance.","The method requires no preset feature count, solving the length-$k$ problem in a single linear pass over the features.","Because only features with zero estimated transfer entropy are dropped, synergistic features—those useful only in combination—are protected by the sequential evaluation.","The LSTM variant can reduce the feature set further on time-dependent attack traffic, since temporal information is included in the importance score."],"supporting_citations":[{"why":"Supplies the neural mutual-information estimator and the Donsker-Varadhan representation used to compute $\\Phi$.","marker":"[5]"},{"why":"Defines transfer entropy, the quantity that $\\Phi$ adapts for feature importance.","marker":"[54]"},{"why":"Provides the information-theoretic bound linking mutual information to classifier accuracy, used to argue that $X^*$ preserves the accuracy ceiling.","marker":"[52]"},{"why":"Gives the null-model method for deciding when an estimated $\\Phi$ value is indistinguishable from zero.","marker":"[69]"},{"why":"Underlies the fixed 95% statistical test applied to the null model.","marker":"[45]"},{"why":"Defines the LSTM layer used to incorporate temporal dependencies into the estimator.","marker":"[23]"}],"fun_headline_variants":["Entropy drop filter keeps only attack-relevant features","Conditional entropy guides feature pruning for NID","Trimmed features, same intrusion detection accuracy","LSTM feature selection shrinks NID data without loss","Pick features by entropy drop to cut NID costs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The selection threshold assumes the estimated transfer-entropy values for both the noise variable and the real features are normally distributed, so a fixed 95% statistical test separates informative from uninformative features.","fun_headline_variants_meta":{"raw":{"variants":["Entropy drop filter keeps only attack-relevant features","Conditional entropy guides feature pruning for NID","Trimmed features, same intrusion detection accuracy","LSTM feature selection shrinks NID data without loss","Pick features by entropy drop to cut NID costs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00033,"raw_usage":{"total_tokens":1840,"prompt_tokens":948,"completion_tokens":892,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":818}},"tokens_in":564,"tokens_out":892,"duration_ms":9573,"temperature":1.0,"reasoning_tokens":818,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:20:54.149298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the $\\Phi$ estimator many times on a dataset with a known-irrelevant shuffled feature and plot the distribution of estimated values; a visibly non-normal or heavy-tailed distribution means the 95% threshold will select a different set than intended. A direct test of the theorem is to estimate $H(Y|X^*)$ and $H(Y|X)$ on held-out data and check whether they are equal or whether the selected subset loses information.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the neural mutual-information estimator and the Donsker-Varadhan representation used to compute $\\Phi$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines transfer entropy, the quantity that $\\Phi$ adapts for feature importance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the information-theoretic bound linking mutual information to classifier accuracy, used to argue that $X^*$ preserves the accuracy ceiling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the null-model method for deciding when an estimated $\\Phi$ value is indistinguishable from zero."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Underlies the fixed 95% statistical test applied to the null model."}],"review_version":1}