{"id":"e2619190-e215-4a94-a8d3-b268f6854f30","arxiv_id":"2504.20681","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On a dataset of 11,928 files encrypted by 75 ransomware families, Hoeffding Tree classifiers best detect full and AES-Base64 encryption, while Random Forest with differential entropy best detects intermittent encryption.","lead":"Ransomware increasingly hides its encryption by only scrambling parts of files or by encoding data to look normal. The authors test machine-learning models that watch file statistics and report which models catch which hiding tricks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unspecified train/test split can inflate accuracy via file-identity or family leakage; the real-time detection claim is not yet supported.","rationale":"The paper's central claim is that online learners detect encryption status in real time with high accuracy. All evidence for this comes from the 70/30 split described in Section 5.4 and the batched stream evaluation of Section 5.9, yet the paper does not specify whether the split is stratified by source file or ransomware family. Because intermittent encryption patterns are family-specific (Section 3.4), a classifier that has seen a family's skip pattern during training can recognize it again at evaluation, yielding high accuracy without demonstrating detection of unseen adversarial strategies. The abstract explicitly promises adaptation to evolving adversaries, so held-out-family accuracy is the appropriate measure. The same leakage risk affects both the Hoeffding Tree results and the warm-start Random Forest results. I also agree with the reader's secondary observation that the abstract's warm-start Random Forest claim is never explicitly defined or tied to a precise experiment, but the split-leakage issue is more load-bearing because it threatens all reported numbers. Since the data and code are withheld, this cannot currently be checked; the CONDITIONAL verdict is therefore appropriate, and no change is needed.","tokens_in":31065,"tokens_out":7112,"duration_ms":79684,"concrete_test":"Request the authors to re-run the Section 5.4/5.9 experiments with a group-wise split: all copies of the same source file (normal and encrypted) and all files encrypted by the same ransomware family must be confined to either training or test. Then compare the Hoeffding Tree and Random Forest accuracies on held-out families against Tables 4 and 5. If accuracy drops materially below the reported ~90%/~80% thresholds, the published numbers reflect memorization rather than generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that Hoeffding Tree and Random Forest classify encryption status in real time with >90%/>80% accuracy—rests entirely on the 70/30 split described in Section 5.4 and the batched stream evaluation of Section 5.9. The paper never states whether the split separates (a) the normal and encrypted versions of the same source file, or (b) files encrypted by the same ransomware family. If either condition is violated, the reported accuracies can reflect memorization of file identity or family-specific encryption patterns rather than learning encryption statistics that generalize to new files or unseen families. Intermittent encryption is explicitly family-specific (Section 3.4, e.g., Black Basta's 64/128-byte skip pattern), so if files from the same family appear in both training and evaluation, classifiers can exploit the family's pattern; a new family with a different pattern could evade detection. Since the abstract frames the contribution as defending against 'evolving adversaries,' the evaluation must measure held-out-family generalization, not within-family interpolation. No split specification, code, or data is provided to rule this out.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies file-level detection of ransomware encryption using statistical features and online incremental machine learning. It systematizes adversarial encryption strategies (Base64 entropy reduction, partial/intermittent encryption), introduces a feature set including entropy variance and percentiles, compares batch classifiers and online learners on a dataset of 11,928 files encrypted by 75 ransomware families, and claims that Hoeffding Tree achieves above 90% accuracy for traditional, AES-Base64, and partial encryption while Random Forest with warm-start functionality achieves above 80% accuracy for intermittent encryption. The paper also sketches a FUSE-based 'DeltaFile Guard' integration for real-time prediction of file encryption status.","tokens_in":31242,"tokens_out":6956,"duration_ms":66896,"significance":"If the reported claims were fully supported, the work would provide a large-scale empirical comparison of online learning methods for ransomware encryption detection and a feature set that could be useful for practitioners. The dataset scale (75 families, 11,928 files) and the use of real-world ransomware are strengths, as is the attention to evasion techniques such as intermittent and partial encryption. However, the current manuscript lacks the methodological detail needed to substantiate the headline claims: the train/test split is underspecified, the warm-start Random Forest claim is absent from the experiments, and the online evaluation protocol is not described. The post hoc introduction of differential entropy and the lack of statistical rigor further weaken the empirical basis. The paper does not ship code or data, which limits reproducibility.","major_comments":[{"comment":"The 70/30 train/test split is not described with respect to file identity and ransomware-family overlap. The manuscript does not state whether the normal and encrypted versions of the same source file are prevented from appearing in different partition splits, nor whether files encrypted by the same ransomware family are allowed to straddle the split. Because Section 3.4 shows that intermittent-encryption patterns are family-specific (e.g., Black Basta's 64/128-byte skip), within-family interpolation can produce the high accuracies in Tables 4 and 5 without any generalization to new families. Provide the exact split construction (or release code/data) and, ideally, a held-out-family evaluation to support the 'evolving adversaries' claim in the abstract.","section":"5.4"},{"comment":"The abstract and conclusion state that the 'Random Forest classifier with warm-start functionality' excels at intermittent encryption, yet warm-start is neither defined nor used in any experiment in Sections 5.4-5.9. Table 5 Row B describes Random Forests with a differential entropy feature; Row C merely reports a 'Random Forest Score' without mentioning warm-start. Consequently the headline claim about warm-start Random Forest is not supported by the presented evidence. In addition, the Section 7 claim of 'a minimum of 80% accuracy' for intermittent encryption is numerically inconsistent with Table 5 Row B, where Random Forests report 78.32% accuracy; if the claim refers to Row C, that row's protocol must be clearly distinguished. Either add a warm-start Random Forest experiment (with implementation details) or revise the abstract and conclusion.","section":"Abstract, Section 7, Table 5"},{"comment":"The evaluation protocol for online learning is underspecified. The text says data are read in rounds of 1000 with a 'sliding window mechanism', but it is not stated whether the accuracy in each row of Tables 4 and 5 is computed on the batch the model was just trained on (test-on-train) or on the next, unseen batch. If the former, the reported accuracies do not measure predictive performance in a real-time file-system setting, which is the paper's central claim. Specify the training/testing order within each batch and report evaluation on a properly held-out portion of the stream.","section":"5.9"},{"comment":"The differential entropy feature was introduced in the manuscript after observing that Hoeffding Tree underperforms on intermittent encryption, and no separate validation procedure is described for this feature or for the subsequent switch to Random Forest. In addition, Eq. (9) defines differential entropy for continuous random variables, whereas the features are derived from discrete byte counts; the application of continuous differential entropy to discrete file data, including the possibility of negative differential entropy values, is not justified. The claim that Random Forest with differential entropy achieves 'superior predictive accuracy' needs a pre-specified or otherwise properly validated feature-selection protocol and a justification for using continuous differential entropy on discrete data.","section":"5.9, Eq. (9), Table 5 Row B"},{"comment":"Tables 4 and 5 report single-run batch accuracies with no error bars, repeated trials, or class-balance information, making it impossible to assess whether the observed differences between classifiers (e.g., Table 5 Row C) are statistically significant. Batch-to-batch fluctuations are large (for example, Table 4 Row D Hoeffding Tree ranges from 0.85 to 1.00), so without repeated trials or confidence intervals the relative superiority claims are not established. Report mean and standard deviation over multiple runs, state class proportions, and reconcile the claimed 'minimum of 80% accuracy' with the reported 0.7832 figure in Table 5 Row B.","section":"Tables 4, 5; Section 7"},{"comment":"The dataset scale is stated inconsistently: Sections 5.1 and 5.6.1 describe a 32.6 GB dataset with 11,928 files, while Section 5.4 refers to a '65.1 GB (70,008,138,878 bytes)' dataset, and Section 3.6.1 additionally describes a separate 3,200-file (8.13 GB) subset. These numbers need to be reconciled, as the inconsistency affects the claimed scale of the empirical evaluation and the interpretation of the feature-extraction resource table.","section":"5.1, 5.4, 5.6.1"}],"minor_comments":[{"comment":"The column headers in Tables 4 and 5 are garbled (for example, 'Batch SGD Perceptron Passive Aggressive Hoeffding Tree' appears as a single row in parts A-D, and batch numbering starts at 0 or 1 inconsistently across rows). Reformat the tables for readability.","section":"Tables 4, 5"},{"comment":"The name 'Hoeffding Tree' appears as 'Hoeffding', 'Hoe ffding', and 'Hoeffding' in different places (e.g., Section 5.8 vs. Table 4 headings). Standardize the spelling.","section":"Throughout"},{"comment":"The 'percentiles utilization' feature is described verbally, but no formula or algorithmic specification is provided; state exactly how the percentiles are computed (e.g., across byte values of the whole file or per fixed-size block).","section":"5.3, feature 10"},{"comment":"The 'SmartPattern [N,P]' notation for BlackCat intermittent encryption is not defined precisely; explain how N and P determine the byte-skipping pattern.","section":"3.4"},{"comment":"The 'DeltaFile Guard' component is introduced as a named system with a conceptual diagram, but no implementation details, pseudo-code, or evaluation of the FUSE integration are provided; clarify what was actually implemented versus envisioned.","section":"3., Figure 3, 5."}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical core is not yet reliable enough for the claims made in the abstract and conclusion. The missing details about the train/test split, the absence of any warm-start Random Forest experiment, and the unclear online evaluation protocol are all fixable in a revision, so I do not recommend rejection. However, the authors should be required to either provide the missing experiments and data/code or substantially weaken the headline claims. The manuscript also mixes a systematization contribution with an empirical comparison; the journal should consider whether the systems-level framing (DeltaFile Guard) is sufficiently developed for the scope of this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful part is the benchmark: 75 real ransomware families, 11,928 files, including AES-Base64 and intermittent encryption, evaluated under online learning. If the leakage question is resolved, practitioners get a concrete model-selection rule: Hoeffding Tree for full and Base64, Random Forest for intermittent. That is worth knowing even though each component (entropy features, online learning, partial encryption) has been seen before. The formalization of partial and intermittent encryption is clear and grounded in real ransomware behavior, and the CPU/memory cost comparison between traditional and online models is a practical addition.\n\nThe soft spot is load-bearing. The paper never states whether normal and encrypted versions of the same source file, or files from the same ransomware family, are separated across the 70/30 split. If they are not, the classifiers can memorize file identity or family-specific skip patterns. Intermittent encryption is explicitly family-specific (Black Basta's 64/128-byte pattern), so a model that sees that family in training will look great on its test files but may fail on a new family with a different pattern. The abstract promises defense against 'evolving adversaries,' so the evaluation has to measure held-out-family generalization, not within-family interpolation. That is the first question I would ask the authors.\n\nLesser issues, in proportion: no error bars or repeated trials, class balance not reported; the warm-start Random Forest claim in the abstract never appears in an experiment, and Table 5 Row C is not enough to verify it; differential entropy was added after observing Hoeffding Tree's poor intermittent performance and is not ablated, so you cannot tell if the gain comes from the feature or the classifier. The dataset size inconsistency (32.6 GB vs 65.1 GB) is minor but needs a sentence. No code or data is released, which matters for a benchmark paper.\n\nThe central claim does not collapse, but it is conditional and addressable. The questions are answerable with a leakage-aware split, error bars, and a defined warm-start condition. I would send this to peer review with major revisions. For a reader wanting a quick model-selection rule, treat the accuracies as promising but unverified. I would not cite it yet, but I would bring it to the reading group to discuss the split problem.","headline":"The paper's headline result—Hoeffding Tree for full/Base64, Random Forest for intermittent—is a useful empirical rule, but the underspecified train/test split makes the accuracy numbers impossible to trust as reported.","tokens_in":693,"tokens_out":765,"would_cite":false,"duration_ms":23187,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that online incremental classifiers, especially the Hoeffding Tree, can tell ransomware-encrypted files from normal files in real time, and that a warm-started Random Forest is the better tool for intermittent encryption.","keywords":["ransomware detection","intermittent encryption","partial encryption","online incremental learning","Hoeffding Tree","Random Forest","Base64 encoding","file entropy"],"falsifier":"Take the 11,928-file dataset and split by source-file identity so that no normal file and its encrypted counterpart appear in different splits, or train on one corpus and test on a fresh corpus of the same formats; if accuracy on genuinely unseen files drops toward the entropy-only baseline, the high figures came from file-identity memorization rather than learned encryption statistics.","tokens_in":30863,"feed_emoji":"🔐","tokens_out":6754,"duration_ms":64397,"temperature":0.7,"pith_summary":"This paper tries to establish that the encryption status of a file can be predicted in real time from statistical features of its content, without waiting for a ransomware binary to be identified. It argues that online incremental learners, above all the Hoeffding Tree, are the right tool for the job: on files encrypted by 75 real ransomware families, the Hoeffding Tree classifies strong, partial, and AES-Base64 encryption with average accuracy above 90%, while a warm-started Random Forest is the strongest option for intermittent encryption, staying at roughly 80% or better. The stakes are practical: a detector that sits in the file-access path could flag encryption as it happens and protect backups or shared storage. The paper also claims that entropy alone is a weak signal, because Base64 encoding and partial or intermittent encryption keep entropy in near-normal ranges, and that a richer feature set restores separability.","feed_headline":"Online models spot ransomware encryption above 90 percent accuracy","feed_subtitle":"Hoeffding Tree catches full and AES-Base64 encryption live; warm-start Random Forest handles intermittent tricks.","key_machinery":"The load-bearing object is a per-file feature vector: byte entropy, byte frequency, byte variance, kurtosis, skewness, average string and word length, entropy variance across 512-byte blocks, percentile utilization of byte values, and, for intermittent encryption, differential entropy. These features are fed to two online-capable learners. The Hoeffding Tree, also called the Very Fast Decision Tree, builds a decision tree incrementally, examining each instance once, which is what lets it track a changing stream. The Random Forest with warm-start keeps previously learned model state and updates as new batches arrive, which helps it handle the irregular byte patterns of intermittent encryption. The argument turns on pairing the right statistic with the right update rule: content-distribution features separate encrypted from plaintext buffers, and the online update rule adapts when the encryption tactic changes.","core_discovery":"The central claim is that file-level encryption is detectable from content statistics and that the best learner depends on the adversary's encryption mode. On the paper's dataset of 11,928 files (32.6 GB) encrypted by 75 ransomware families, the Hoeffding Tree, a decision tree grown incrementally from a data stream, reaches above 90% accuracy on average against GrandCrab, the lab-developed AES-Base64 pipeline, Paradise's partial encryption, and CryptFile2's RSA-2048 encryption, often hitting perfect accuracy in later batches. Against Black Basta's intermittent encryption, the Hoeffding Tree underperforms, and a Random Forest classifier with warm-start does better, exceeding 80% and reaching about 95% in several batches. The paper further reports that entropy-only classifiers stay near chance on intermittently encrypted files, with the best accuracy at 0.5331 for Decision Trees, while the full feature vector lifts batch Decision Trees to 98.08% accuracy. The online learner also uses far fewer resources than the batch model, about 0.27 seconds of CPU time and 0.016 MB of memory versus 3.76 seconds and 7.47 MB.","pith_inferences":["A step the paper does not take: test how quickly each learner recovers when the encryption tactic switches mid-stream, since real attacks can mix tactics.","A natural extension is chunk-level detection, classifying 512-byte blocks rather than whole files so a guard can alert before an entire large file is encrypted.","The reported 442-second feature-extraction time for a 3.58 GB Excel file suggests whole-file statistics may be too slow for hot storage, so sampling or streaming moments would decide practical feasibility."],"forward_implications":["A user-space file-system guard can classify writes in real time, because the online model needs only about 0.27 seconds of CPU time and 0.016 MB of memory per evaluation, making live interception feasible.","Deployments should be tactic-aware: the Hoeffding Tree for full, partial, and AES-Base64 encryption, and a warm-started Random Forest when intermittent encryption is suspected.","Entropy-only detectors will miss modern ransomware because Base64 encoding and intermittent encryption keep entropy in normal-looking ranges; the richer feature set is the reason the classifiers work.","Because online models update without full retraining, they can keep pace with new ransomware families as long as the underlying statistical features remain discriminative."],"supporting_citations":[{"why":"Supplies the Hoeffding Tree / VFDT algorithm and the result that its incremental trees approximate batch-learned trees.","marker":"[42]"},{"why":"Defines the online-learning setting and first-order algorithms (SGD, Perceptron, Passive-Aggressive) that the paper benchmarks.","marker":"[38]"},{"why":"Argues entropy-based ransomware detection is inadequate, motivating the richer feature set and adversarial encryption evaluation.","marker":"[8]"},{"why":"Shows Base64 encoding can neutralize entropy-based detection, grounding the AES-Base64 adversarial pipeline.","marker":"[7]"},{"why":"Documents Royal ransomware's partial encryption with a 5.245 MB threshold, formalized as partial-encryption modes.","marker":"[25]"},{"why":"Documents intermittent encryption used by LockFile and Black Basta, which the paper formalizes and tests against.","marker":"[26]"},{"why":"Provides Paradise ransomware's size-based partial-encryption behavior used in the evaluation.","marker":"[43]"},{"why":"Supplies the differential-entropy definition added as a feature for the intermittent-encryption case.","marker":"[44]"}],"fun_headline_variants":["Adaptive ML catches ransomware encryption: Hoeffding vs Random Forest","Online models tell apart full and intermittent ransomware encryption","Ransomware battle: incremental learning adapts to ever-changing encryption","Detect ransomware encryption live: two ML tactics for two attack styles"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy depends on the way data was split for training and testing: if the same original file appears in both its normal and encrypted form on both sides of the split, the model can memorize file identity rather than learn what encryption looks like.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive ML catches ransomware encryption: Hoeffding vs Random Forest","Online models tell apart full and intermittent ransomware encryption","Ransomware battle: incremental learning adapts to ever-changing encryption","Detect ransomware encryption live: two ML tactics for two attack styles"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1500,"prompt_tokens":1018,"completion_tokens":482,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":634,"completion_tokens_details":{"reasoning_tokens":411}},"tokens_in":634,"tokens_out":482,"duration_ms":5191,"temperature":1.0,"reasoning_tokens":411,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:22:42.682892+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the 11,928-file dataset and split by source-file identity so that no normal file and its encrypted counterpart appear in different splits, or train on one corpus and test on a fresh corpus of the same formats; if accuracy on genuinely unseen files drops toward the entropy-only baseline, the high figures came from file-identity memorization rather than learned encryption statistics.","supporting_citations":[{"cited_title":"McIntosh, J","cited_arxiv_id":null,"evidence_quote":"Argues entropy-based ransomware detection is inadequate, motivating the richer feature set and adversarial encryption evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows Base64 encoding can neutralize entropy-based detection, grounding the AES-Base64 adversarial pipeline."},{"cited_title":"Constantin, New royal ransomware group evades detection with partial encryption, CSO (December 2022)","cited_arxiv_id":null,"evidence_quote":"Documents Royal ransomware's partial encryption with a 5.245 MB threshold, formalized as partial-encryption modes."},{"cited_title":"Toulas, Ransomware gangs switching to new intermittent encryption tactic, Bleeping Computer (September 2022)","cited_arxiv_id":null,"evidence_quote":"Documents intermittent encryption used by LockFile and Black Basta, which the paper formalizes and tests against."},{"cited_title":"Cocomazzi, Custom-branded ransomware: The vice society group and 22 the threat of outsourced development, accessed: 2023-12-10 (2022)","cited_arxiv_id":null,"evidence_quote":"Provides Paradise ransomware's size-based partial-encryption behavior used in the evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the differential-entropy definition added as a feature for the intermittent-encryption case."}],"review_version":1}