{"id":"a62b65be-6ec2-474b-b37f-bfca6f4d4e81","arxiv_id":"2504.16075","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Training a random forest to discriminate real data from uniform synthetic data yields distances that stretch the boundary of the data, improving unsupervised anomaly detection accuracy on ADBench benchmarks.","lead":"This paper shows that an unsupervised random forest trained to tell real data from synthetic data spread evenly across a dataset's range produces distances that exaggerate how isolated outliers are. That transformation improves anomaly detection on standard benchmarks and adds a way to explain why individual points are flagged.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dataset selection is load-bearing: Table 1 shows RFuni underperforms on most excluded ADBench datasets, so the claimed superiority may be an artifact of post-hoc exclusions.","rationale":"The reader's weakest assumption is dataset exclusions, and I agree. The paper's own Table 1 is the decisive evidence: on excluded datasets that are large and not obviously class-labeled anomalies (e.g., Skin, SpamBase, InternetAds), RFuni's AUC is well below the best baseline. This means the exclusions are not neutral pruning; they remove cases that contradict the central claim. Other issues raised by the reader (missing error bars, no code/hyperparameters) affect confidence and reproducibility but do not have direct evidence of directional bias. A re-run on the full ADBench set is a feasible, decisive check. If the result persists, the concern is resolved; if not, the paper should be revised to either justify the selection with pre-specified criteria or soften the claim. Since this condition is exactly what the reader's CONDITIONAL verdict requires, I keep the verdict unchanged.","tokens_in":14090,"tokens_out":8952,"duration_ms":86843,"concrete_test":"Re-run the complete ADBench benchmark on all 47 datasets using the ADBench standard protocol (full datasets, no label-based exclusions). Recompute RFuni's median rank and the Holm-Bonferroni-adjusted Conover tests against the 13 baselines. If RFuni's median rank is no longer 3 or it is not significantly better than at least half of the baselines at alpha = 0.05, the claimed superiority is an artifact of the dataset selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an empirical benchmark claim, so the composition of the benchmark is the load-bearing element. The paper excludes 21 of the 47 ADBench datasets: all datasets with fewer than 1,000 observations plus nine datasets whose anomaly labels the authors judge not to be true anomalies. These exclusions are not performance-neutral. In the paper's own Table 1, RFuni is at or near the bottom on several of the excluded datasets (Skin: 0.35 vs IForest 0.67; SpamBase: 0.49 vs KNN 0.73; InternetAds: 0.56 vs KNN 0.69; Optdigits: 0.45 vs COPOD 0.70; Landsat: 0.56 vs KNN 0.62). If the 26-dataset result were re-aggregated with these excluded datasets, RFuni's median rank and its number of significant wins in the Conover/Holm-Bonferroni comparison would very likely deteriorate. The label-quality criterion is applied judgmentally and only to datasets that, by the authors' own table, disfavor the proposed method; without an operationalized, pre-registered inclusion rule, the 'large number of benchmark datasets' claim is not a stable empirical finding. The paper is transparent in reporting excluded results, but transparency does not remove the selection dependence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes RFuni, an unsupervised anomaly detection pipeline that (i) trains a random forest to discriminate between the observed data and synthetic points drawn from a uniform distribution over the observed feature bounds (the 'Addcl2' protocol of Shi and Horvath), (ii) computes geometry- and accuracy-preserving (GAP) random-forest distances from leaf co-occurrence, and (iii) scores each point by its median GAP distance to the most central 50% of the sample, with centrality defined by each point's median GAP distance to all other points. The paper's central claim, stated in the abstract, is that RFuni distances improve unsupervised anomaly detection accuracy relative to distances from ExtraTrees and relative to 13 standard unsupervised detectors, as demonstrated on a curated subset of the ADBench benchmark (26 of 47 datasets). Supporting analyses include a two-dimensional toy model demonstrating that RFuni anisometrically expands the boundary of the data manifold (with ExtraTrees distances approximating Euclidean distances), a missing-data robustness study under MCAR up to 60% missingness, and an explainability method that constructs counterfactual trajectories along the local gradient of the outlier score and attributes feature importance by counting random-forest partition crossings along each trajectory, illustrated with an MNIST digit-4-versus-digit-9 experiment.","tokens_in":14378,"tokens_out":25145,"duration_ms":218878,"significance":"If the benchmark claim survives scrutiny, the contribution is meaningful: RFuni is an off-the-shelf detector that needs no per-dataset feature scaling or categorical encoding, runs on data with missing values, and can attach local counterfactual explanations to predictions, which few proximity-based unsupervised detectors offer. The evaluation has real strengths: the Euclidean-distance control with the identical scoring function isolates the effect of the RFuni transformation; the main comparison uses Wilcoxon signed-rank tests and a Friedman/Conover post-hoc analysis with Holm-Bonferroni correction; and the excluded-dataset results are reported in full (Table 1) rather than hidden. The toy-data analysis (Spearman rank correlations, MDS stress, distance histograms) provides concrete evidence for the claimed geometric mechanism. The main risk is selection dependence: the 26-dataset benchmark is chosen by criteria that are partly subjective and, by the paper's own Table 1, partly correlated with poor RFuni performance; on the full ADBench the headline claim may not hold. This is fixable within the scope of a revision.","major_comments":[{"comment":"The central benchmark claim is established only on 26 of the 47 ADBench datasets, and the exclusion is not performance-neutral. From the paper's own Table 1, on the ten excluded datasets with at least 1,000 samples (ALOI, Cardiotocography, Fault, InternetAds, Landsat, Magic.gamma, Optdigits, Skin, SpamBase, Yeast), RFuni's AUCROC is below at least three of the five comparison detectors on seven of the ten datasets (notably Skin 0.35 vs KNN 0.72/IForest 0.67; SpamBase 0.49 vs KNN 0.73; Optdigits 0.45 vs COPOD 0.70; InternetAds 0.56 vs KNN 0.69; Fault 0.46 vs KNN 0.73; Magic.gamma 0.69 vs KNN 0.77; Yeast 0.39 vs LOF 0.46). The statement in Sec. 2.4 that 'no significant differences' were observed among detectors on excluded datasets is not sufficient to neutralize this: with 21-22 datasets and many detectors, the test is underpowered, and the claim does not address RFuni's median rank or the Conover/Holm-Bonferroni comparison specifically. Because the abstract's claim is a benchmark claim, the paper should report the full pairwise comparisons on all 47 ADBench datasets (or on a standard, pre-specified subset of ADBench) and show RFuni's rank distribution on included versus excluded datasets; otherwise the headline result cannot be distinguished from a selection effect.","section":"Sec. 2.4, App. 6.1, Table 1"},{"comment":"The exclusion protocol is not auditable and contains a performance-based criterion. Appendix 6.1 lists three exclusion categories '# Samples', '% Anomalies', and 'Dataset', but no thresholds are provided for sample size or anomaly proportion, and dataset-specific reasons are given for only two datasets (Optdigits and ALOI, footnote 5). The ALOI justification is explicitly outcome-based ('almost all unsupervised detectors performed below chance on this dataset'); excluding a dataset because detectors fail on it is circular in a benchmark whose purpose is to evaluate detectors, and it removes precisely the kind of data on which the claim could fail. There is also a numerical inconsistency: the text states that nine datasets were excluded on label-quality grounds and that size filtering leaves 26 of 47 datasets, which implies 21 excluded datasets, while Table 1 lists 22 excluded datasets; these counts must be reconciled. In addition, the size-exclusion justification cites a paper on conformal out-of-distribution evaluation ([Novello et al., 2024]), which does not establish that AUCROC comparisons on datasets with fewer than 1,000 points are invalid. I ask the authors to replace the current post-hoc protocol with a pre-specified, data-independent inclusion rule (e.g., minimum anomaly count and maximum anomaly proportion chosen before looking at results), or to present the full-ADBench analysis as primary and the 26-dataset analysis as a sensitivity check.","section":"App. 6.1, Sec. 2.4"}],"minor_comments":[{"comment":"The cross-detector comparison is reported at a lenient significance level (α=0.1 in both the Conover post-hoc table and the critical-difference diagram), and the Friedman test statistic that justifies the post-hoc analysis is not reported; the authors should state whether the claim that RFuni is significantly better than half of the other detectors survives α=0.05.","section":"Sec. 3.2, Fig. 2.b"},{"comment":"The definition of Xcentral and O(xi) does not explicitly exclude the self-distance (which is zero in Eq. (2)); because xi belongs to Xcentral for many points, the median distance to Xcentral can include this zero, and the paper should state whether self-distances are excluded and confirm the result is insensitive to this choice.","section":"Sec. 2.3"},{"comment":"The random-forest configuration (number of trees, split criterion, minimum samples per leaf), the number of synthetic points used to train RFuni, the neighborhood size k in the gradient estimate, and the trajectory learning rate l are not reported; for a benchmark paper, a reproducibility table with these values, software versions, and code (or a code repository) is needed.","section":"Sec. 2.2, 2.3, 3.4"},{"comment":"The normalization in Fig. 3.b (AUCROC relative to the best detector on complete data) conflates absolute degradation with relative ranking; reporting absolute AUCROC, or normalizing each detector to its own complete-data performance, would make the missingness comparison more interpretable.","section":"Sec. 3.3, Fig. 3.b"},{"comment":"The references 'Rhodes et al., 2023a' and 'Rhodes et al., 2023b' are the same publication listed twice, and only the 2023b entry is cited in the text (Sec. 2.2); the duplicate entry should be removed and the in-text citation reconciled.","section":"References"},{"comment":"The discussion acknowledges that RFuni is sensitive to extreme values that determine the uniform bounds and proposes percentile-based bounds as a mitigation, but no experiment quantifies this sensitivity; given the abstract's claim of minimal preprocessing, an experiment comparing bound-based and percentile-based synthetic sampling would strengthen the robustness story.","section":"Sec. 4"},{"comment":"Typos and caption artifacts: 'maximnum' should be 'maximum' in Sec. 3.2, and the label 'LODASOD COF' in the Fig. 2.a(i) axis appears to be a rendering artifact; these should be corrected.","section":"Sec. 3.2, Fig. 2 captions"}],"recommendation":"major_revision","confidential_remarks":"Major revision is appropriate rather than rejection: the dataset-selection sensitivity is the one load-bearing problem, and it is fixable by re-analysis (full-benchmark results or pre-specified inclusion rules). The authors are transparent about the excluded datasets, and the direct controls are carefully designed; I would welcome a second opinion on whether the exclusion protocol follows ADBench community norms, since the original ADBench evaluation uses all 47 datasets. Novelty is incremental but adequate for a specialized ML journal, and the benchmark claim is the main contribution. No concerns about citation practice beyond the duplicate Rhodes entry noted in the minor comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: RFuni is a reasonable idea, and the paper is more transparent than most, but the headline claim of superior performance across 'a large number of benchmark datasets' depends on dataset exclusions that are doing a lot of work. I would want the inclusion rule nailed down before believing the general claim.\n\nWhat's actually new: combining the Addcl2 uniform-synthetic-data RF distance with GAP proximities and a hyperparameter-free central-median score. The control experiment—same scoring on Euclidean distances—is exactly the right check, and it supports the claim that the RFuni transformation, not the scoring, drives the improvement. The comparison against 14 detectors with Holm-Bonferroni-corrected Conover tests is careful. The missing-data experiment is a genuine plus, as is the visualization showing that RFuni expands distances at the boundary of the data manifold.\n\nThe soft spots are real but not fatal. The evaluation excludes 21 of 47 ADBench datasets; a size cutoff is defensible, but the judgment-based exclusion of datasets where the 'anomalies' are really a distinct class is subjective, and the appendix table shows RFuni at or near the bottom on several of those excluded datasets (Skin, SpamBase, InternetAds, Optdigits, Landsat). That means the main result is sensitive to exactly the datasets the authors chose to set aside. They are transparent about it, which earns credit, but transparency doesn't make the selection neutral. No code or hyperparameters are provided, so the core comparison cannot be independently reproduced. The main benchmark figure also lacks error bars, though the statistical tests partly compensate. The explainability section is a sketch with a single MNIST example, not a systematic evaluation.\n\nBottom line: this deserves peer review. The method is plausible, the mechanism is clear, and the empirical work is mostly careful. But the revision needs to operationalize the inclusion criteria, release code or at least detailed hyperparameters, and re-run the analysis under a sensitivity check that includes the excluded datasets. For a reading group, it's worth a discussion of benchmark selection more than a definitive method win.","headline":"Worth a look for the RFuni distance idea and the honest benchmark reporting, but the headline 'large number of datasets' claim is more fragile than it looks once you see what got excluded.","tokens_in":14872,"tokens_out":4460,"would_cite":false,"duration_ms":41586,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A random forest trained to tell real data from uniform synthetic noise learns distances that flag anomalies more accurately than common detectors.","keywords":["unsupervised anomaly detection","random forest","similarity learning","GAP proximity","counterfactual explanations","feature importance","ADBench","missing data"],"falsifier":"Run the same unsupervised anomaly detection comparison on all 47 ADBench datasets without exclusions, using the dataset labels as ground truth, and check whether RFuni still shows a statistically significant edge over the other detectors; alternatively, test RFuni specifically on the nine excluded datasets and see whether it loses its advantage there.","tokens_in":13938,"feed_emoji":"🌲","tokens_out":6157,"duration_ms":51296,"temperature":0.7,"pith_summary":"This paper proposes a fully unsupervised anomaly detector built from a Random Forest that is trained to tell real data apart from synthetic data drawn uniformly over the observed feature bounds. The claim is that the resulting RFuni distances anisometrically reshape the data, pulling inliers closer together and pushing boundary points farther out, and that this transformation makes outlier detection more accurate than common unsupervised detectors across 26 benchmark datasets. The approach needs almost no preprocessing, tolerates missing values, and can be packaged with a local explanation mechanism that attributes outlier scores to specific feature changes.","feed_headline":"Train a forest against uniform noise to catch more anomalies","feed_subtitle":"RFuni expands distances at the data boundary, boosting outlier detection and explaining its own scores.","key_machinery":"The load-bearing component is the RFuni representation: a Random Forest trained to discriminate real data from synthetic data sampled uniformly over the ranges of the real features, the scheme originally called Addcl2. Distances are computed from the forest's leaf structure using geometry- and accuracy-preserving (GAP) proximities, which weight in-bag and out-of-bag co-locations differently so that the distances reconstruct the forest's prediction geometry. Outlier scores are then taken as the median GAP distance from each point to the subset of points with the lowest median distances, a hyperparameter-free centrality measure. For explanations, the gradient of the outlier score field yields counterfactual trajectories, and feature importance is read from which forest partition hyperrectangles each trajectory segment crosses.","core_discovery":"The central discovery is that training an unsupervised Random Forest against uniformly distributed synthetic data yields a similarity measure that is not metric-preserving: it deliberately expands distances at the boundary of the data manifold while compressing them near the bulk. The paper shows that outlier scores computed from these GAP distances, taken as the median distance to the most central half of the data, rank anomalies more accurately than distances learned by ExtraTrees or Euclidean baselines, and they outperform about half of the standard detectors in a benchmark comparison at the 0.1 significance level. The same forest partitions then serve as a basis for counterfactual explanations, because each trajectory that reduces an outlier score can be decomposed into the hyperrectangular partitions it crosses.","pith_inferences":["The boundary-expanding property might generalize beyond RFuni's specific scoring rule, so any distance-based detector applied to RFuni distances should inherit the improved separation of outliers.","Because RFuni is sensitive to absolute feature values rather than ranks, monotone feature transformations should change its behavior predictably, offering a direct experimental handle on the mechanism.","Using percentile-based bounds instead of raw data bounds for the uniform synthetic samples could make the method robust to extreme-value contamination, a limitation the paper explicitly flags.","The partition-crossing explanation could be adapted to any tree ensemble, including supervised forests, as a way to explain model predictions via geometric feature attributions."],"forward_implications":["RFuni can be applied to raw datasets with categorical features and missing values, reducing the need for dataset-specific preprocessing in automated pipelines.","The learned distances support low-dimensional visualization via multidimensional scaling, making the boundary-expanding geometry directly inspectable.","Up to 60% missing-at-random data, RFuni remains competitive provided missing values are mean-imputed first, and it can also run natively on data with missing entries.","Feature importance for a detected anomaly can be read from the forest partitions crossed by a counterfactual trajectory, yielding local, model-native explanations.","The advantage is attributable to the RFuni transformation itself: applying the same outlier-scoring rule to Euclidean distances behaves like ExtraTrees, not like RFuni."],"supporting_citations":[{"why":"Introduces the Addcl1/Addcl2 distinction for generating synthetic data in unsupervised random forests; Addcl2 is the core training scheme used here.","marker":"[Shi and Horvath, 2006]"},{"why":"Provides the GAP proximity definition used to convert forest leaf co-locations into geometry-preserving distances.","marker":"[Rhodes et al., 2023b]"},{"why":"Defines ExtraTrees, the randomized-split forest baseline whose distances are shown to approximate Euclidean distances.","marker":"[Geurts et al., 2006]"},{"why":"Supplies the ADBench benchmark datasets on which the performance comparison is carried out.","marker":"[Han et al., 2022]"},{"why":"Defines the kNN outlier scoring approach that serves as a comparison detector and a baseline for distance-based scoring.","marker":"[Ramaswamy et al., 2000]"},{"why":"Originates the idea of training random forests to discriminate real from synthetic data and computing proximities.","marker":"[Breiman and Cutler, 2001]"},{"why":"Provides the critical-difference diagram and statistical comparison methodology used to rank detectors across datasets.","marker":"[Demšar, 2006]"}],"fun_headline_variants":["Forest vs. uniform noise: better anomaly detection with explanations","Train a forest on synthetic uniform data to stretch outlier boundaries","Anisometric forest distances improve unsupervised outlier detection","Explainable outliers: random forest learns boundary-expanding distances"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison that establishes superiority rests on excluding nine ADBench datasets whose anomaly labels the authors judge not to represent genuine anomalies, plus all datasets with fewer than 1,000 rows; if those exclusions are not defensible, the claimed advantage may not hold on the full benchmark.","fun_headline_variants_meta":{"raw":{"variants":["Forest vs. uniform noise: better anomaly detection with explanations","Train a forest on synthetic uniform data to stretch outlier boundaries","Anisometric forest distances improve unsupervised outlier detection","Explainable outliers: random forest learns boundary-expanding distances"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000276,"raw_usage":{"total_tokens":1579,"prompt_tokens":810,"completion_tokens":769,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":426,"completion_tokens_details":{"reasoning_tokens":704}},"tokens_in":426,"tokens_out":769,"duration_ms":7291,"temperature":1.0,"reasoning_tokens":704,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:10:31.084281+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same unsupervised anomaly detection comparison on all 47 ADBench datasets without exclusions, using the dataset labels as ground truth, and check whether RFuni still shows a statistically significant edge over the other detectors; alternatively, test RFuni specifically on the nine excluded datasets and see whether it loses its advantage there.","supporting_citations":[],"review_version":1}