Pith. sign in

REVIEW 3 major objections 5 minor 19 references

Optimising the attribute order in Fuzzy Rough Rule Induction

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that reordering attributes does not improve FRRI's performance, while removing a small fraction of attributes with fuzzy rough feature selection improves balanced accuracy and shortens rules.

desk verdict Useful negative result on attribute ordering in FRRI, but the positive FRFS recommendation rests on an unstated protocol detail that could easily flip it. read the letter →

arxiv 2506.02805 v1 pith:PPA7MJIU submitted 2025-06-03 cs.AI

Henri Bollaert , Chris Cornelis , Marko Palangetić , Salvatore Greco , Roman S{l}owiński This is my paper
classification cs.AI
keywords fuzzyroughsetsruleinductionattributeorderingfeatureselectionQuickReductbalancedaccuracyinterpretabilityFRRI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the order in which attributes are presented to a greedy rule-shortening step changes the rules that Fuzzy Rough Rule Induction (FRRI) produces. Across 18 benchmark datasets, it finds that reordering attributes with three different feature-selection methods—fuzzy rough feature selection, mutual information, and Pearson correlation—does not significantly change balanced accuracy, rule count, or rule length. However, using fuzzy rough feature selection to also drop about 10% of the least relevant attributes does improve balanced accuracy and shortens the average rule, with little effect on the number of rules. The practical upshot is that FRRI users can ignore attribute ordering but may gain from a light feature-selection preprocessing step before running the algorithm.

What carries the argument

The load-bearing mechanism is the greedy rule-shortening pass of FRRI: for each training object, the algorithm walks through attributes in a fixed order and, for each, tries setting the condition to unused, dominant, dominated, or similar, keeping the first type that leaves the rule's covering set inside the object's decision class. Because it is greedy, the resulting rule can depend on the order visited; the paper probes this dependency by feeding FRRI attribute lists reordered by three feature-selection criteria, and by the hill-climbing QuickReduct algorithm used for fuzzy rough feature selection. The fuzzy B-indiscernibility relation (the t-norm aggregation of per-attribute similarity or dominance relations) and the fuzzy positive region's dependency degree $\gamma_B$ are what let both FRRI's shortening and QuickReduct's search evaluate attribute subsets.

What would settle it

On a small dataset with six attributes or fewer, compute FRRI's balanced accuracy and average rule length under every possible global attribute permutation; if any permutation differs from the default by more than the cross-validation variance, the claim that reordering has no effect is falsified. The same test can be run with per-object random permutations, which the paper does not examine.

Watch

Extended reading notes

Core claim

FRRI constructs a fuzzy indiscernibility relation over training objects and turns each object into a total rule whose conditions are greedily shortened attribute by attribute; the authors hypothesised that changing this attribute order would produce smaller, shorter, more accurate rule sets, just as sorted input improves bin packing. The experiments show the hypothesis is false: reordering alone, whether by QuickReduct-style fuzzy rough feature selection, mutual information, or Pearson correlation, leaves balanced accuracy, ruleset size, and average rule length statistically unchanged. When the reordering is paired with the removal of a small fraction (about 10%) of attributes by fuzzy rough feature selection, balanced accuracy improves and rules become shorter, and this gain does not appear with the other two selection methods or with deeper pruning, which degrades accuracy and inflates the ruleset. The paper explains the asymmetry by noting that FRRI removes attributes per object, whereas preprocessing removes attributes globally, so dropping more than a few attributes can discard features that are essential for distinguishing particular objects.

Load-bearing premise

The experiments vary attribute order only through a single global ordering chosen before FRRI runs; the conclusion that order does not matter assumes that this is a faithful test of the per-object attribute order the greedy shortening step actually processes.

Editorial extensions

If this is right

  • FRRI's default attribute order can be kept; no ordering preprocessing is needed.
  • Removing up to about 10% of attributes with fuzzy rough feature selection is a safe preprocessing step that yields shorter rules and slightly higher balanced accuracy.
  • Aggressive pre-pruning, such as cutting to a minimal superreduct, should be avoided because it sharply lowers accuracy and increases the number of rules.
  • The two classical scoring methods, mutual information and Pearson correlation, do not reproduce the accuracy gain, so the choice of feature-selection method matters for FRRI.
  • The results support a division of labour: fuzzy rough feature selection removes globally redundant attributes, while FRRI's shortening handles object-specific redundancy during rule construction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to vary attribute order per training object rather than through a single global ordering; the paper's null result does not cover that setting.
  • Because the paper's own explanation says global pruning can discard object-specific attributes, a natural next experiment is to compare global fuzzy rough feature selection against a per-object minimal-redundancy criterion.
  • The planned reformulation of rule shortening as an exact optimisation problem would make attribute order irrelevant by construction, and this null result is consistent evidence that such a reformulation may be worth the extra cost.
  • On very high-dimensional datasets with hundreds of attributes, the greedy shortening step may become more sensitive to order, so re-testing across a wider range of feature counts would sharpen the scope of the claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper empirically tests whether preprocessing the FRRI rule induction algorithm with feature selection affects its performance. It compares an unmodified control with variants that reorder attributes only (using QuickReduct, mutual information, and Pearson correlation) and variants that reorder and retain 90% or 80% of the attributes, plus a version of FRFS that stops when the dependency reaches 1. The experiments are run on 18 KEEL datasets with ten-fold cross-validation, measuring balanced accuracy, number of rules, and average rule length. The authors conclude that attribute reordering alone does not significantly affect FRRI, while removing up to 10% of attributes with fuzzy rough feature selection can improve balanced accuracy and shorten rules.

Significance. If the conclusions are robust, the paper offers practical guidance for FRRI users and a useful negative result about greedy attribute ordering. The study uses standard benchmarks and appropriate statistical machinery, and it reports three metrics rather than accuracy alone. However, the positive recommendation rests on a one-sided Wilcoxon p-value of 0.071 for balanced accuracy, and the paper does not release code or data. The main strength is the clarity of the research question and the systematic comparison across 18 datasets; the main weakness is that the experimental protocol and statistical evidence are not yet strong enough to support the abstract's positive claim as stated.

major comments (3)
  1. [Section 5.1] The experimental protocol does not state whether the QuickReduct ordering and the 90%/80% retention subsets are computed on the full dataset before the folds are separated or inside each training fold. If the former is the case, the held-out fold contributes to choosing the attribute order and subset used by FRRI, while the control condition receives no such test-set information, which biases the comparison in favor of the preprocessed variants. This is load-bearing because the positive recommendation for ofrfs-0.9 rests on a one-sided Wilcoxon p-value of 0.071 for balanced accuracy and on the corresponding rule-length result; even a small amount of leakage could plausibly produce such differences. Please clarify the protocol and, if feature selection was fitted on the full data, rerun the experiments with feature selection nested inside each training fold.
  2. [Section 5.2, Table 2] The evidence for improved balanced accuracy is weaker than the abstract and conclusion suggest. The only directly reported comparison between ofrfs-0.9 and control is a one-sided Wilcoxon p-value of 0.071, which is not significant at the usual 0.05 level, and the 10% retention level was selected after inspecting Figure 1 across many retention levels without any correction for multiple comparisons. The Conover post-hoc tests only establish that ofrfs-0.9 beats pcc-0.9 and pcc-0.8, not that it beats control. Please report two-sided paired tests and multiplicity-corrected p-values for all retention levels, or downgrade the 'positively affects balanced accuracy' claim to an exploratory finding.
  3. [Section 3.2 and Section 5.2] The negative conclusion that attribute order does not matter is based only on three global orderings applied before FRRI runs. Because rule shortening processes each training object independently over the fixed global sequence, a single global ordering is a restricted probe of the order effect; per-object attribute orderings might behave differently. The discussion paragraph before Section 6 partly acknowledges that feature selection removes attributes globally, but the abstract and conclusion state the broader conclusion without this qualification. Please restrict the claim to global orderings or provide experiments that vary per-object order.
minor comments (5)
  1. [Tables 2-4] The tables report only mean values; adding standard deviations or per-fold results would help readers assess variability, especially for the p=0.071 result.
  2. [Section 5.1, Table 1] The footnote says categorical features were removed from some datasets, but Table 1 does not state which datasets were affected or how many features were removed; please add this information.
  3. [Section 5.1] The phrase 'if this amount is 1 or 0, we default to the next method' is unclear; please specify which method is the fallback and whether it was triggered for any dataset.
  4. [Section 5.2, Figure 1] The label 'average balanced accuracy ... over all datasets' does not specify whether the average is over the ten folds first and then over the 18 datasets, or over fold-dataset combinations; please clarify.
  5. [Section 6] The conclusion says the rule-length improvement appears in 'the majority of benchmark datasets,' but the reported Wilcoxon test is across datasets and does not directly establish a per-dataset majority; a sign test or per-dataset counts would support this phrasing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical comparison of attribute-order preprocessing, with no derivation step that reduces to its inputs.

full rationale

The paper is an empirical comparative study, not a derivation chain. FRRI is taken as a fixed algorithm from prior work (citation [2]), and the manipulated variables — attribute order and attribute subset produced by ofrfs, mi, and pcc — are defined independently of the measured outcomes: balanced accuracy, rule count, and rule length. The central negative conclusion that reordering alone does not improve FRRI is obtained by executing FRRI with different global attribute orders on fixed KEEL folds, not by any equation that equates input and output. The positive finding for ofrfs-0.9 is a statistical comparison against the control, not a fitted parameter renamed as a prediction; no quantity in the paper is defined in terms of the outcome it is used to predict. The self-citation to [2] supplies the algorithm under test and some prior context, but the current conclusions do not depend on the truth of that prior performance claim, and the algorithm's behavior is evaluated externally on benchmark data. A possible protocol ambiguity about where the feature-selection preprocessing is fitted relative to cross-validation folds is a data-leakage or experimental-design risk, not a circularity, since the feature selector remains external to the FRRI objective. No circular step can be quoted from the paper, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the chosen fuzzy rough modeling operators (t-norm, implicator, indiscernibility relation), the normalization scheme, and the benchmark dataset selection. No free parameters are fitted to data; the retention levels are experimental design choices. No new entities are introduced.

assumptions (3)
  • domain assumption All condition attributes are min-max normalized to [0,1] as described in Section 2.3.
    The fuzzy relations Ri and Rd rely on this normalization; results may differ for non-normalized data.
  • domain assumption The fuzzy B-indiscernibility relation is computed with the minimum t-norm over attributes (Equation 5).
    This choice affects the rule shortening and positive region computations.
  • domain assumption The considered benchmark datasets and preprocessing (removing categorical features) are representative for evaluating FRRI.
    Claims about FRRI's order invariance are made on the basis of 18 KEEL datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimising the attribute order in Fuzzy Rough Rule Induction." pith.science (2026). https://pith.science/paper/PPA7MJIU

@misc{pith2026250602805,
  author       = {Pith},
  title        = {Pith review of: Optimising the attribute order in Fuzzy Rough Rule Induction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PPA7MJIU}},
  note         = {Machine review of arXiv:2506.02805}
}
read the original abstract

Interpretability is the next pivotal frontier in machine learning research. In the pursuit of glass box models - as opposed to black box models, like random forests or neural networks - rule induction algorithms are a logical and promising avenue, as the rules can easily be understood by humans. In our previous work, we introduced FRRI, a novel rule induction algorithm based on fuzzy rough set theory. We demonstrated experimentally that FRRI outperformed other rule induction methods with regards to accuracy and number of rules. FRRI leverages a fuzzy indiscernibility relation to partition the data space into fuzzy granules, which are then combined into a minimal covering set of rules. This indiscernibility relation is constructed by removing attributes from rules in a greedy way. This raises the question: does the order of the attributes matter? In this paper, we show that optimising only the order of attributes using known methods from fuzzy rough set theory and classical machine learning does not improve the performance of FRRI on multiple metrics. However, removing a small number of attributes using fuzzy rough feature selection during this step positively affects balanced accuracy and the average rule length.

Figures

Figures reproduced from arXiv: 2506.02805 by the authors.

Figure 1
Figure 1. The average balanced accuracy of ofrfs with an increasing number of retained attributes over all datasets. 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Ratio of remaining number of attributes over the total amount of attributes 0.40 0.45 0.50 0.55 0.60 0.65 0.70 Balanced accuracy is significantly better than pcc-0.9 and pcc-0.8. With a p-value of 0.071, a one-sided Wilcoxon test to determine whether ofrfs-.9 is more … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 14 canonical work pages

  1. [1]

    Journal of Multiple-Valued Logic and Soft Computing17, 255–287 (01 2010) 14 H

    Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework. Journal of Multiple-Valued Logic and Soft Computing17, 255–287 (01 2010) 14 H. Bollaert et al

    work page 2010

  2. [2]

    Information Sciences686, 121362 (2025)

    Bollaert, H., Palangetić, M., Cornelis, C., Greco, S., Słowiński, R.: Frri: A novel algorithm for fuzzy-rough rule induction. Information Sciences686, 121362 (2025). https://doi.org/10.1016/j.ins.2024.121362

    work page arXiv 2025

  3. [3]

    informal report (2 1979)

    Conover, W., Iman, R.: Multiple-comparisons procedures. informal report (2 1979). https://doi.org/10.2172/6057803

    work page doi:10.2172/6057803 1979

  4. [4]

    The American Statistician 35(3), 124–129 (1981)

    Conover, W., Iman, R.: Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician 35(3), 124–129 (1981). https://doi.org/10.1080/00031305.1981.10479327

    arXiv 1981

  5. [5]

    Information Sciences 180(2), 209–224 (2010)

    Cornelis, C., Jensen, R., Hurtado, G., Ślezak, D.: Attribute selection with fuzzy decision reducts. Information Sciences 180(2), 209–224 (2010). https://doi.org/ https://doi.org/10.1016/j.ins.2009.09.008

    work page doi:10.1016/j.ins.2009.09.008 2010

  6. [6]

    In: Wang, G., Li, T., Grzymala-Busse, J.W., Miao, D., Skowron, A., Yao, Y

    Cornelis, C., Martín, G.H., Jensen, R., Ślęzak, D.: Feature selection with fuzzy decision reducts. In: Wang, G., Li, T., Grzymala-Busse, J.W., Miao, D., Skowron, A., Yao, Y. (eds.) Rough Sets and Knowledge Technology. pp. 284–291. Springer Berlin Heidelberg, Berlin, Heidelberg (2008)

    work page 2008

  7. [7]

    International Journal of General Systems 17(2-3), 191–209 (1990)

    Dubois, D., Prade, H.: Rough fuzzy sets and fuzzy rough sets. International Journal of General Systems 17(2-3), 191–209 (1990). https://doi.org/10.1080/ 03081079008935107

    work page 1990

  8. [8]

    Knowledge-Based Systems 80, 109–121 (2015)

    Fernández, A., López, V., del Jesus, M.J., Herrera, F.: Revisiting evolu- tionary fuzzy systems: Taxonomy, applications, new trends and challenges. Knowledge-Based Systems 80, 109–121 (2015). https://doi.org/https://doi.org/ 10.1016/j.knosys.2015.01.013, https://www.sciencedirect.com/science/article/pii/ S0950705115000209, 25th anniversary of Knowledge-Ba...

    work page doi:10.1016/j.knosys.2015.01.013 2015

Show all 19 references
  1. [9]

    Journal of the American Statistical Association32(200), 675–701 (1937)

    Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association32(200), 675–701 (1937)

    1937

  2. [10]

    Gurobi Optimization, LLC: Gurobi Optimizer Reference Manual (2023), https: //www.gurobi.com

    2023

  3. [11]

    In: Proc

    Hinojosa-Cardenas, E., Sarmiento-Calisaya, E., Camargo, H.A., Sanz, J.A.: Im- proving michigan-style fuzzy-rule base classification generation using a choquet- like copula-based aggregation function. In: Proc. WILF 13th Int. Workshop Fuzzy Log. Appl. pp. 1–9 (2021)

    2021

  4. [12]

    Journal of Computer and System Sciences 8(3), 272–314 (1974)

    Johnson, D.S.: Fast algorithms for bin packing. Journal of Computer and System Sciences 8(3), 272–314 (1974). https://doi.org/https://doi.org/10. 1016/S0022-0000(74)80026-7, https://www.sciencedirect.com/science/article/pii/ S0022000074800267

    1974

  5. [13]

    In: FUZZ-IEEE 2022: Pro- ceedings of the IEEE International Conference on Fuzzy Systems

    Lenz, O.U., Cornelis, C., Peralta, D.: fuzzy-rough-learn 0.2: a Python library for fuzzy rough set algorithms and one-class classification. In: FUZZ-IEEE 2022: Pro- ceedings of the IEEE International Conference on Fuzzy Systems. IEEE (2022)

    2022

  6. [14]

    In: International Joint Conference on Rough Sets

    Lenz, O.U., Peralta, D., Cornelis, C.: Fuzzy-rough-learn 0.1: A python library for machine learning with fuzzy rough sets. In: International Joint Conference on Rough Sets. pp. 491–499. Springer (2020)

    2020

  7. [15]

    International Journal of Computer & Information Sciences 11(5), 341–356 (1982)

    Pawlak, Z.: Rough sets. International Journal of Computer & Information Sciences 11(5), 341–356 (1982)

    1982

  8. [16]

    Journal of Machine Learning Research12, 2825–2830 (2011)

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Mach...

    2011

  9. [17]

    Knowledge and Information Systems 66(3), 1575–1637 (2024) Optimising the attribute order in Fuzzy Rough Rule Induction 15

    Theng, D., Bhoyar, K.K.: Feature selection techniques for machine learning: a survey of more than two decades of research. Knowledge and Information Systems 66(3), 1575–1637 (2024) Optimising the attribute order in Fuzzy Rough Rule Induction 15

    2024

  10. [18]

    Biometrics Bulletin 1(6), 80–83 (1945), http://www.jstor.org/stable/3001968

    Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bulletin 1(6), 80–83 (1945), http://www.jstor.org/stable/3001968

    1945

  11. [19]

    Information and Control 8(3), 338–353 (1965)

    Zadeh, L.A.: Fuzzy sets. Information and Control 8(3), 338–353 (1965). https: //doi.org/10.1016/S0019-9958(65)90241-X

    1965 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.