Feature selection of neural networks is skewed towards the less abstract cue

Babette Dellen; Marcell Wolnitza

REVIEW 3 major objections 6 minor 16 references

Feature selection of neural networks is skewed towards the less abstract cue

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read When a neural network can classify from either a low-level statistical pattern or a higher-level symbolic code, it learns the pattern and ignores the symbols; only corrupting the pattern forces the symbols to be learned.

desk verdict The central result is real but the abstraction conclusion is untested: early stopping cuts training before the symbolic cue converges, so the paper shows the first-converging cue wins, not that less abstract cues are preferred. read the letter →

arxiv 1908.03000 v1 pith:ZM4NDLZE submitted 2019-08-08 cs.CV cs.LG

Marcell Wolnitza , Babette Dellen This is my paper · ORCID

classification cs.CVcs.LG

keywords featureselectionneuralnetworksabstractionlevelsyntheticimagessymboliccuesstatisticalpatternslearningdynamics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which of two equally predictive image cues a neural network actually uses for classification. Training on 30 x 30 binary images that contain both a class-correlated random pattern and a class-correlated combination of symbols, the authors find that networks converge to using the pattern and leave symbol-test accuracy near chance (about 33 percent) while pattern-test accuracy exceeds 95 percent. The symbol code is learned only when 23 percent of pattern labels are corrupted, lowering the pattern's reliability. The authors interpret the preference as a consequence of learning difficulty: the pattern needs about 95 training epochs, the symbols about 850, so gradient descent reaches the easier local minimum first.

What carries the argument

The central object is a synthetic dataset of 30 x 30 binary images in which class membership is correlated with two cues: a pixel pattern drawn from one of three distributions and a combination of three '+' and 'x' symbols. Abstraction level is defined by the paper as learning complexity, measured by the number of epochs until the network classifies the cue correctly. The load-bearing mechanism is the convergence race between two local minima in the loss landscape: when both cues are equally predictive, the network finds the pattern minimum after about 95 epochs, whereas the symbol minimum would require roughly 850 epochs to reach.

What would settle it

Train the same networks on the both-cues dataset for many more epochs without early stopping; if symbol test accuracy rises well above chance while pattern accuracy remains high, the cue preference is an effect of training duration, not of abstraction level.

Watch

Extended reading notes

Core claim

The paper's central claim is that abstraction level, operationalized as the number of training iterations needed for the network to classify a cue correctly, determines which cue is selected. When both cues are present and equally informative, networks with one to ten hidden layers converge to solutions that classify the pattern (96.17 percent test accuracy for a representative one-hidden-layer, 100-neuron network) and fail on the symbols (33.26 percent, near chance). Introducing false pattern labels in 23 percent of the training set raises symbol test accuracy to roughly 73 to 86 percent, indicating the network now exploits both cues. The authors conclude that gradient descent settles into the local minimum that can be reached in fewer iterations.

Load-bearing premise

The paper defines a cue's abstraction level as how many training iterations it takes the network to learn it, so the claim that the less abstract cue wins could simply mean that the easier cue is learned within the early-stopping horizon.

Editorial extensions

If this is right

Networks trained on datasets with redundant cues may silently specialize to the statistically simplest cue, even when a more meaningful cue is available.
Evaluating a classifier only on the training distribution can miss that it ignores an entire class of informative cues.
Corrupting or weakening the easy cue is an effective lever to force the network to use a desired harder cue.
Adding more layers or neurons does not by itself overcome the bias toward the easier cue.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the result suggests a general simplicity bias in which any cue that converges faster dominates; testing with a symbol set that converges faster than the pattern should invert the preference.
Editorial inference: the early-stopping protocol likely drives the effect, since symbols alone need about 850 epochs; removing the cap might let networks learn both cues on the both-cues dataset.
Editorial inference: the setup parallels shortcut learning in natural images, where models latch onto spurious statistical regularities; corrupting the shortcut's labels is a known strategy to force robust features.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague — here's my read on Wolnitza & Dellen. The headline result — fully-connected nets ignore a symbolic code when a pixel-statistics cue is equally predictive — is real in this setup, but the paper's own operationalization of 'abstraction' makes the conclusion partly a restatement of 'the cue that converges first wins under early stopping.'

The good: the two-cue synthetic dataset is clean, the scan across widths and depths is thorough, and the corruption experiment (23% label noise on the pattern flips the network toward symbols) is a nice demonstration that the network's preference can be shifted. The accuracy tables are internally consistent and the main observation replicates across architectures. The paper also honestly reports the epoch counts (Table 6): symbols alone take ~849 epochs to converge, both-cues training stops at ~46 epochs. That is the crux.

The problem: training on Both Cues uses early stopping, and the network stops at 46 epochs — long before the symbolic cue would have been learned. The paper concludes that the 'less abstract' cue is preferred, but 'abstraction' is defined as learning speed, so the result is close to tautological: the faster-to-learn cue wins because the training is cut short. The Dist. Both Cues result actually shows the timing interpretation is live — with the pattern corrupted, training runs to ~70 epochs and the network reaches 85% on symbols. So symbols are not structurally inaccessible; they just need more time. A no-early-stopping run on Both Cues, or a long-training run, would be needed to support the abstraction claim. The paper's own Section 4 explanation ('minimum that can be reached with fewer iterations') makes the dependence on stopping time explicit.

Other notes: the 23% corruption level is never swept, so we don't know how sensitive the shift is. And the paper does not engage with the existing shortcut-learning / texture-bias literature (e.g., Geirhos et al. 2018; Jo & Bengio 2017), which has reported similar phenomena — the novelty lies in the controlled demonstration, not the qualitative finding.

Who this is for: researchers studying shortcut learning, simplicity bias, and interpretability, and anyone designing curricula to steer networks toward symbolic features. The paper is worth a serious referee — the setup is clean and the question is real — but the central claim needs reframing: what is actually shown is that under early stopping, the first-converging cue dominates, and that corrupting the easy cue forces the network to learn the harder one. That is a useful empirical result, but it is not the 'abstraction' story the abstract sells.

Recommendation: send it to review, but ask the authors to run without early stopping and to either decouple 'abstraction' from convergence time or justify the operationalization. I would not cite the abstraction claim as it stands.

Referee Report

3 major / 6 minor

Summary. This paper studies how feed-forward fully-connected neural networks select among two image cues that both correlate with class labels. The low-level cue is a random pattern (uniform, centered, or cornered pixel distributions), and the high-level cue is a code of three symbols ('+' and 'x'). Using synthetic 30x30 binary images, the authors train MLPs on four datasets: Both Cues, Symbol-only, Pattern-only, and a Distorted Both Cues dataset in which 23% of pattern labels are corrupted. The central empirical result (Section 3.1, Tables 4-5) is that networks trained on Both Cues classify the pattern test set at 95-97% accuracy but the symbol test set at only ~33% (chance), so the network appears to rely on the pattern. When the pattern cue is partially corrupted, Symbol test accuracy rises to 73-93% depending on architecture (Section 3.2). The paper interprets this as evidence that neural networks are skewed toward the less abstract cue, where abstraction is defined as the number of training iterations required to learn the cue (Section 2.1.1), and proposes that the network converges to the local minimum reachable in fewer iterations (Section 4).

Significance. Conditional on the interpretation surviving scrutiny, this would be a useful empirical datum for understanding feature selection and inductive bias in neural networks: it provides a controlled synthetic setup in which a specific cue is systematically ignored, and it shows that corrupting the preferred cue can shift the network to the other cue. The paper's strengths are its clean synthetic construction, the systematic sweep over hidden-layer counts (1, 2, 3, 10) and hidden-neuron counts (10, 100, 500), and the transparent reporting of mean accuracies and epoch counts in Tables 7-8 and Table 6. The central observation that the pattern wins when both cues are present is robust across architectures. However, the manuscript overreaches as written: because abstraction is operationalized as convergence speed and because early stopping truncates training on the symbolic cue, the current experiments do not establish a preference for the less abstract cue in any sense independent of learning dynamics. The finding as stated needs either additional experiments without early stopping or a reframed claim about convergence-order effects.

major comments (3)

[Section 2.1.1 and Section 4] The definition of abstraction is circular with the result. The manuscript defines complexity as the number of iterations needed for the neural network to classify the feature correctly (Section 2.1.1), and then in Section 4 explains the Both Cues result by saying that two equally deep local minima exist and the network will converge to the configuration corresponding to the minimum that can be reached with fewer iterations. Under this definition, the less abstract cue is preferred is equivalent to the cue that converges faster is preferred, which is a restatement of the observed learning dynamics rather than an independent explanation. To support the abstraction claim, the authors should either define abstraction using an independent criterion (for example compositional structure or a human-labeling study) and then show the network prefers the lower-abstraction cue, or explicitly relabel the claim as a preference for faster-to-learn cues.
[Section 2.2.2 and Table 6] Early stopping confounds the central result. Training is capped at 1000 epochs with early stopping (Section 2.2.2), and Table 6 shows that Both Cues training converges in 45.8 ± 0.8 epochs while Symbol-only training requires 849.0 ± 115.45 epochs. Thus a network trained on Both Cues is stopped long before the symbolic cue has had the opportunity to be learned, so the low Symbol test accuracy may simply reflect an inadequate training budget rather than a structural or abstraction-based preference. The Dist. Both Cues row in Table 5 shows that symbols can be learned to 85-93% accuracy within 69.6 epochs when the pattern is corrupted, which demonstrates that the symbolic cue is not inherently inaccessible. The authors should run the Both Cues condition without early stopping (for example, a fixed 1000-epoch budget or training until Symbol test accuracy saturates) and report whether Symbol test accuracy remains at chance; otherwise the claim in the Abstract and Section 3.1 overreaches.
[Section 2.1.2 and Section 3.2] The secondary claim about shifting learning to the high-level cue rests on a single arbitrary corruption level. The 23% corruption fraction is introduced without justification and no sweep is performed, so the statement that symbol-like entities are only learned if the importance of low-level cues is reduced is supported by one operating point. A corruption-fraction sweep (for example, 0%, 10%, 23%, 40%, and 50%) with corresponding Symbol test accuracies is needed to establish the threshold or monotonic relationship implied by the claim.

minor comments (6)

[Throughout] There are several typographical issues, including P atterntest and P atterntrain spacing in Table 3 and the sentence The results suggests in Section 3.1; these should be corrected in a revision.
[Section 2.2.2] The training description does not state the early-stopping patience threshold (how many epochs without improvement trigger stopping), which matters for interpreting the epoch counts in Table 6.
[Section 3.3] The claim that batch normalization did not have a significant impact is reported without any supporting results; either provide the data or remove the statement.
[Section 4] The sentence asserting that two equally deep local minima exist is speculative and is not established by the experiments; if retained, it should be explicitly labeled as a hypothesis.
[Section 2.1] The paper does not state random seeds or provide code or dataset generation scripts; since the datasets are synthetic, reporting seeds and releasing the generator would improve reproducibility.
[Table 8] The 10-hidden-layer results show high variance (for example, Symbol train with 10 neurons gives 85.13 ± 12.80%), which complicates the claim that the effect holds in networks with many layers; the authors should discuss this variance explicitly.

Circularity Check

1 steps flagged · score 4.0 of 10

The 'less abstract cue wins' claim is partly a restatement of the paper's own operational definition of abstraction as training iterations, although the accuracy measurements themselves are not circular.

self definitional [Section 2.1.1 (Cues) and Section 4 (Conclusions)]
"We define complexity as the number of iterations needed for the neural network to classify the feature correctly. In this case higher level means that more iterations are needed (see Section 4). [...] Thus, if both cues are present in the dataset, two equally deep local minima exist, and the network will converge to the configuration corresponding the minimum that can be reached with fewer iterations."

The paper defines 'higher/lower abstraction level' through 'complexity', i.e., the number of training iterations needed to classify the cue. The conclusion that the 'lower abstraction' cue is preferred therefore reduces to the statement that the cue requiring fewer iterations is preferred. Section 4 makes this explicit: the network converges to the minimum reachable with fewer iterations. The Both Cues run stops at 45.8±0.8 epochs (Table 6) while Symbol-only needs 849.0±115.45 epochs, so under early stopping the pattern is the only cue with time to be learned. The accuracy asymmetry is an empirical measurement, but the 'abstraction' explanation is a renaming of convergence speed.

full rationale

The paper's core numbers are held-out test accuracies (e.g., 96.17% on Patterntest vs 33.26% on Symboltest after Both Cues training, Table 4) and are not themselves circular: they are external measurements on test subsets. No self-citations or imported uniqueness theorems are load-bearing. The circularity is confined to the conceptual labeling: Section 2.1.1 defines complexity/abstraction as the number of iterations needed to classify a feature, and Section 4 then 'explains' the preference for the low-level cue by the fact that it converges in fewer iterations. With that definition, 'the less abstract cue is preferred' is equivalent to 'the faster-converging cue is preferred.' The early-stopping protocol (maximum 1000 epochs, Section 2.2.2) plus the convergence times in Table 6 (Both Cues 45.8 epochs vs Symbol 849.0 epochs) makes the timing interpretation concrete; the Dist. Both Cues result (69.6 epochs, ~85% Symbol test accuracy) shows the symbolic cue can be learned when the pattern is corrupted, so the conclusion about abstraction level as such is not independently established. This is a validity concern more than a circularity of the measurements, so the score is moderate rather than high.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper's contribution is an experimental measurement, not a derivation. Its conclusion depends on a hand-selected corruption level (23 percent), an ad hoc definition of abstraction via training iterations, and an untested conjecture about local minima. No new physical or mathematical entities are introduced.

free parameters (1)

Pattern corruption fraction in Dist. Both Cues dataset = 0.23 (23%)
Hand-chosen to reduce the correlation of the low-level cue with class labels. No sensitivity analysis is reported, so the conclusion that symbols are learned depends on this specific value.

assumptions (3)

ad hoc to paper Abstraction level of a cue is defined by the number of iterations needed for the network to classify it correctly.
Section 2.1.1 defines complexity this way. The central conclusion ('prefers less abstract cue') then reduces to 'prefers the faster-to-learn cue', making the main claim partly definitional.
ad hoc to paper When both cues are present, two equally deep local minima exist and the network converges to the one reachable in fewer iterations.
Section 4 proposes this explanation without direct evidence. It is a conjecture used to interpret the observed cue preference.
domain assumption Early stopping and the 1000-epoch cap allow convergence to the final solution for all datasets.
Section 2.2.2. If the symbol solution requires more than 1000 epochs or more patience, the observed preference would be an artifact of the stopping rule, which is not tested.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Feature selection of neural networks is skewed towards the less abstract cue." pith.science (2026). https://pith.science/paper/ZM4NDLZE

@misc{pith2026190803000,
  author       = {Pith},
  title        = {Pith review of: Feature selection of neural networks is skewed towards the less abstract cue},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZM4NDLZE}},
  note         = {Machine review of arXiv:1908.03000}
}

read the original abstract

Artificial neural networks (ANNs) have become an important tool for image classification with many applications in research and industry. However, it remains largely unknown how relevant image features are selected and how data properties affect this process. In particular, we are interested whether the abstraction level of image cues correlating with class membership influences feature selection. We perform experiments with binary images that contain a combination of cues, representing two different levels of abstractions: one is a pattern drawn from a random distribution where class membership correlates with the statistics of the pattern, the other a combination of symbol-like entities, where the symbolic code correlates with class membership. When the network is trained with data in which both cues are equally significant, we observe that the cues at the lower abstraction level, i.e., the pattern, is learned, while the symbolic information is largely ignored, even in networks with many layers. Symbol-like entities are only learned if the importance of low-level cues is reduced compared to the high-level ones. These findings raise important questions about the relevance of features that are learned by deep ANNs and how learning could be shifted towards symbolic features.

Figures

Figures reproduced from arXiv: 1908.03000 by the authors.

**Figure 1.** Depiction of the training scenarios. We train a neural network for each dataset: Both Cuestrain A, Symboltrain B, Patterntrain C and Dist. Both Cuestrain D. Datasets Artificial Neural Networks (A/B/C/D) Class Probabilities [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Depiction of the test procedures. We evalu [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Neural network architecture with one hidden layer [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Examples of decaying test accuracies by adding mor [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 12 canonical work pages

[1]

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeﬀrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoﬀrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray,...

work page 2015
[2]

Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classiﬁer probes. arXiv preprint arXiv:1610.01644 (2016)

arXiv 2016
[3]

Rana Ali Amjad and Bernhard C Geiger. 2018. Learning Representations for Neural Network-Based Classiﬁcation Using the Information Bottleneck Principle. arXiv preprint arXiv:1802.09766 (2018)

arXiv 2018
[4]

Geoﬀrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine 29, 6 (2012), 82–97

work page 2012
[5]

David H Hubel and Torsten N Wiesel. 1962. Receptive ﬁelds, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology 160, 1 (1962), 106–154

work page 1962
[6]

Alex Krizhevsky, Ilya Sutskever, and Geoﬀrey E Hinton. 2012. Imagenet classiﬁcation with deep convolutional neural networks. In Advances in neural information processing systems . 1097–1105

work page 2012
[7]

Yann LeCun, Yoshua Bengio, and Geoﬀrey Hinton

work page
[8]

Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. 2014. On the number of linear regions of deep neural networks. In Advances in neural information processing systems . 2924–2932

work page 2014

Show all 16 references

[9]

Razvan Pascanu, Guido Montufar, and Yoshua Bengio
[10]

Andrew Michael Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan Daniel Tracey, and David Daniel Cox. 2018. On the information bottleneck theory of deep learning. https://openreview.net/forum?id=ry_WPG-A-. Proc. International Conference on Learning Re...

2018
[11]

Ravid Shwartz-Ziv and Naftali Tishby. 2017. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810 (2017)

2017 arXiv
[12]

Naftali Tishby and Noga Zaslavsky. 2015. Deep learning and the information bottleneck principle. In Information Theory Workshop (ITW), 2015 IEEE . IEEE, 1–5

2015
[13]

Vladimir N Vapnik. 2000. The nature of statistical learning theory. Springer Science & Business Media

2000
[14]

Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In European conference on computer vision . Springer, 818–833. APPENDIX A: ALL RESULTS Table 7: Mean test accuracies for diﬀerent setups of neural n etworks and training datasets. # hi...

2014
[2013]

arXiv preprint arXiv:1312.6098 (2013)

On the number of response regions of deep feed forward networks with piece-wise linear activations. arXiv preprint arXiv:1312.6098 (2013)

2013 arXiv
[2015]

nature 521, 7553 (2015), 436

Deep learning. nature 521, 7553 (2015), 436

2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeﬀrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoﬀrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray,...

work page 2015

[2] [2]

Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classiﬁer probes. arXiv preprint arXiv:1610.01644 (2016)

arXiv 2016

[3] [3]

Rana Ali Amjad and Bernhard C Geiger. 2018. Learning Representations for Neural Network-Based Classiﬁcation Using the Information Bottleneck Principle. arXiv preprint arXiv:1802.09766 (2018)

arXiv 2018

[4] [4]

Geoﬀrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine 29, 6 (2012), 82–97

work page 2012

[5] [5]

David H Hubel and Torsten N Wiesel. 1962. Receptive ﬁelds, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology 160, 1 (1962), 106–154

work page 1962

[6] [6]

Alex Krizhevsky, Ilya Sutskever, and Geoﬀrey E Hinton. 2012. Imagenet classiﬁcation with deep convolutional neural networks. In Advances in neural information processing systems . 1097–1105

work page 2012

[7] [7]

Yann LeCun, Yoshua Bengio, and Geoﬀrey Hinton

work page

[8] [8]

Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. 2014. On the number of linear regions of deep neural networks. In Advances in neural information processing systems . 2924–2932

work page 2014