{"id":"3ec3a11c-2e30-4e0f-9ae4-904691e7dd1a","arxiv_id":"1908.03000","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Fully-connected networks trained on images with two equally valid class cues systematically learn the low-level statistical pattern and ignore the symbolic code, unless the pattern is partially corrupted.","lead":"This paper tests whether artificial neural networks prefer simpler, statistical image cues over more abstract, symbolic ones when both predict the class equally well. In controlled binary images, the networks learned the statistical pattern and ignored the symbols, unless the pattern was made unreliable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Early stopping before the symbolic cue converges makes the 'less abstract cue wins' result an artifact of stopping time; the abstraction claim is untested.","rationale":"The paper's experiments are internally consistent, and the reported tables support the narrow observation that early-stopped networks trained on Both Cues use the pattern cue. The reader's conditional verdict identifies the same load-bearing weakness: the operational definition of abstraction as convergence speed, combined with early stopping, makes the central claim depend on training duration. The Symbol-only convergence figure of 849 epochs versus the Both Cues stopping point of 45.8 epochs is the concrete piece of evidence; the Dist. Both Cues condition, where the corrupted pattern delays pattern convergence and the network begins to use symbols at 69.6 epochs, further indicates that cue selection is governed by learning dynamics rather than by abstraction level per se. This is not a mathematical inconsistency but an untested confound, so the appropriate verdict remains CONDITIONAL rather than REJECT or ACCEPT. Since the reader already reached CONDITIONAL, my stress-test does not change the verdict; it sharpens the required test.","tokens_in":9444,"tokens_out":5279,"duration_ms":62300,"concrete_test":"Re-run the Both Cues training (1 hidden layer, 100 neurons, learning rate 1e-3, batch size 32) with early stopping disabled and a 2000-epoch cap, logging Symbol test and Patterntest accuracy every 25 epochs. If Symbol test remains at chance while Patterntest stays above 90% across all later epochs, the pattern-preference claim survives. If Symbol test rises above 90% after roughly 50-100 epochs, the headline result is a stopping-time artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central observation (Section 3.1, Table 4) is that networks trained on Both Cues reach 96.17% on Patterntest but only 33.26% on Symboltest, and Section 4 interprets this as a preference for the less abstract cue. The load-bearing premise is that 'abstraction level' is independent of training time: Section 2.1.1 defines complexity as the number of iterations needed to classify the cue, so 'less abstract' is operationalized as 'converges faster.' Section 2.2.2 caps training at 1000 epochs with early stopping, and Table 6 shows Both Cues training stops at 45.8 ± 0.8 epochs, while Symbol-only training requires 849.0 ± 115.45 epochs. A network that stops at 46 epochs has not been given the opportunity to learn the symbolic cue. The experiment therefore demonstrates that the first-converging cue wins under early stopping, not that the symbolic cue is structurally inaccessible or that abstraction level determines feature selection. The Dist. Both Cues result shows the timing interpretation is live: with the pattern corrupted, training stops at 69.6 epochs and Symbol test accuracy rises to about 85%, so the symbolic cue can be learned well before the 849-epoch Symbol-only figure. The paper's own Section 4 explanation ('minimum that can be reached with fewer iterations') makes the dependence on stopping time explicit. Without a no-early-stopping run, the claim overreaches.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies how feed-forward fully-connected neural networks select among two image cues that both correlate with class labels. The low-level cue is a random pattern (uniform, centered, or cornered pixel distributions), and the high-level cue is a code of three symbols ('+' and 'x'). Using synthetic 30x30 binary images, the authors train MLPs on four datasets: Both Cues, Symbol-only, Pattern-only, and a Distorted Both Cues dataset in which 23% of pattern labels are corrupted. The central empirical result (Section 3.1, Tables 4-5) is that networks trained on Both Cues classify the pattern test set at 95-97% accuracy but the symbol test set at only ~33% (chance), so the network appears to rely on the pattern. When the pattern cue is partially corrupted, Symbol test accuracy rises to 73-93% depending on architecture (Section 3.2). The paper interprets this as evidence that neural networks are skewed toward the less abstract cue, where abstraction is defined as the number of training iterations required to learn the cue (Section 2.1.1), and proposes that the network converges to the local minimum reachable in fewer iterations (Section 4).","tokens_in":9742,"tokens_out":6433,"duration_ms":63908,"significance":"Conditional on the interpretation surviving scrutiny, this would be a useful empirical datum for understanding feature selection and inductive bias in neural networks: it provides a controlled synthetic setup in which a specific cue is systematically ignored, and it shows that corrupting the preferred cue can shift the network to the other cue. The paper's strengths are its clean synthetic construction, the systematic sweep over hidden-layer counts (1, 2, 3, 10) and hidden-neuron counts (10, 100, 500), and the transparent reporting of mean accuracies and epoch counts in Tables 7-8 and Table 6. The central observation that the pattern wins when both cues are present is robust across architectures. However, the manuscript overreaches as written: because abstraction is operationalized as convergence speed and because early stopping truncates training on the symbolic cue, the current experiments do not establish a preference for the less abstract cue in any sense independent of learning dynamics. The finding as stated needs either additional experiments without early stopping or a reframed claim about convergence-order effects.","major_comments":[{"comment":"The definition of abstraction is circular with the result. The manuscript defines complexity as the number of iterations needed for the neural network to classify the feature correctly (Section 2.1.1), and then in Section 4 explains the Both Cues result by saying that two equally deep local minima exist and the network will converge to the configuration corresponding to the minimum that can be reached with fewer iterations. Under this definition, the less abstract cue is preferred is equivalent to the cue that converges faster is preferred, which is a restatement of the observed learning dynamics rather than an independent explanation. To support the abstraction claim, the authors should either define abstraction using an independent criterion (for example compositional structure or a human-labeling study) and then show the network prefers the lower-abstraction cue, or explicitly relabel the claim as a preference for faster-to-learn cues.","section":"Section 2.1.1 and Section 4"},{"comment":"Early stopping confounds the central result. Training is capped at 1000 epochs with early stopping (Section 2.2.2), and Table 6 shows that Both Cues training converges in 45.8 ± 0.8 epochs while Symbol-only training requires 849.0 ± 115.45 epochs. Thus a network trained on Both Cues is stopped long before the symbolic cue has had the opportunity to be learned, so the low Symbol test accuracy may simply reflect an inadequate training budget rather than a structural or abstraction-based preference. The Dist. Both Cues row in Table 5 shows that symbols can be learned to 85-93% accuracy within 69.6 epochs when the pattern is corrupted, which demonstrates that the symbolic cue is not inherently inaccessible. The authors should run the Both Cues condition without early stopping (for example, a fixed 1000-epoch budget or training until Symbol test accuracy saturates) and report whether Symbol test accuracy remains at chance; otherwise the claim in the Abstract and Section 3.1 overreaches.","section":"Section 2.2.2 and Table 6"},{"comment":"The secondary claim about shifting learning to the high-level cue rests on a single arbitrary corruption level. The 23% corruption fraction is introduced without justification and no sweep is performed, so the statement that symbol-like entities are only learned if the importance of low-level cues is reduced is supported by one operating point. A corruption-fraction sweep (for example, 0%, 10%, 23%, 40%, and 50%) with corresponding Symbol test accuracies is needed to establish the threshold or monotonic relationship implied by the claim.","section":"Section 2.1.2 and Section 3.2"}],"minor_comments":[{"comment":"There are several typographical issues, including P atterntest and P atterntrain spacing in Table 3 and the sentence The results suggests in Section 3.1; these should be corrected in a revision.","section":"Throughout"},{"comment":"The training description does not state the early-stopping patience threshold (how many epochs without improvement trigger stopping), which matters for interpreting the epoch counts in Table 6.","section":"Section 2.2.2"},{"comment":"The claim that batch normalization did not have a significant impact is reported without any supporting results; either provide the data or remove the statement.","section":"Section 3.3"},{"comment":"The sentence asserting that two equally deep local minima exist is speculative and is not established by the experiments; if retained, it should be explicitly labeled as a hypothesis.","section":"Section 4"},{"comment":"The paper does not state random seeds or provide code or dataset generation scripts; since the datasets are synthetic, reporting seeds and releasing the generator would improve reproducibility.","section":"Section 2.1"},{"comment":"The 10-hidden-layer results show high variance (for example, Symbol train with 10 neurons gives 85.13 ± 12.80%), which complicates the claim that the effect holds in networks with many layers; the authors should discuss this variance explicitly.","section":"Table 8"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the manuscript's title and abstract promise a result about abstraction, but the operational definition of abstraction and the early-stopping protocol make the observed effect more plausibly a convergence-time artifact. The empirical setup is transparent and the asymmetry is real, so I view this as fixable with additional runs and a more cautious framing, rather than as a case for rejection. The paper would also benefit from a methods check on the early-stopping protocol and a corruption-level sweep."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — here's my read on Wolnitza & Dellen. The headline result — fully-connected nets ignore a symbolic code when a pixel-statistics cue is equally predictive — is real in this setup, but the paper's own operationalization of 'abstraction' makes the conclusion partly a restatement of 'the cue that converges first wins under early stopping.'\n\nThe good: the two-cue synthetic dataset is clean, the scan across widths and depths is thorough, and the corruption experiment (23% label noise on the pattern flips the network toward symbols) is a nice demonstration that the network's preference can be shifted. The accuracy tables are internally consistent and the main observation replicates across architectures. The paper also honestly reports the epoch counts (Table 6): symbols alone take ~849 epochs to converge, both-cues training stops at ~46 epochs. That is the crux.\n\nThe problem: training on Both Cues uses early stopping, and the network stops at 46 epochs — long before the symbolic cue would have been learned. The paper concludes that the 'less abstract' cue is preferred, but 'abstraction' is defined as learning speed, so the result is close to tautological: the faster-to-learn cue wins because the training is cut short. The Dist. Both Cues result actually shows the timing interpretation is live — with the pattern corrupted, training runs to ~70 epochs and the network reaches 85% on symbols. So symbols are not structurally inaccessible; they just need more time. A no-early-stopping run on Both Cues, or a long-training run, would be needed to support the abstraction claim. The paper's own Section 4 explanation ('minimum that can be reached with fewer iterations') makes the dependence on stopping time explicit.\n\nOther notes: the 23% corruption level is never swept, so we don't know how sensitive the shift is. And the paper does not engage with the existing shortcut-learning / texture-bias literature (e.g., Geirhos et al. 2018; Jo & Bengio 2017), which has reported similar phenomena — the novelty lies in the controlled demonstration, not the qualitative finding.\n\nWho this is for: researchers studying shortcut learning, simplicity bias, and interpretability, and anyone designing curricula to steer networks toward symbolic features. The paper is worth a serious referee — the setup is clean and the question is real — but the central claim needs reframing: what is actually shown is that under early stopping, the first-converging cue dominates, and that corrupting the easy cue forces the network to learn the harder one. That is a useful empirical result, but it is not the 'abstraction' story the abstract sells.\n\nRecommendation: send it to review, but ask the authors to run without early stopping and to either decouple 'abstraction' from convergence time or justify the operationalization. I would not cite the abstraction claim as it stands.","headline":"The central result is real but the abstraction conclusion is untested: early stopping cuts training before the symbolic cue converges, so the paper shows the first-converging cue wins, not that less abstract cues are preferred.","tokens_in":10216,"tokens_out":3246,"would_cite":false,"duration_ms":32796,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"When a neural network can classify from either a low-level statistical pattern or a higher-level symbolic code, it learns the pattern and ignores the symbols; only corrupting the pattern forces the symbols to be learned.","keywords":["feature selection","neural networks","abstraction level","synthetic images","symbolic cues","statistical patterns","learning dynamics"],"falsifier":"Train the same networks on the both-cues dataset for many more epochs without early stopping; if symbol test accuracy rises well above chance while pattern accuracy remains high, the cue preference is an effect of training duration, not of abstraction level.","tokens_in":9216,"feed_emoji":"🧠","tokens_out":6779,"duration_ms":58225,"temperature":0.7,"pith_summary":"This paper asks which of two equally predictive image cues a neural network actually uses for classification. Training on 30 x 30 binary images that contain both a class-correlated random pattern and a class-correlated combination of symbols, the authors find that networks converge to using the pattern and leave symbol-test accuracy near chance (about 33 percent) while pattern-test accuracy exceeds 95 percent. The symbol code is learned only when 23 percent of pattern labels are corrupted, lowering the pattern's reliability. The authors interpret the preference as a consequence of learning difficulty: the pattern needs about 95 training epochs, the symbols about 850, so gradient descent reaches the easier local minimum first.","feed_headline":"When images carry two cues, neural nets pick the simpler one","feed_subtitle":"Networks trained on both cues master the simple pattern and ignore the symbolic code unless that pattern is corrupted.","key_machinery":"The central object is a synthetic dataset of 30 x 30 binary images in which class membership is correlated with two cues: a pixel pattern drawn from one of three distributions and a combination of three '+' and 'x' symbols. Abstraction level is defined by the paper as learning complexity, measured by the number of epochs until the network classifies the cue correctly. The load-bearing mechanism is the convergence race between two local minima in the loss landscape: when both cues are equally predictive, the network finds the pattern minimum after about 95 epochs, whereas the symbol minimum would require roughly 850 epochs to reach.","core_discovery":"The paper's central claim is that abstraction level, operationalized as the number of training iterations needed for the network to classify a cue correctly, determines which cue is selected. When both cues are present and equally informative, networks with one to ten hidden layers converge to solutions that classify the pattern (96.17 percent test accuracy for a representative one-hidden-layer, 100-neuron network) and fail on the symbols (33.26 percent, near chance). Introducing false pattern labels in 23 percent of the training set raises symbol test accuracy to roughly 73 to 86 percent, indicating the network now exploits both cues. The authors conclude that gradient descent settles into the local minimum that can be reached in fewer iterations.","pith_inferences":["Editorial inference: the result suggests a general simplicity bias in which any cue that converges faster dominates; testing with a symbol set that converges faster than the pattern should invert the preference.","Editorial inference: the early-stopping protocol likely drives the effect, since symbols alone need about 850 epochs; removing the cap might let networks learn both cues on the both-cues dataset.","Editorial inference: the setup parallels shortcut learning in natural images, where models latch onto spurious statistical regularities; corrupting the shortcut's labels is a known strategy to force robust features."],"forward_implications":["Networks trained on datasets with redundant cues may silently specialize to the statistically simplest cue, even when a more meaningful cue is available.","Evaluating a classifier only on the training distribution can miss that it ignores an entire class of informative cues.","Corrupting or weakening the easy cue is an effective lever to force the network to use a desired harder cue.","Adding more layers or neurons does not by itself overcome the bias toward the easier cue."],"supporting_citations":[],"fun_headline_variants":["Neural nets choose easy patterns over symbolic codes","Networks learn simple cues, ignore abstract ones by default","Feature selection in NNs favors less abstract cues","When both cues exist, networks pick the pattern first","Corrupting simple cues forces nets to learn symbols"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper defines a cue's abstraction level as how many training iterations it takes the network to learn it, so the claim that the less abstract cue wins could simply mean that the easier cue is learned within the early-stopping horizon.","fun_headline_variants_meta":{"raw":{"variants":["Neural nets choose easy patterns over symbolic codes","Networks learn simple cues, ignore abstract ones by default","Feature selection in NNs favors less abstract cues","When both cues exist, networks pick the pattern first","Corrupting simple cues forces nets to learn symbols"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1319,"prompt_tokens":875,"completion_tokens":444,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":491,"tokens_out":444,"duration_ms":5583,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:26:44.308571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same networks on the both-cues dataset for many more epochs without early stopping; if symbol test accuracy rises well above chance while pattern accuracy remains high, the cue preference is an effect of training duration, not of abstraction level.","supporting_citations":[],"review_version":1}