Learning Visual Abstract Reasoning through Dual-Stream Networks

Bailu Si; Chang Xu; Kai Zhao

REVIEW 2 major objections 8 minor 63 references

Learning Visual Abstract Reasoning through Dual-Stream Networks

T0 review · 2 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DRNet claims 97.78% average accuracy and large out-of-distribution gains across four Raven's Progressive Matrices benchmarks by combining a CNN branch and a ViT branch before rule extraction.

desk verdict DRNet is a solid dual-stream CNN+ViT for RPM with strong reported numbers, but the unanalyzed flip augmentation makes the headline results conditional on a code check. read the letter →

arxiv 2411.19451 v1 pith:ED54H5P2 submitted 2024-11-29 cs.CV cs.AIcs.LG

Kai Zhao , Chang Xu , Bailu Si This is my paper · ORCID

classification cs.CVcs.AIcs.LG

keywords Raven'sProgressiveMatricesabstractvisualreasoningdual-streamnetworkvisiontransformerout-of-distributiongeneralizationruleextractionrelationalPGMbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DRNet, a dual-stream network for Raven's Progressive Matrices (RPM), and tries to establish that separating local and spatial visual processing before reasoning improves abstract visual reasoning. The authors report a state-of-the-art average accuracy of 97.78% on the PGM-Neutral, RAVEN, I-RAVEN, and RAVEN-FAIR benchmarks, surpassing the previous best average of 96.7%. The architecture pairs a CNN branch that extracts object-level features with a ViT branch that retains spatial layout, fuses both through a learned linear operator, and then uses a rule extractor to score the eight candidate answers. They also report out-of-distribution gains, most strikingly 93.74% on PGM held-out attribute pairs versus 63.4% for the previous best model. If the results hold, they support a practical design principle: two complementary visual pathways, one for objects and one for locations, can be fused to produce discrete, transferable rules.

What carries the argument

The central mechanism is the dual encoder plus reasoning module. Each of the 16 panels (8 context, 8 candidates) is processed in parallel by a two-ResBlock CNN branch and a 12-layer ViT branch; the CNN output is reshaped to match the ViT's feature vector, and the integration operator LIN, a learnable linear layer over their concatenation, fuses them. For each candidate, the rule extractor prepends the eight fused context features, runs 1D convolutional ResBlocks over the nine-vector sequence, and pools it into a 1024-dimensional embedding; an MLP scores the eight embeddings and selects the highest. The two-stream separation is what carries the argument: ablations show either stream alone is far weaker, while two CNN streams or two ViT streams lose the OOD advantage.

What would settle it

Take one training sample, apply the paper's vertical/horizontal flip augmentation, and compare the correct candidate index in the flipped sample with the index assigned to the blank; if the index no longer points to the correct panel, the augmentation is mislabeling training data and the accuracy gains need re-evaluation.

Watch

Extended reading notes

Core claim

The central claim is that DRNet reaches the best reported average accuracy, 97.78%, across PGM-N, RAVEN, I-RAVEN, and RAVEN-FAIR, and that the advantage grows under distribution shift. The two streams are not redundant: the CNN stream learns local object features, the ViT stream learns spatial position, and the learned integration operator LIN combines them before a rule extractor builds an embedding for each candidate-context group. The authors show that removing either stream drops the held-out attribute-pair score from 93.74% to 72.26% or 62.87%, and that the rule embeddings of different rule types form near-orthogonal clusters. This is presented as evidence that the network discovers discrete abstract rules rather than memorizing panel layouts.

Load-bearing premise

The load-bearing premise is that flipping the entire 3x3 grid together with the candidate panels during data augmentation always keeps the correct candidate matched to the blank; if the flip moves the blank without re-labeling candidates, the model is trained on wrong answers.

Editorial extensions

If this is right

Reported accuracy rises on all four benchmarks individually, with gains of 1.09 points on RAVEN, 1.12 on I-RAVEN, and 0.48 on RAVEN-FAIR over PredRNet.
On PGM held-out attribute pairs, the reported score jumps from 63.4% (PredRNet) to 93.74% (DRNet), a 30.34-point increase that far exceeds the in-distribution gap.
Ablation results indicate the dual-stream design is necessary: with only the CNN branch and data augmentation, I-RAVEN accuracy is 95.50% in-distribution but only 62.87% on the OOD regime; with only ViT, the OOD score is 72.26%.
A parameter-reduced version, DRNet-P, keeps 96.06% on I-RAVEN and 91.23% on the OOD regime while cutting parameters by 86.2%, so the dual-stream benefit is not an artifact of model size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A causal test of the claimed what/where division would replace the attention-map visualizations with input lesions (e.g., scrambling object locations for the ViT stream and object textures for the CNN stream); the paper does not provide such evidence.
The label-correctness of the flip augmentation is untested; if the flipped grid no longer aligns the correct candidate with the blank, part of the reported gain could come from training on corrupted labels rather than from rule extraction.
Because the rule-extraction module treats any ordered set of nine feature vectors as a sequence, it could transfer to other analogy benchmarks, such as letter-string completion or visual odd-one-out, without architectural change.
The measured near-orthogonality of rule embeddings suggests an explicit orthogonality regularizer during training might further improve OOD generalization, a conjecture the paper leaves open.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is a clean, well-engineered application of a dual-stream CNN+ViT architecture to Raven's Progressive Matrices. That dual-stream setup for RPM is new, and the ablations are genuinely informative: they show both branches earn their keep, especially for out-of-distribution generalization, and that a much smaller patch-based variant loses only about 2 points. The rule representation visualization is a nice sanity check, and the comparison table is broad and careful about source attribution.

The soft spots, in order of severity. First, the flip augmentation. The paper says 'vertical/horizontal flip data augmentation with a probability of 0.3 for RPM training samples' and never analyzes whether flipping the 16 panels preserves the correct candidate. The reader's stress-test concern lands: the 3x3 matrix has a blank position, so a naive flip changes the geometry; the context panels need to be re-indexed to the new grid, not just mirrored in place. If the implementation flips panel contents but keeps the panel order, a meaningful fraction of training labels are wrong. That would not necessarily tank the reported accuracy, but it would mean every headline number, including the 93.74% on PGM HO AP, is built on partially mislabeled training data. This is testable from the code, but the code link has no pinned commit, so the manuscript alone cannot resolve it. This is the most load-bearing ambiguity in the paper.

Second, PGM results are from a single run. The authors say this aligns with prior work, and it does, but given how large the OOD gap is (93.74 vs 63.4 for PredRNet), run-to-run variability would be reassuring. This is a minor point. Third, the 30-point OOD jump is unexplained. The ablation points to the ViT branch as crucial, but there is no mechanistic account for why adding a ViT to a CNN should produce a gain of that size. The near-orthogonal rule embeddings are suggestive, not causal. That is a reason to be skeptical, not a reason to reject.

My overall take: the central architecture claim holds up. The augmentation concern is a real caveat, not a refutation. If a code check confirms that flips are applied consistently to all panels with proper re-indexing, the results are credible. I would send this to peer review and ask the authors to pin a code snapshot and clarify the augmentation implementation. The paper is aimed at anyone working on abstract visual reasoning or benchmarking RPM, and it is a solid empirical entry.

Referee Report

2 major / 8 minor

Summary. The paper proposes DRNet, a dual-stream network for abstract visual reasoning on Raven's Progressive Matrices (RPM). The model uses a CNN branch and a ViT branch to extract complementary image features, then a reasoning module fuses these features, extracts discrete abstract rule representations with 1D convolutional layers, and scores the eight candidate answers with an MLP. The authors report state-of-the-art average accuracy of 97.78% across PGM-N, RAVEN, I-RAVEN, and RAVEN-FAIR, and notable out-of-distribution results on PGM held-out regimes, especially 93.74% on held-out attribute pairs. The paper includes ablations on fusion operators, branch configurations, hyperparameters, and visualization analyses of the learned rule representations.

Significance. If the reported numbers are correct, this is a meaningful empirical advance for RPM benchmarks, with the dual-stream design offering a simple and effective alternative to single-stream approaches. The paper provides extensive ablations that isolate the contribution of each branch and of data augmentation, and it makes code available. The particularly large improvement on PGM held-out attribute pairs (93.74% vs. 63.4% for PredRNet) is striking, and the t-SNE and cosine-similarity analyses give some interpretability. However, the central empirical claims rest on a training-detail ambiguity (data augmentation) and a reproducibility-relevant inconsistency in the architecture description, so confidence in the numbers is conditional.

major comments (2)

[Main Results and Table 4] The paper applies vertical/horizontal flip data augmentation with probability 0.3 to RPM training samples, but does not demonstrate that this augmentation preserves the correct candidate index. Because the input I is a list of 16 separate 80x80 panels, a flip either applied per panel or applied to the full tensor without reordering the context panels can alter the relational structure of the 3x3 matrix (for example, a directional position rule can become its mirror image), making the original label incorrect. The ablation in Table 4 shows that data augmentation contributes substantially to the reported performance (I-RAVEN improves from 91.68 to 97.62 and PGM HO AP from 90.46 to 93.74 when augmentation is added), so the SOTA claims are directly conditioned on this detail. The authors should either prove that the correct answer index is invariant under their augmentation for all RPM instances, describe the exact implementation and verify label correctness on a subset of the training set, or re-run the main experiments without this augmentation and report both sets of results.
[Dual Encoder Module, Eq. (1)-(3)] The CNN branch description is internally inconsistent. The text states that the stream has two ResBlocks, each with two convolutional layers of stride 2, which would reduce an 80x80 input to 5x5 after four down-sampling convolutions. However, the paper then states that x_cnn out has shape R(B×16,1,20,20), which would correspond to only one ResBlock. The filter list [64, 64, 64, 16] and the subsequent flattening to 400 dimensions suggest the intended output is (B×16,16,5,5). Please clarify whether the second ResBlock uses stride 1 or whether the stated 20x20 shape is a typo. This is essential for reproducing the architecture from the paper.

minor comments (8)

[Table 1 and text] The average accuracy is reported as 97.78% in the text but 97.79% in Table 1; please make these consistent.
[State-of-the-art Comparisons] The text refers to 'MRet' but the intended model is MRNet; please correct the typo.
[Table 2 caption] The caption lists 'HP AP' but the abbreviation for Held-Out Attribute Pairs is 'HO AP'; please correct.
[Out-of-Distribution Generalization in PGM] The phrase 'reaching up to 11.23%' is ambiguous: the 11.23-percentage-point gap is the difference in the average over all eight PGM regimes, whereas the OOD-only average improvement is different. Please state the comparison precisely.
[Implementation Details] Please specify whether 'vertical/horizontal flip' means that each training sample is randomly flipped vertically or horizontally, or both, and whether the flip probability is applied per sample or per axis.
[Rule Representations] The text says 'we visualized 200k test samples' for t-SNE; t-SNE on 200,000 points is unusual and likely refers to a sampled subset. Please clarify the subsampling procedure.
[Integration Module] In the definition of the LIN operator, the dimensions of the linear layer (A and b) are not specified; please provide the input and output dimensions for clarity.
[Reproducibility] The GitHub link is provided, but no commit hash or version is pinned; please add a specific commit identifier to support reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical held-out benchmark evaluations, not derivations that reduce to their inputs.

full rationale

This is an empirical supervised-learning paper. DRNet is trained on the standard training splits of PGM, RAVEN, I-RAVEN, and RAVEN-FAIR, and all reported accuracies (Tables 1 and 2) are measured on held-out test splits using the best-validation checkpoint. No parameter is fitted to a target quantity and then reported as a prediction of that same quantity; no rule representation is defined in terms of the accuracy it is said to explain. The only augmentation noted, 'vertical/horizontal flip data augmentation with a probability of 0.3 for RPM training samples,' is a training-data choice, not a derivation step, and any concern about label preservation is a correctness risk rather than a circularity. The comparison numbers for prior models are taken from published papers, and the authors' own ablation studies independently vary architectures and report separate results on I-RAVEN and PGM HO AP. The neuroscience framing ('two-stream hypothesis') is motivational and does not do load-bearing inferential work. Self-citations are not present in a load-bearing chain: the cited prior work by overlapping authors (e.g., PredRNet, which shares some acknowledged collaborators) is used as a baseline and a source of unreported results, not as justification for DRNet's central claim. Accordingly, no circular step can be quoted, and the score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The paper's central claim rests on standard supervised learning assumptions and on an untested data augmentation choice. The model has several hand-selected architectural hyperparameters, and its internal 'rule representations' are not validated as genuine rule encodings beyond a qualitative visualization.

free parameters (6)

ViT depth = 12
Selected from {4,8,12,16} based on I-RAVEN and PGM HO AP validation accuracy (Table 3).
CNN kernel size = 7
Selected from {3,5,7} based on same validation sets (Table 3).
Feature fusion operator = LIN (linear layer on concatenation)
Selected from {SUM, MEA, AUT, LIN} based on RAVEN accuracy (Figure 3).
Data augmentation probability = 0.3
Chosen without analysis; flip augmentation may not be label-preserving for RPM.
Learning rate = 3e-4
Standard Adam hyperparameter, not tuned per dataset.
Weight decay = 1e-6
Standard setting.

assumptions (4)

domain assumption The RPM benchmarks (PGM, RAVEN, I-RAVEN, RAVEN-FAIR) have correct labels and disjoint train/test splits.
The paper reports test accuracy using the official splits; no independent verification of benchmark integrity is provided.
ad hoc to paper The vertical/horizontal flip augmentation preserves the identity of the correct answer for each RPM sample.
The paper applies flip augmentation with probability 0.3 but does not show that flipping the grid and candidate set keeps the label index unchanged; this is load-bearing for the validity of training data.
domain assumption A CNN branch and a ViT branch capture complementary 'local' and 'spatial' information, and their fusion improves reasoning.
This is the design hypothesis of DRNet, supported only by ablations on two datasets.
domain assumption The 1D rule extractor can recover abstract relational rules from concatenated features of 8 context images and a candidate.
No theoretical guarantee is given; the claim is supported only by empirical accuracy and t-SNE visualization.

invented entities (1)

Discrete abstract rule representations
purpose: 1024-dimensional embeddings produced by the rule extractor that are claimed to encode the abstract rules of each RPM problem
The representations are internal to the trained network and are only visualized via t-SNE; the paper does not use them to make out-of-sample predictions that could falsify their semantic content.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Visual Abstract Reasoning through Dual-Stream Networks." pith.science (2026). https://pith.science/paper/ED54H5P2

@misc{pith2026241119451,
  author       = {Pith},
  title        = {Pith review of: Learning Visual Abstract Reasoning through Dual-Stream Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ED54H5P2}},
  note         = {Machine review of arXiv:2411.19451}
}

read the original abstract

Visual abstract reasoning tasks present challenges for deep neural networks, exposing limitations in their capabilities. In this work, we present a neural network model that addresses the challenges posed by Raven's Progressive Matrices (RPM). Inspired by the two-stream hypothesis of visual processing, we introduce the Dual-stream Reasoning Network (DRNet), which utilizes two parallel branches to capture image features. On top of the two streams, a reasoning module first learns to merge the high-level features of the same image. Then, it employs a rule extractor to handle combinations involving the eight context images and each candidate image, extracting discrete abstract rules and utilizing an multilayer perceptron (MLP) to make predictions. Empirical results demonstrate that the proposed DRNet achieves state-of-the-art average performance across multiple RPM benchmarks. Furthermore, DRNet demonstrates robust generalization capabilities, even extending to various out-of-distribution scenarios. The dual streams within DRNet serve distinct functions by addressing local or spatial information. They are then integrated into the reasoning module, leveraging abstract rules to facilitate the execution of visual reasoning tasks. These findings indicate that the dual-stream architecture could play a crucial role in visual abstract reasoning.

Figures

Figures reproduced from arXiv: 2411.19451 by the authors.

**Figure 2.** An overview of our DRNet. DRNet consists of a dual encoder module and a reasoning module, where (B [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 2.** Rule Extractor. The rule extractor consists of two ResBlocks. Each residual branch has two 1D convolutional layers with a kernel size of 7. Each convolutional layer learns to expand the receptive fields of the neurons to extract higherlevel relations with a stride of 1. The shortcut connection applies a 1D convolutional layer to the two ResBlocks with a kernel size and stride of 1. In total, our rule extractor ca… view at source ↗

Figures from the paper (4 more)

**Figure 3.** Figure 3: The performance analysis of various operators in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]

**Figure 4.** Figure 4: T-SNE visualization of abstract rules in PGM Neu [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Illustrative examples from ViT and CNN streams. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Cosine similarity for dual encoder representations. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 47 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Bakhtiari, S.; Mineault, P.; Lillicrap, T.; Pack, C.; and Richards, B. 2021. The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning. Advances in Neural Information Processing Systems, 34: 25164--25178

work page 2021
[4]

E.; Muller, T

Behrens, T. E.; Muller, T. H.; Whittington, J. C.; Mark, S.; Baram, A. B.; Stachenfeld, K. L.; and Kurth-Nelson, Z. 2018. What is a cognitive map? Organizing knowledge for flexible behavior. Neuron, 100(2): 490--509

work page 2018
[5]

Benny, Y.; Pekar, N.; and Wolf, L. 2021. Scale-localized abstract reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12557--12565

work page 2021
[6]

R.; and Shalev-Schwartz, S

Brutzkus, A.; Globerson, A.; Malach, E.; Netser, A. R.; and Shalev-Schwartz, S. 2022. Efficient Learning of CNNs using Patch Based Features. In International Conference on Machine Learning, 2336--2356. PMLR

work page 2022
[7]

F.; and Olshausen, B

Cadieu, C. F.; and Olshausen, B. A. 2012. Learning intermediate-level representations of form and motion from natural movies. Neural computation, 24(4): 827--866

work page 2012
[8]

Carreira, J.; and Zisserman, A. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6299--6308

2017

Show all 63 references

[9]

Chen, Y.; Zuo, R.; Wei, F.; Wu, Y.; Liu, S.; and Mak, B. 2022. Two-stream network for sign language recognition and translation. Advances in Neural Information Processing Systems, 35: 17043--17056

2022
[10]

Court, J. 1982. Manual for Raven’s Progressive Matrices and Vocabulary Scales

1982
[11]

Cui, R.; Liu, H.; and Zhang, C. 2019. A deep neural framework for continuous sign language recognition by iterative training. IEEE Transactions on Multimedia, 21(7): 1880--1891

2019
[12]

Da, K. 2014. A method for stochastic optimization. arXiv preprint arXiv:1412.6980

2014 arXiv
[13]

Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In 9th International Con...

2021
[14]

Feichtenhofer, C.; Fan, H.; Malik, J.; and He, K. 2019. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, 6202--6211

2019
[15]

Feichtenhofer, C.; Pinz, A.; and Zisserman, A. 2016. Convolutional two-stream network fusion for video action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1933--1941

2016
[16]

Girshick, R.; Donahue, J.; Darrell, T.; and Malik, J. 2015. Region-based convolutional networks for accurate object detection and segmentation. IEEE transactions on pattern analysis and machine intelligence, 38(1): 142--158

2015
[17]

A.; and Milner, A

Goodale, M. A.; and Milner, A. D. 1992. Separate visual pathways for perception and action. Trends in neurosciences, 15(1): 20--25

1992
[18]

Grezes, J.; and Decety, J. 2002. Does visual perception of object afford action? Evidence from a neuroimaging study. Neuropsychologia, 40(2): 212--222

2002
[19]

u ddecke, T.; W \

Hahne, L.; L \"u ddecke, T.; W \"o rg \"o tter, F.; and Kappel, D. 2019. Attention on abstract visual reasoning. arXiv preprint arXiv:1911.05990

2019 arXiv
[20]

He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778

2016
[21]

Hersche, M.; Zeqiri, M.; Benini, L.; Sebastian, A.; and Rahimi, A. 2023. A neuro-vector-symbolic architecture for solving Raven’s progressive matrices. Nature Machine Intelligence, 1--13

2023
[22]

Hoshen, D.; and Werman, M. 2017. Iq of neural networks. arXiv preprint arXiv:1710.01692

2017 arXiv
[23]

Hu, S.; Ma, Y.; Liu, X.; Wei, Y.; and Bai, S. 2021. Stratified Rule-Aware Network for Abstract Visual Reasoning. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The...

2021
[24]

what” and “where

Huang, C.-C.; Rolls, E. T.; Hsu, C.-C. H.; Feng, J.; and Lin, C.-P. 2021. Extensive cortical connectivity of the human hippocampal memory system: beyond the “what” and “where” dual stream model. Cerebral Cortex, 31(10): 4652--4669

2021
[25]

K.; Patra, B.; Liu, Q.; Aggarwal, K.; Chi, Z.; Bjorck, J.; Chaudhary, V.; Som, S.; Song, X.; and Wei, F

Huang, S.; Dong, L.; Wang, W.; Hao, Y.; Singhal, S.; Ma, S.; Lv, T.; Cui, L.; Mohammed, O. K.; Patra, B.; Liu, Q.; Aggarwal, K.; Chi, Z.; Bjorck, J.; Chaudhary, V.; Som, S.; Song, X.; and Wei, F. 2023. Language Is Not All You Need: Aligning Perception with Language Models. CoR...

2023 arXiv
[26]

M.; Buschkuehl, M.; Jonides, J.; and Perrig, W

Jaeggi, S. M.; Buschkuehl, M.; Jonides, J.; and Perrig, W. J. 2008. Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences, 105(19): 6829--6833

2008
[27]

Jahrens, M.; and Martinetz, T. 2020. Solving raven’s progressive matrices with multi-layer relation networks. In 2020 International Joint Conference on Neural Networks (IJCNN), 1--6. IEEE

2020
[28]

LeCun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278--2324

1998
[29]

Ma, X.; Nie, W.; Yu, Z.; Jiang, H.; Xiao, C.; Zhu, Y.; Zhu, S.; and Anandkumar, A. 2022. RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . ...

2022
[30]

A.; Burgess, N.; and O’Keefe, J

Maguire, E. A.; Burgess, N.; and O’Keefe, J. 1999. Human spatial navigation: cognitive maps, sexual dimorphism, and neural substrates. Current opinion in neurobiology, 9(2): 171--177

1999
[31]

Ma ki \'n ski, M.; and Ma \'n dziuk, J. 2022 a . Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices. arXiv preprint arXiv:2201.12382

2022 arXiv
[32]

Ma ki \'n ski, M.; and Ma \'n dziuk, J. 2022 b . Multi-label contrastive learning for abstract visual reasoning. IEEE Transactions on Neural Networks and Learning Systems

2022
[33]

Mao, M.; Zhang, R.; Zheng, H.; Ma, T.; Peng, Y.; Ding, E.; Zhang, B.; Han, S.; et al. 2021. Dual-stream network for visual recognition. Advances in Neural Information Processing Systems, 34: 25346--25358

2021
[34]

S.; Webb, T

Mondal, S. S.; Webb, T. W.; and Cohen, J. 2023. Learning to reason over visual objects. In International Conference on Learning Representations

2023
[35]

Niebur, E.; and Koch, C. 1995. Control of selective visual attention: Modeling the" where" pathway. Advances in neural information processing systems, 8

1995
[36]

L.; and Onofrj, M

Perfetti, B.; Saggino, A.; Ferretti, A.; Caulo, M.; Romani, G. L.; and Onofrj, M. 2009. Differential patterns of cortical activation as a function of fluid reasoning complexity. Human brain mapping, 30(2): 497--510

2009
[37]

A.; Desmond, J

Prabhakaran, V.; Smith, J. A.; Desmond, J. E.; Glover, G. H.; and Gabrieli, J. D. 1997. Neural substrates of fluid reasoning: an fMRI study of neocortical activation during performance of the Raven's Progressive Matrices Test. Cognitive psychology, 33(1): 43--63

1997
[38]

Raghu, M.; Unterthiner, T.; Kornblith, S.; Zhang, C.; and Dosovitskiy, A. 2021. Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems, 34: 12116--12128

2021
[39]

W.; Joshi, S.; Gehler, P.; Bengio, Y.; Locatello, F.; and Sch \"o lkopf, B

Rahaman, N.; Gondal, M. W.; Joshi, S.; Gehler, P.; Bengio, Y.; Locatello, F.; and Sch \"o lkopf, B. 2021. Dynamic inference with neural interpreters. Advances in Neural Information Processing Systems, 34: 10985--10998

2021
[40]

Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part II...

2015
[41]

Sahu, P.; Basioti, K.; and Pavlovic, V. 2022. SAViR-T: Spatially Attentive Visual Reasoning with Transformers. CoRR, abs/2206.09265

2022 arXiv
[42]

Santoro, A.; Hill, F.; Barrett, D.; Morcos, A.; and Lillicrap, T. 2018. Measuring abstract reasoning in neural networks. In International Conference on Machine Learning, 4477--4486

2018
[43]

G.; Malinowski, M.; Pascanu, R.; Battaglia, P.; and Lillicrap, T

Santoro, A.; Raposo, D.; Barrett, D. G.; Malinowski, M.; Pascanu, R.; Battaglia, P.; and Lillicrap, T. 2017. A simple neural network module for relational reasoning. Advances in neural information processing systems, 30

2017
[44]

Simonyan, K.; and Zisserman, A. 2014. Two-stream convolutional networks for action recognition in videos. Advances in neural information processing systems, 27

2014
[45]

E.; Kyllonen, P

Snow, R. E.; Kyllonen, P. C.; Marshalek, B.; et al. 1984. The topography of ability and learning correlations. Advances in the psychology of human intelligence, 2(S 47): 103

1984
[46]

E.; and Lohman, D

Snow, R. E.; and Lohman, D. F. 1984. Toward a theory of cognitive aptitude for learning from instruction. Journal of educational psychology, 76(3): 347

1984
[47]

Spratley, S.; Ehinger, K.; and Miller, T. 2020. A closer look at generalisation in raven. In European Conference on Computer Vision, 601--616. Springer

2020
[48]

van der Maaten, L.; and Hinton, G. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86): 2579--2605

2008
[49]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

2017
[50]

Wang, D.; Jamnik, M.; and Li \` o , P. 2020. Abstract Diagrammatic Reasoning with Multiplex Graph Networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net

2020
[51]

C.; McCaffary, D.; Bakermans, J

Whittington, J. C.; McCaffary, D.; Bakermans, J. J.; and Behrens, T. E. 2022. How to build a cognitive map. Nature Neuroscience, 25(10): 1257--1272

2022
[52]

C.; Muller, T

Whittington, J. C.; Muller, T. H.; Mark, S.; Chen, G.; Barry, C.; Burgess, N.; and Behrens, T. E. 2020. The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell, 183(5): 1249--1263

2020
[53]

Wu, Y.; Dong, H.; Grosse, R.; and Ba, J. 2020. The scattering compositional learner: Discovering objects, attributes, relationships in analogical reasoning. arXiv preprint arXiv:2007.04212

2020 arXiv
[54]

Yang, L.; You, H.; Zhen, Z.; Wang, D.; Wan, X.; Xie, X.; and Zhang, R.-Y. 2023. Neural prediction errors enable analogical visual reasoning in human standard intelligence tests. In International Conference on Machine Learning, 39572--39583. PMLR

2023
[55]

Zhang, C.; Gao, F.; Jia, B.; Zhu, Y.; and Zhu, S.-C. 2019 a . Raven: A dataset for relational and analogical visual reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5317--5327

2019
[56]

Zhang, C.; Jia, B.; Gao, F.; Zhu, Y.; Lu, H.; and Zhu, S.-C. 2019 b . Learning perceptual inference by contrasting. Advances in Neural Information Processing Systems, 32

2019
[57]

Zhang, C.; Jia, B.; Zhu, S.-C.; and Zhu, Y. 2021. Abstract spatial-temporal reasoning via probabilistic abduction and execution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9736--9746

2021
[58]

N.; Zhu, S.-C.; and Zhu, Y

Zhang, C.; Xie, S.; Jia, B.; Wu, Y. N.; Zhu, S.-C.; and Zhu, Y. 2022 a . Learning algebraic representation for systematic generalization in abstract reasoning. In European Conference on Computer Vision, 692--709. Springer

2022
[59]

Zhang, W.; Tang, L.; Mo, S.; Liu, X.; and Song, S. 2022 b . Learning Robust Rule Representations for Abstract Reasoning via Internal Inferences. In Advances in Neural Information Processing Systems

2022
[60]

Zheng, K.; Zha, Z.-J.; and Wei, W. 2019. Abstract reasoning with distracting features. Advances in Neural Information Processing Systems, 32

2019
[61]

Zhou, H.; Zhou, W.; Zhou, Y.; and Li, H. 2021. Spatial-temporal multi-cue network for sign language recognition and translation. IEEE Transactions on Multimedia, 24: 768--779

2021
[62]

Zhuo, T.; and Kankanhalli, M. 2020. Effective abstract reasoning with dual-contrast network. In International Conference on Learning Representations

2020
[63]

L.; Sedaghat, N.; and Brox, T

Zolfaghari, M.; Oliveira, G. L.; Sedaghat, N.; and Brox, T. 2017. Chained multi-stream networks exploiting pose, motion, and appearance for action classification and detection. In Proceedings of the IEEE International Conference on Computer Vision, 2904--2913

2017

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Bakhtiari, S.; Mineault, P.; Lillicrap, T.; Pack, C.; and Richards, B. 2021. The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning. Advances in Neural Information Processing Systems, 34: 25164--25178

work page 2021

[4] [4]

E.; Muller, T

Behrens, T. E.; Muller, T. H.; Whittington, J. C.; Mark, S.; Baram, A. B.; Stachenfeld, K. L.; and Kurth-Nelson, Z. 2018. What is a cognitive map? Organizing knowledge for flexible behavior. Neuron, 100(2): 490--509

work page 2018

[5] [5]

Benny, Y.; Pekar, N.; and Wolf, L. 2021. Scale-localized abstract reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12557--12565

work page 2021

[6] [6]

R.; and Shalev-Schwartz, S

Brutzkus, A.; Globerson, A.; Malach, E.; Netser, A. R.; and Shalev-Schwartz, S. 2022. Efficient Learning of CNNs using Patch Based Features. In International Conference on Machine Learning, 2336--2356. PMLR

work page 2022

[7] [7]

F.; and Olshausen, B

Cadieu, C. F.; and Olshausen, B. A. 2012. Learning intermediate-level representations of form and motion from natural movies. Neural computation, 24(4): 827--866

work page 2012

[8] [8]

Carreira, J.; and Zisserman, A. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6299--6308

2017