Learning Non-Parametric Invariances from Data with Permanent Random Connectomes

Akshay Chawla; Dipan K. Pal; Marios Savvides

RECORD

Learning Non-Parametric Invariances from Data with Permanent Random Connectomes

Dipan K. Pal, Akshay Chawla, Marios Savvides

Machine Learning · Submitted 13 Nov 2019

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim We find that these randomly initialized permanent connections have positive effects on generalization, outperform much larger ConvNet baselines and the recently proposed Non-Parametric Transformation Network (NPTN) on benchmarks that enforce learning invariances from the data itself.

how it is stated · asserted 1.00 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID1911.05266 (version 3)

Pith ID2NNPKWWD, Pith's permanent record of this paper and its review

CitationAkshay Chawla, Dipan K. Pal, Marios Savvides. "Learning Non-Parametric Invariances from Data with Permanent Random Connectomes." arXiv:1911.05266 (2019). https://pith.science/paper/2NNPKWWD

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 7,627 input tokens.

What the abstract itself states, as probabilities: claims first 0.12 · states limitation 0.12 · human participants 0.03 · controlled comparison 0.91 · randomized assignment 0.02 · states quantitative result 0.06 · compares with prior results 0.98 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v3. Results and the main claim are underlined; show every role.

One of the fundamental problems in supervised classification and in machine learning in general, is the modelling of non-parametric invariances that exist in data. Most prior art has focused on enforcing priors in the form of invariances to parametric nuisance transformations that are expected to be present in data. Learning non-parametric invariances directly from data remains an important open problem. In this paper, we introduce a new architectural layer for convolutional networks which is capable of learning general invariances from data itself. This layer can learn invariance to non-parametric transformations and interestingly, motivates and incorporates permanent random connectomes, thereby being called Permanent Random Connectome Non-Parametric Transformation Networks (PRC-NPTN). PRC-NPTN networks are initialized with random connections (not just weights) which are a small subset of the connections in a fully connected convolution layer. Importantly, these connections in PRC-NPTNs once initialized remain permanent throughout training and testing. Permanent random connectomes make these architectures loosely more biologically plausible than many other mainstream network architectures which require highly ordered structures. We motivate randomly initialized connections as a simple method to learn invariance from data itself while invoking invariance towards multiple nuisance transformations simultaneously. We find that these randomly initialized permanent connections have positive effects on generalization, outperform much larger ConvNet baselines and the recently proposed Non-Parametric Transformation Network (NPTN) on benchmarks that enforce learning invariances from the data itself.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	One of the fundamental problems in supervised classification and in machine learning in general, is the modelling of non-parametric invariances that exist in data.	background 1.00	0.04	0.02	asserted 1.00
s1	Most prior art has focused on enforcing priors in the form of invariances to parametric nuisance transformations that are expected to be present in data.	background 0.96	0.03	0.02	asserted 1.00
s2	Learning non-parametric invariances directly from data remains an important open problem.	gap 0.98	0.05	0.03	asserted 1.00
s3	In this paper, we introduce a new architectural layer for convolutional networks which is capable of learning general invariances from data itself.	contribution 0.99	0.91	0.21	asserted 1.00
s4	This layer can learn invariance to non-parametric transformations and interestingly, motivates and incorporates permanent random connectomes, thereby being called Permanent Random Connectome Non-Parametric Transformation Networks (PRC-NPTN).	contribution 0.76	0.87	0.19	asserted 0.99
s5	PRC-NPTN networks are initialized with random connections (not just weights) which are a small subset of the connections in a fully connected convolution layer.	method 0.98	0.75	0.04	asserted 1.00
s6	Importantly, these connections in PRC-NPTNs once initialized remain permanent throughout training and testing.	method 0.97	0.64	0.06	asserted 1.00
s7	Permanent random connectomes make these architectures loosely more biologically plausible than many other mainstream network architectures which require highly ordered structures.	interpretation 0.98	0.25	0.04	asserted 0.56
s8	We motivate randomly initialized connections as a simple method to learn invariance from data itself while invoking invariance towards multiple nuisance transformations simultaneously.	method 0.62	0.53	0.09	asserted 0.88
s9	We find that these randomly initialized permanent connections have positive effects on generalization, outperform much larger ConvNet baselines and the recently proposed Non-Parametric Transformation Network (NPTN) on benchmarks that enforce learning invariances from the data itself.	result 1.00	0.97	0.77	asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.51.

What was examined, and what was not

sources read

arXiv listing abstract v3

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

One of the fundamental problems in supervised classification and in machine learning in general, is the modelling of non-parametric invariances that exist in data. Most prior art has focused on enforcing priors in the form of invariances to parametric nuisance transformations that are expected to be present in data. Learning non-parametric invariances directly from data remains an important open problem. In this paper, we introduce a new architectural layer for convolutional networks which is capable of learning general invariances from data itself. This layer can learn invariance to non-parametric transformations and interestingly, motivates and incorporates permanent random connectomes, thereby being called Permanent Random Connectome Non-Parametric Transformation Networks (PRC-NPTN). PRC-NPTN networks are initialized with random connections (not just weights) which are a small subset of the connections in a fully connected convolution layer. Importantly, these connections in PRC-NPTNs once initialized remain permanent throughout training and testing. Permanent random connectomes make these architectures loosely more biologically plausible than many other mainstream network architectures which require highly ordered structures. We motivate randomly initialized connections as a simple method to learn invariance from data itself while invoking invariance towards multiple nuisance transformations simultaneously. We find that these randomly initialized permanent connections have positive effects on generalization, outperform much larger ConvNet baselines and the recently proposed Non-Parametric Transformation Network (NPTN) on benchmarks that enforce learning invariances from the data itself.

Human review

How Pith checks papers → · Full audit of this paper →

How the literature uses it

What this paper does with the arXiv papers it cites

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 8 sentences without a confident label

Uses the method or tool of 1 paper

However, in order to validate permanent random connectomes on more realistic data, we utilize the CIFAR10 dataset and AutoAugmentation [ref] as the nuisance transformation.

AutoAugment: Learning Augmentation Policies from Data · Appendix · 0.91

Cites as background 5 papers

Learning invariances to nuisance transformations in data has emerged to be a core problem in machine learning [ref] .

Unsupervised Learning of Invariant Representations in Hierarchical Architectures · Introduction · 0.99
There have been many seminal works that have indeed explored the role of temporary random connections in deep networks such as DropOut [ref] , DropConnect [ref] and Stochastic Pooling [ref] .

Stochastic Pooling for Regularization of Deep Convolutional Neural Networks · Introduction · 0.92
Over the years, there have been efforts in investigating what other transformations result in useful hand-crafted priors in data such as rotation and scale [ref] .

Deep Learning by Scattering · Introduction · 0.97

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.