RECORD
Patch-level Neighborhood Interpolation: A General and Effective Graph-based Regularization Strategy
Machine Learning · Submitted 21 Nov 2019
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Extensive experiments have verified the effectiveness of our Pani approach in both supervised and semi-supervised settings.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Regularization plays a crucial role in machine learning models, especially for deep neural networks. The existing regularization techniques mainly rely on the i.i.d. assumption and only consider the knowledge from the current sample, without the leverage of the neighboring relationship between samples. In this work, we propose a general regularizer called \textbf{Patch-level Neighborhood Interpolation~(Pani)} that conducts a non-local representation in the computation of networks. Our proposal explicitly constructs patch-level graphs in different layers and then linearly interpolates neighborhood patch features, serving as a general and effective regularization strategy. Further, we customize our approach into two kinds of popular regularization methods, namely Virtual Adversarial Training (VAT) and MixUp as well as its variants. The first derived \textbf{Pani VAT} presents a novel way to construct non-local adversarial smoothness by employing patch-level interpolated perturbations. The second derived \textbf{Pani MixUp} method extends the MixUp, and achieves superiority over MixUp and competitive performance over state-of-the-art variants of MixUp method with a significant advantage in computational efficiency. Extensive experiments have verified the effectiveness of our Pani approach in both supervised and semi-supervised settings.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Regularization plays a crucial role in machine learning models, especially for deep neural networks. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | The existing regularization techniques mainly rely on the i.i.d. assumption and only consider the knowledge from the current sample, without the leverage of the neighboring relationship between samples. | gap 0.95 | 0.04 | 0.03 | asserted 1.00 |
| s2 | In this work, we propose a general regularizer called \textbf{Patch-level Neighborhood Interpolation~(Pani)} that conducts a non-local representation in the computation of networks. | contribution 0.97 | 0.90 | 0.14 | asserted 0.99 |
| s3 | Our proposal explicitly constructs patch-level graphs in different layers and then linearly interpolates neighborhood patch features, serving as a general and effective regularization strategy. | method 0.93 | 0.92 | 0.20 | asserted 1.00 |
| s4 | Further, we customize our approach into two kinds of popular regularization methods, namely Virtual Adversarial Training (VAT) and MixUp as well as its variants. | contribution 0.56 | 0.91 | 0.04 | asserted 0.99 |
| s5 | The first derived \textbf{Pani VAT} presents a novel way to construct non-local adversarial smoothness by employing patch-level interpolated perturbations. | method 0.69 | 0.92 | 0.07 | asserted 1.00 |
| s6 | The second derived \textbf{Pani MixUp} method extends the MixUp, and achieves superiority over MixUp and competitive performance over state-of-the-art variants of MixUp method with a significant advantage in computational efficiency. | result 0.91 | 0.94 | 0.22 | asserted 1.00 |
| s7 | Extensive experiments have verified the effectiveness of our Pani approach in both supervised and semi-supervised settings. | result 0.98 | 0.95 | 0.61 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Regularization plays a crucial role in machine learning models, especially for deep neural networks. The existing regularization techniques mainly rely on the i.i.d. assumption and only consider the knowledge from the current sample, without the leverage of the neighboring relationship between samples. In this work, we propose a general regularizer called \textbf{Patch-level Neighborhood Interpolation~(Pani)} that conducts a non-local representation in the computation of networks. Our proposal explicitly constructs patch-level graphs in different layers and then linearly interpolates neighborhood patch features, serving as a general and effective regularization strategy. Further, we customize our approach into two kinds of popular regularization methods, namely Virtual Adversarial Training (VAT) and MixUp as well as its variants. The first derived \textbf{Pani VAT} presents a novel way to construct non-local adversarial smoothness by employing patch-level interpolated perturbations. The second derived \textbf{Pani MixUp} method extends the MixUp, and achieves superiority over MixUp and competitive performance over state-of-the-art variants of MixUp method with a significant advantage in computational efficiency. Extensive experiments have verified the effectiveness of our Pani approach in both supervised and semi-supervised settings.
Human review
How the literature uses it
1 arXiv paper cite this in 1 sentence
What it cites · 1 sentence without a confident label
Uses the method or tool of 1 paper
-
Next, we can still utilize the similar power iteration and finite difference proposed in VAT [ref] to compute the desired perturbation $η^*$.
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning · Pani VAT · 0.99
Compares with 1 paper
-
Implement Details For fair comparison especially with VAT and its variants, such as VAT + SNTG [ref] and TNAR [ref] , we choose the standard large convolutional network as classifier as in [ref] .
Smooth Neighbors on Teacher Graphs for Semi-supervised Learning · Pani VAT · 0.80
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.