RECORD
ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
Computer Vision and Pattern Recognition · Submitted 22 Nov 2015
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global information. Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features. Upsampling layers follow ReNet layers to recover the original image resolution in the final predictions. The proposed ReSeg architecture is efficient, flexible and suitable for a variety of semantic segmentation tasks. We evaluate ReSeg on several widely-used semantic segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid; achieving state-of-the-art performance. Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems. The source code and model hyperparameters are available on https://github.com/fvisin/reseg.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. | contribution 0.85 | 0.81 | 0.08 | asserted 0.99 |
| s1 | The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. | method 0.67 | 0.33 | 0.03 | asserted 1.00 |
| s2 | We modify and extend it to perform the more challenging task of semantic segmentation. | contribution 0.56 | 0.89 | 0.04 | asserted 0.98 |
| s3 | Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global information. | method 1.00 | 0.25 | 0.03 | asserted 1.00 |
| s4 | Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features. | method 0.99 | 0.40 | 0.03 | asserted 1.00 |
| s5 | Upsampling layers follow ReNet layers to recover the original image resolution in the final predictions. | method 1.00 | 0.64 | 0.03 | asserted 1.00 |
| s6 | The proposed ReSeg architecture is efficient, flexible and suitable for a variety of semantic segmentation tasks. | interpretation 0.56 | 0.43 | 0.17 | asserted 1.00 |
| s7 | We evaluate ReSeg on several widely-used semantic segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid; achieving state-of-the-art performance. | result 0.93 | 0.97 | 0.20 | asserted 1.00 |
| s8 | Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems. | interpretation 0.91 | 0.89 | 0.76 | hedged possibility 0.87 |
| s9 | The source code and model hyperparameters are available on https://github.com/fvisin/reseg. | contribution 0.58 | 0.38 | 0.02 | asserted 0.99 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global information. Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features. Upsampling layers follow ReNet layers to recover the original image resolution in the final predictions. The proposed ReSeg architecture is efficient, flexible and suitable for a variety of semantic segmentation tasks. We evaluate ReSeg on several widely-used semantic segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid; achieving state-of-the-art performance. Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems. The source code and model hyperparameters are available on https://github.com/fvisin/reseg.
Human review
How the literature uses it
10 arXiv papers cite this in 13 sentences
Papers citing it · 5 sentences without a confident label
Uses its method or tool 1 paper
-
ReSeg combines Convolutional (CNN) and Recurrent Neural Networks (RNN) to perform semantic segmentation [ref] .
Encoder-decoder neural network for solving the nonlinear Fokker-Planck-Landau collision operator in XGC · Training Setup and Architecture · 0.83
Cites it as background 6 papers
-
The work in [ref] provided a method to incorporate contextual information using recurrent neural networks.
Convolutional Gated Recurrent Networks for Video Segmentation · Introduction · 0.90
-
Examples of Deep learning or ANN approach to image segmentation include, but not limited to, Convolutional Network models such as VGG16, GoogLeNet, Fast R-CNN, Faster R-CNN [ref] , U-NET [ref] , V-NET [ref] and Mask-RCNN [ref] , Recurrent Neural Network models include ReSeg [ref] ,Generative and Adversarial (GAN) Models e.g. [ref] , a comprehensive list of GAN-based techniques can be found in [ref] .
To GAN or Not To GAN: Segmentation Analysis on Mars DEM · Introduction · 0.96
-
An even harder task that greatly progressed is semantic segmentation, which provides per pixel labelling as introduced in [ref] [ref] [ref] .
Recurrent Fully Convolutional Networks for Video Segmentation · Introduction · 0.94
What it cites · 1 sentence without a confident label
Builds on 1 paper
-
The proposed ReSeg model builds on top of ReNet [ref] and extends it to address the task of semantic segmentation.
ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks · Model Description · 1.00
Uses the method or tool of 1 paper
-
We trained all our models with the Adadelta [ref] optimization algorithm, for its desired property of not requiring a specific hyperparameter tuning.
ADADELTA: An Adaptive Learning Rate Method · Experimental settings · 1.00
Cites as background 7 papers
-
Finally, in [ref] , ReNet is proposed to model pixel dependencies in the context of image classification.
ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks · Related Work · 0.88
-
Recent advances in semantic segmentation tend to convert the standard deep CNN classifier into Fully Convolutional Networks (FCN) [ref] to obtain coarse image representations, which are subsequently upsampled to recover the lost resolution.
Learning Deconvolution Network for Semantic Segmentation · Introduction · 0.93
-
Recent advances in semantic segmentation tend to convert the standard deep CNN classifier into Fully Convolutional Networks (FCN) [ref] to obtain coarse image representations, which are subsequently upsampled to recover the lost resolution.
U-Net: Convolutional Networks for Biomedical Image Segmentation · Introduction · 0.93
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.