Learning Rich Representations For Structured Visual Prediction Tasks

Mohammadreza Mostajabi

REVIEW 3 major objections 5 minor 143 references

Learning Rich Representations For Structured Visual Prediction Tasks

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A feedforward classifier that sees each pixel's features across nested, increasingly broad image regions — the 'zoom-out' descriptor — can match or beat segmentation models built on explicit structured prediction.

desk verdict The thesis is mostly a retrospective of the author's own published work, but the depth-from-segmentation experiments are genuinely new and make the document worth a careful read. read the letter →

arxiv 1908.11820 v1 pith:GVOU4JMN submitted 2019-08-30 cs.CV cs.LG

Mohammadreza Mostajabi This is my paper · ORCID

classification cs.CVcs.LG

keywords semanticsegmentationzoom-outfeaturesmulti-scalecontextstructuredpredictionweaklysuperviseddiversesamplinglabelembeddingdeepconvolutionalnetworks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis's central claim is that a plain feedforward classifier can perform semantic segmentation as accurately as models that explicitly reason about label structure, provided each pixel or superpixel is described by features taken from a sequence of nested regions of increasing spatial extent. This 'zoom-out' descriptor bundles local appearance with proximal, distant, and scene-level context into one vector, capturing structure implicitly rather than through conditional random fields or structured support vector machines. The approach is shown to surpass prior hand-crafted-feature-plus-CRF pipelines on PASCAL VOC 2012 and to reach 86.6 mean IoU on the VOC 2012 test set when paired with a NASNet backbone. If the claim holds, explicit structured prediction becomes an optional refinement rather than a necessary component for competitive segmentation.

What carries the argument

The zoom-out descriptor: for each image element, a single vector $\phi_{\text{zoom-out}}(s,I) = [\phi_1(s,I), \dots, \phi_L(s,I)]$ formed by concatenating CNN features computed over $L$ nested regions of increasing spatial extent, starting at the superpixel or pixel, expanding through proximal and distant neighborhoods, adding a sub-scene window, and ending at the whole scene. Each level's features come from a convolutional layer whose receptive field defines the level, so the fixed architecture of a backbone CNN (the 16-layer network used initially, then ResNet, DenseNet, and NASNet) supplies both the features and the levels. This mechanism replaces explicit pairwise or higher-order potentials with implicitly encoded smoothness: nearby superpixels share largely overlapping zoom-out regions, so their feature vectors already reflect that context, and the final classifier is a feedforward multilayer network rather than an inference procedure over a graph.

What would settle it

On a suite of validation images with long-range label dependencies, compare raw zoom-out predictions to the same predictions after dense CRF post-processing; if the CRF improves mean IoU by more than about one point, the thesis's claim that the feedforward descriptor already captures the structure is falsified, since the thesis itself reports only a 0.1-point improvement for its strongest variant on VOC 2012.

Watch

Extended reading notes

Core claim

On its own terms, the thesis establishes that per-element image labeling can be treated as plain classification if the feature vector for each element incorporates context at multiple scales. For each superpixel (or pixel), the method concatenates CNN activations from nested regions, from the element itself through sub-scene windows and up to the entire image, then feeds the resulting vector to a multilayer network with skip connections spanning the zoom-out levels. The thesis reports that this representation, with no explicit structured prediction machinery, surpasses prior state of the art built on hand-crafted features with CRFs and structured SVMs on PASCAL VOC 2012, and that adding an off-the-shelf CRF on top of its best variant changes mean IoU by only 0.1 points — evidence that the feedforward model already captures the label dependencies the CRF would add. The same descriptor then supports a weakly supervised pipeline (localization score maps plus diverse point sampling) and a label-space autoencoder regularizer, each presented as a further extension of the same representation idea.

Load-bearing premise

The load-bearing premise is that the label dependencies needed for accurate segmentation are present in the multiscale feature vector of each element alone, so no explicit pairwise or higher-order model is required; if there exist scene structures whose resolution requires reasoning about distant elements that the zoom-out vector cannot encode, the central claim collapses.

Editorial extensions

If this is right

A segmentation network can be built by attaching a small classifier on top of any modern CNN backbone; the thesis shows zoom-out turning VGG-16, ResNet, DenseNet, and NASNet into competitive segmenters without dilated convolutions or CRFs.
Because each level's features are concatenated rather than summed, a nonlinear classifier can learn how much to rely on local versus global evidence; the paper reports that every zoom-out level group contributes to accuracy, with the full feature set reaching 58.6 mIoU on VOC 2012 val under a linear model.
The same representation drives a weakly supervised pipeline: class-specific localization score maps, combined with feature-space diverse sampling of 20 points per class per image, yield 40.6 mIoU on VOC 2012 val (45.2 with CRF), competitive with human point supervision.
Learning an autoencoder over ground-truth label maps and asking the network to predict through its frozen decoder improves a trained-from-scratch DenseNet-67 by 4.7 mIoU (40.5 to 45.2) and remains beneficial with ImageNet or COCO pretraining.
Depth maps can be predicted from semantic segmentation masks with lower error than from RGB in the thesis's NYUv2 experiments (linear RMSE 0.548 versus 0.628), indicating segmentation is a strong intermediate representation for jointly estimating depth.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A consequence the author leaves implicit is that the zoom-out recipe is a cheap, backbone-agnostic way to retrofit any image-classification network for dense prediction; a testable extension is whether the same concatenation scheme transfers unchanged to instance or panoptic segmentation, where the label structure is richer than in semantic segmentation.
The near-zero CRF gain on VOC suggests the representation saturates existing benchmark structure; a stress test with long-range label dependencies (for example, scenes where correctly labeling a region requires global competition among all object classes) would separate implicit structure capture from explicit structured reasoning.
The diverse-sampling rule penalizes feature-space similarity rather than spatial proximity; a direct extension would be to learn the embedding used for both the score map and the diversity term, so selected points maximize expected information gain for the downstream pixel classifier.
Label autoencoding could transfer to other structured outputs such as depth, surface normals, or pose, since the regularizer is derived purely from annotations; any label space with recoverable structure could receive the same two-phase training procedure.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here is my take on arXiv:1908.11820. It is a PhD thesis that repackages three of the author's own papers (zoom-out CVPR'15, diverse sampling arXiv'16, label regularization CVPR'18) and adds a relatively new depth-from-segmentation study. If you have seen those papers, not much in the first five chapters will surprise you. The genuinely new contribution is Section 5.5: predicting depth maps from segmentation maps. Table 5.5 shows that feeding segmentation alone into the depth network beats feeding RGB on NYUv2 (RMSE 0.548 vs 0.628), and combining both does better still. That is a strong, falsifiable result on a standard benchmark, and the two-part architecture in Table 5.6 also gives a clean gain (41 vs 38 mIoU segmentation) from using extra depth-only data. This is the part worth reading.

What the thesis does well is careful ablation. The superpixel oracle (94.4 mIoU) is a good sanity check. Table 3.4 shows each zoom-out level group contributes under linear classification, and Tables 4.2/4.3 show the value of diverse sampling over top-k or dense sampling. The comparisons are honest: in Table 4.4 the author acknowledges SEC is better in the weakly supervised regime, and the CRF gains are reported as small.

Soft spots. The biggest is the 'first to show segmentation benefits from deep CNNs' claim in Sections 1.1 and 3.6. It is inaccurate: FCN and Hypercolumns, both cited, appeared at the same time and made the same point. The thesis should say 'concurrently with' or 'we were among the first'. Second, the stress-test concern about the modern-backbone experiments is real. Table 3.9 compares zoom-out ResNet-101 to a dilated ResNet-101, but that is not the same network with the zoom-out head removed; the ablations in Table 3.4 are in a different regime (VGG-16, linear superpixel classifier). So the evidence that zoom-out, rather than backbone/inference protocol, drives the 86.6 mIoU number is weaker than it could be. Third, the weakly supervised point generation in Chapter 4 relies on the assumption that max activations of a class scoremap land on the class, verified only qualitatively (Section 4.3.2). That could be measured, and it is load-bearing. Fourth, no error bars anywhere, and Table 3.8 has a duplicated row (DenseNet-201 256x256 appears twice with different values).

Who is this for? A reader who wants the full derivation and ablation behind the zoom-out idea, or anyone interested in segmentation-to-depth transfer. It is a solid piece of empirical work, but the novelty is thin relative to the author's own prior publications.

I would send it to serious referees if it were submitted as a paper, mainly because the depth-from-segmentation result deserves scrutiny and replication. It is not desk-reject material. However, the 'first to show' overclaim and the missing controlled baseline should be fixed before publication.

Referee Report

3 major / 5 minor

Summary. The thesis presents three connected contributions to structured visual prediction. The core contribution is the zoom-out feature representation: for each pixel or superpixel, features are extracted from a nested sequence of regions of increasing spatial extent and concatenated, and a feedforward network classifies each element independently. The thesis argues that this simple construction captures local, proximal, distant, and scene-level context, making explicit structured prediction mechanisms such as CRFs unnecessary for competitive semantic segmentation. The second contribution is a weakly supervised segmentation pipeline that uses image-level class tags, class-specific localization score maps, and a diverse point-sampling procedure to generate automatic point supervision. The third contribution is a label-embedding regularizer: an autoencoder is learned over ground-truth label maps, and its frozen decoder is used as an auxiliary prediction branch during CNN training; the thesis also proposes a two-part architecture for joint depth and semantic segmentation. Experiments are reported on PASCAL VOC 2012, Stanford Background, Cityscapes, and NYUv2, with detailed ablations and test-set numbers.

Significance. If the claims hold, the zoom-out representation would be a simple and general recipe for injecting multiscale context into per-element classifiers, and the label-embedding regularizer would provide a way to exploit structure in annotations beyond the usual training loss. The manuscript has notable strengths: it reports extensive ablations on public benchmarks, including a superpixel oracle bound (94.4 mIoU, Section 3.4.2), level-by-level contribution analysis (Table 3.4), sampling-strategy comparisons (Tables 4.2 and 4.3), regularizer ablations (Tables 5.1 and 5.2), and held-out VOC 2012 test numbers that are falsifiable. However, two load-bearing points need work: the modern-backbone version of the zoom-out claim lacks a controlled baseline isolating the zoom-out construction from backbone strength and training protocol, and the weakly supervised pipeline rests on an assumption about score-map-to-pixel correspondence that the text admits was only qualitatively checked.

major comments (3)

[Section 3.5, Table 3.9] The central claim that zoom-out features, rather than the backbone or the training/inference protocol, are responsible for the reported accuracy is not directly tested in the modern-backbone experiments. Table 3.9 compares zoom-out ResNet-101 (77.6 mIoU) with dilated ResNet-101 (77.4 mIoU), but these are different architectures, and the dilated baseline is not the same network with the zoom-out head removed. No result is reported for a plain DenseNet-121, DenseNet-201, or NASNet-A FCN trained with the same crop size, multi-scale inference, and COCO pretraining but using only the final-layer feature map or a simple skip-connection head. The ablations in Table 3.4 that show the contribution of individual levels use VGG-16 features with a linear superpixel classifier, a substantially different regime. Adding a controlled baseline with the same backbone and training protocol would make the zoom-out contribution identifiable.
[Section 4.3.2, Eqs. (4.3.3)-(4.3.5)] The weakly supervised pipeline assumes that maximum activations of a class-specific score map correspond to pixels of that class when the class is present. The manuscript states: 'the maximum activations of a class scoremap when that class is present appear to reliably correspond with pixels containing the correct class. (We verified this qualitatively, on a few classes and a number of training images)' (Section 4.3.2). This assumption is load-bearing for the Chapter 4 results (40.6 mIoU on val, 45.2 with CRF), yet the correctness of the automatically generated point labels is never measured. I recommend quantifying label precision on a held-out subset with available ground truth, or otherwise providing a direct evaluation of the point labels, so that the reported segmentation accuracy can be attributed to the proposed sampling mechanism rather than to an untested localization assumption.
[Section 5.3.3, Tables 5.1-5.2] The label-embedding regularizer is evaluated through several ablations, which is a strength. However, Table 5.1 reports gains that in some settings are small (e.g., DenseNet-121 with ImageNet pretraining: 71.6 to 71.9 mIoU), and the paper does not report variance or significance over multiple runs. Given that the central claim of Chapter 5 is a consistent accuracy boost, reporting the mean and spread over at least a few random seeds for the main comparisons would strengthen the claim and rule out optimization noise.

minor comments (5)

[Table 3.8] The row 'zoom-out DenseNet-201 256x256' appears twice with different values (77.4 and 78.6 mIoU) under the same column settings; the difference (for example, COCO pretraining or some other configuration) should be stated explicitly.
[Section 3.4.2] The text refers to 'asymmetric log-loss (3.4.1)', but Equation 3.4.1 defines IoU; the loss is defined in Equation 3.3.1. The cross-reference should be corrected.
[Chapter 4, reference [10]] In the Introduction and in Section 4.3, the VOC 2012 benchmark is cited with reference [10], which is the author's own arXiv preprint on diverse sampling; this citation mismatch should be fixed to the PASCAL VOC reference.
[Figure 3.1 caption] The caption contains a typo: 'comcantenated' should be 'concatenated'.
[Section 4.4.1] The final segmentation model is trained on the selected points with batch size 100 and for two epochs in less than three minutes; reporting the number of optimization steps and convergence behavior would make this speed claim more reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the thesis's claims are evaluated against external benchmarks and each stage is empirically grounded rather than reducing to its inputs.

full rationale

The thesis does not derive its headline results from definitions that already contain those results. The zoom-out claim in Chapter 3 is an empirical architecture proposal: features are concatenated from nested regions and classified by a feedforward network, with accuracy measured on held-out VOC test server, Stanford Background, and later Cityscapes and NYUv2. The VOC 2012 test numbers are genuine external benchmark submissions, not quantities fitted into the model. The weak supervision pipeline in Chapter 4 trains localization models from image-level tags, samples diverse points, and then trains a segmentation model on those points; the reported mIoU is computed against independent validation/test ground truth, so the evaluation is not forced by the label-generation procedure. The label-embedding regularizer in Chapter 5 is an auxiliary training signal learned from training annotations and measured on held-out validation, not a redefinition of the target. The paper does cite the author's prior work ([91], [137], [10], [17]), but the thesis is self-contained: the zoom-out method is fully described in Chapter 3, and the citations do not supply an unverified premise that carries the argument. One limitation in Section 4.3.2 is the qualitative verification of score-to-pixel correspondence, but that is an unsupported assumption about localization quality, not a circular reduction. Missing controlled baselines in the modern-backbone experiments would be a correctness or experimental-design concern, not a circularity, and therefore does not raise the circularity score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

This is an empirical machine learning thesis, so the ledger records validation-chosen hyperparameters, domain assumptions about transfer and label structure, and the trained modules treated as invented entities. No code is shipped, so nothing is machine-checked; the supporting evidence is benchmark accuracy on held-out data.

free parameters (5)

diverse sampling count k = 20
Chosen on VOC 2012 validation by sweeping {1, 5, 10, 20, 50} (Table 4.2); the number of point labels per class per image drives the weakly supervised pipeline.
auxiliary loss weight = 0.5 relative weight
Tuned on the PASCAL validation curve in Figure 5.5; the regularizer's benefit and the close-margin gains in Tables 5.1 through 5.4 depend on this weight.
SLIC superpixel count and compactness = about 500 superpixels, m = 15
Chosen in Section 3.4.1 as preprocessing; the oracle test (94.4 mIoU) validates the choice, but the final accuracy depends on it.
zoom-out level set = 13 conv layers plus window and scene levels (15 total)
Determined by the VGG-16 backbone and two extra levels (Table 3.1); the ablation in Table 3.4 shows each group contributes.
autoencoder bottleneck width = 128 channels
Among {32, 128, 256}, the 128-channel decoder gives the best PASCAL result (Table 5.1) and is used in the pretrained-regime experiments.

assumptions (5)

domain assumption ImageNet-pretrained CNN features, without fine-tuning on VOC, serve as useful spatial features for segmentation and localization.
Used in Section 3.4.1.3 and Chapter 4; the thesis states VGG-16 was not fine-tuned on VOC, so transferability of ImageNet features is assumed and only indirectly validated by the final results.
domain assumption Locations of maximum class score in global-softmax localization maps coincide with pixels of that class when the class is present.
Load-bearing for Chapter 4's point sampling; the stated support is qualitative (Section 4.3.2).
domain assumption Autoencoding ground-truth label maps through a bottleneck yields structure (shape, co-occurrence) whose prediction regularizes the segmentation network.
Chapter 5 assumes decoder-forced predictions act as a useful regularizer; supported indirectly by introspection (Figure 5.7) and by validation gains.
domain assumption Majority-vote labeling of SLIC superpixels loses little information relative to pixel-level supervision.
Section 3.4.2 checks this empirically (oracle 94.4 mIoU); if superpixel boundaries misaligned with objects, the Chapter 3 method would be capped below that oracle.
domain assumption External implementations (Caffe, Torch7, PyTorch, torchvision pretrained weights, fully connected CRF of Krahenbuhl and Koltun) behave as documented.
All experiments rely on third-party frameworks and pretrained models; no code is shipped to verify the exact pipeline.

invented entities (2)

zoom-out feature representation independent evidence
purpose: Concatenated CNN activations from nested regions of increasing extent around each pixel or superpixel, used as the input descriptor for a feedforward classifier.
It is a concrete, measurable vector (12,416 dimensions with VGG-16, Table 3.1) benchmarked on VOC, Stanford Background, and Cityscapes, not an unobservable postulate.
label-space autoencoder decoder branch independent evidence
purpose: Auxiliary training branch that routes predictions through a frozen decoder learned from ground-truth label maps and is discarded at test time.
Its value is evidenced by consistent PASCAL validation gains (Tables 5.1, 5.3, 5.4) and by introspection (Figure 5.7); it adds no test-time machinery.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Rich Representations For Structured Visual Prediction Tasks." pith.science (2026). https://pith.science/paper/GVOU4JMN

@misc{pith2026190811820,
  author       = {Pith},
  title        = {Pith review of: Learning Rich Representations For Structured Visual Prediction Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GVOU4JMN}},
  note         = {Machine review of arXiv:1908.11820}
}

read the original abstract

We describe an approach to learning rich representations for images, that enables simple and effective predictors in a range of vision tasks involving spatially structured maps. Our key idea is to map small image elements to feature representations extracted from a sequence of nested regions of increasing spatial extent. These regions are obtained by "zooming out" from the pixel/superpixel all the way to scene-level resolution, and hence we call these zoom-out features. Applied to semantic segmentation and other structured prediction tasks, our approach exploits statistical structure in the image and in the label space without setting up explicit structured prediction mechanisms, and thus avoids complex and expensive inference. Instead image elements are classified by a feedforward multilayer network with skip-layer connections spanning the zoom-out levels. When used in conjunction with modern neural architectures such as ResNet, DenseNet and NASNet (to which it is complementary) our approach achieves competitive accuracy on segmentation benchmarks. In addition, we propose an approach for learning category-level semantic segmentation purely from image-level classification tag. It exploits localization cues that emerge from training a modified zoom-out architecture tailored for classification tasks, to drive a weakly supervised process that automatically labels a sparse, diverse training set of points likely to belong to classes of interest. Finally, we introduce data-driven regularization functions for the supervised training of CNNs. Our innovation takes the form of a regularizer derived by learning an autoencoder over the set of annotations. This approach leverages an improved representation of label space to inform extraction of features from images

Figures

Figures reproduced from arXiv: 1908.11820 by the authors.

**Figure 2.1.** An example image from BSDS dataset with its segmentation annotations provided [PITH_FULL_IMAGE:figures/full_fig_p020_2_1.png] view at source ↗

**Figure 2.2.** SLIC segmentation results with different values for number of superpixels [PITH_FULL_IMAGE:figures/full_fig_p023_2_2.png] view at source ↗

**Figure 3.1.** Schematic description of our approach. Features are extracted from a nested [PITH_FULL_IMAGE:figures/full_fig_p029_3_1.png] view at source ↗

Figures from the paper (24 more)

**Figure 3.2.** Figure 3.2: Examples of zoom-out regions. We show four out of fifteen levels: [PITH_FULL_IMAGE:figures/full_fig_p033_3_2.png]

**Figure 3.3.** Figure 3.3: Showing three superpixels in each image (top), followed by corresponding zoom [PITH_FULL_IMAGE:figures/full_fig_p034_3_3.png]

**Figure 3.4.** Figure 3.4: Illustration of zoom-out feature extraction procedure for a simple network with [PITH_FULL_IMAGE:figures/full_fig_p038_3_4.png]

**Figure 3.5.** Figure 3.5: Color code for VOC categories. Background is black. [PITH_FULL_IMAGE:figures/full_fig_p046_3_5.png]

**Figure 3.6.** Figure 3.6: Examples illustrating the effect of zoom-out levels. From left: original image, [PITH_FULL_IMAGE:figures/full_fig_p046_3_6.png]

**Figure 3.7.** Figure 3.7: Example segmentations on VOC 2012 val with 3-layer neural network used to [PITH_FULL_IMAGE:figures/full_fig_p049_3_7.png]

**Figure 3.8.** Figure 3.8: Illustration of zoom-out DenseNet with three dense blocks. [PITH_FULL_IMAGE:figures/full_fig_p051_3_8.png]

**Figure 3.9.** Figure 3.9: Illustration of zoom-out ResNet with three ResNet blocks. Dashed lines denote [PITH_FULL_IMAGE:figures/full_fig_p052_3_9.png]

**Figure 3.10.** Figure 3.10: Example segmentations on VOC 2012 test set with our zoom-out NASNet-A. [PITH_FULL_IMAGE:figures/full_fig_p055_3_10.png]

**Figure 3.11.** Figure 3.11: Visualization results of zoom-out dilated ResNet-101 on Cityscapes validation [PITH_FULL_IMAGE:figures/full_fig_p059_3_11.png]

**Figure 4.1.** Figure 4.1: Our diverse sampling procedure. Left: input image, labeled as containing class [PITH_FULL_IMAGE:figures/full_fig_p066_4_1.png]

**Figure 4.2.** Figure 4.2: Example segmentations on extra classes from MS-COCO dataset added to the [PITH_FULL_IMAGE:figures/full_fig_p076_4_2.png]

**Figure 4.3.** Figure 4.3: Examples of diverse sampling outputs. For each foreground class, we show [PITH_FULL_IMAGE:figures/full_fig_p077_4_3.png]

**Figure 4.4.** Figure 4.4: Examples of segmentations learned through our weakly supervised approach. [PITH_FULL_IMAGE:figures/full_fig_p078_4_4.png]

**Figure 5.1.** Figure 5.1: Exploiting label structure when training semantic segmentation. Top: An initial [PITH_FULL_IMAGE:figures/full_fig_p080_5_1.png]

**Figure 5.2.** Figure 5.2: Informative structure in annotation. The shape of labeled semantic regions hints at [PITH_FULL_IMAGE:figures/full_fig_p084_5_2.png]

**Figure 5.3.** Figure 5.3: DenseNet architectural specifications. 5.3.1 Modeling labels As shown in [PITH_FULL_IMAGE:figures/full_fig_p086_5_3.png]

**Figure 5.4.** Figure 5.4: Alternative regularization scheme. Instead of predicting a representation to pass [PITH_FULL_IMAGE:figures/full_fig_p087_5_4.png]

**Figure 5.5.** Figure 5.5: Auxiliary loss weighting. We plot validation performance as a function of the [PITH_FULL_IMAGE:figures/full_fig_p092_5_5.png]

**Figure 5.6.** Figure 5.6: Semantic segmentation results on PASCAL. We show the output of a baseline [PITH_FULL_IMAGE:figures/full_fig_p093_5_6.png]

**Figure 5.** Figure 5: visualizes the impact of training with our learned label decoder as a regularizer. [PITH_FULL_IMAGE:figures/full_fig_p096_5.png]

**Figure 5.7.** Figure 5.7: Finding regions with similar representations. For each [PITH_FULL_IMAGE:figures/full_fig_p097_5_7.png]

**Figure 5.8.** Figure 5.8: Left image shows semantic segmentation map and the right image is its coresspond [PITH_FULL_IMAGE:figures/full_fig_p100_5_8.png]

**Figure 5.9.** Figure 5.9: Depth predictions results on NYUv2. We show not only it is possible to predict [PITH_FULL_IMAGE:figures/full_fig_p101_5_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

143 extracted references · 59 canonical work pages

[1]

Pushmeet Kohli, L’ubor Ladicky, and Philip H. S. Torr. Robust higher order potentials for enforcing label consistency. In CVPR, 2008. ISBN 978-1-4244-2242-5. doi: http://doi.ieeecomputersociety.org/10.1109/CVPR.2008.4587417

arXiv 2008
[2]

Shotton, J

J.D.J. Shotton, J. Winn, C. Rother, and A. Criminisi. Textonboost for image under- standing: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV, 81(1), 2009

2009
[3]

Efﬁcient inference in fully connected crfs with gaussian edge potentials

Philipp Krähenbühl and Vladlen Koltun. Efﬁcient inference in fully connected crfs with gaussian edge potentials. Adv. Neural Inf. Process. Syst, 2011

2011
[4]

Ladick`y, C

L. Ladick`y, C. Russell, P. Kohli, and P. H. S. Torr. Associative hierarchical CRFs for object class image segmentation. ICCV, 2009

2009
[5]

Pylon model for semantic segmentation

Victor Lempitsky, Andrea Vedaldi, and Andrew Zisserman. Pylon model for semantic segmentation. In NIPS, pages 1485–1493, 2011

2011
[6]

Discriminative re-ranking of diverse segmentations

Payman Yadollahpour, Dhruv Batra, and Gregory Shakhnarovich. Discriminative re-ranking of diverse segmentations. In CVPR, 2013

2013
[7]

Diverse M-Best Solutions in Markov Random Fields

Dhruv Batra, Payman Yadollahpour, Abner Guzman-Rivera, and Greg Shakhnarovich. Diverse M-Best Solutions in Markov Random Fields. In ECCV, 2012. 108

2012
[8]

Gonfaus, Joost van de Weijer, Andrew D

Xavier Boix, Josep M. Gonfaus, Joost van de Weijer, Andrew D. Bagdanov, Joan Serrat Gual, and Jordi Gonzàlez. Harmony potentials - fusing global and local scale for semantic image segmentation. IJCV, 96(1):83–102, 2012

2012

Show all 143 references

[9]

A robust multilevel segment de- scription for multi-class object recognition

Mohammadreza Mostajabi and Iman Gholampour. A robust multilevel segment de- scription for multi-class object recognition. Machine Vision and Applications, 2014

2014
[10]

Kolkin, and Gregory Shakhnarovich

Mohammadreza Mostajabi, Nicholas I. Kolkin, and Gregory Shakhnarovich. Diverse sampling for self-supervised learning of semantic segmentation. arXiv preprint, 2016

2016
[11]

ImageNet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. CVPR, 2009

2009
[12]

Semantic image segmentation via deep parsing network

Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, and Xiaoou Tang. Semantic image segmentation via deep parsing network. ICCV, 2015

2015
[14]

Pyramid scene parsing network

Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. CVPR, 2017

2017
[15]

The PASCAL Visual Object Classes (VOC) challenge

Mark Everingham, Luc van Gool, Chris Williams, John Winn, and Andrew Zisserman. The PASCAL Visual Object Classes (VOC) challenge. IJCV, 2010

2010
[16]

Lawrence Zitnick

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. ECCV, 2014

2014
[17]

Regularizing 109 deep networks by modeling and predicting label structure

Mohammadreza Mostajabi, Michael Maire, and Gregory Shakhnarovich. Regularizing 109 deep networks by modeling and predicting label structure. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

2018
[18]

Arbelaez, M

P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik. Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):898–916, May 2011

2011
[19]

Martin, C

D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring eco- logical statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001

2001
[20]

Slic superpixels compared to state-of-the-art superpixel methods

Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE TPAMI, 34(11), 2012

2012
[21]

Comaniciu and P

D. Comaniciu and P. Meer. Mean shift: a robust approach toward feature space analysis. PAMI, 24(5):603–619, 2002

2002
[22]

Carreira and C

J. Carreira and C. Sminchisescu. Constrained parametric min-cuts for automatic object segmentation. In CVPR, 2010. doi: 10.1109/CVPR.2010.5540063

2010
[23]

Normalized cuts and image segmentation

Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. PAMI, 22(8): 888–905, 2000. ISSN 0162-8828. doi: http://doi.ieeecomputersociety.org/10.1109/34. 868688

2000 doi
[24]

W. Tu, M. Liu, V . Jampani, D. Sun, S. Chien, M. Yang, and J. Kautz. Learning superpixels with segmentation-aware afﬁnity loss. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 568–576, June 2018. doi: 10.1109/ CVPR.2018.00066. 110

2018
[25]

W-net: A deep model for fully unsupervised image segmen- tation

Xide Xia and Brian Kulis. W-net: A deep model for fully unsupervised image segmen- tation. arXiv preprint, abs/1711.08506, 2017. URL

2017 arXiv
[26]

Girshick, Carsten Rother, and Piotr Dollár

Alexander Kirillov, Kaiming He, Ross B. Girshick, Carsten Rother, and Piotr Dollár. Panoptic segmentation. arXiv preprint, abs/1801.00868, 2018

2018 arXiv
[27]

Cpmc: Automatic object segmentation using constrained parametric min-cuts

Joao Carreira and Cristian Sminchisescu. Cpmc: Automatic object segmentation using constrained parametric min-cuts. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(7), 2012

2012
[28]

J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders. Selective search for object recognition. International Journal of Computer Vision, 104(2), 2013

2013
[29]

Arbelaez, B

P. Arbelaez, B. Hariharan, Chunhui Gu, S. Gupta, L. Bourdev, and J. Malik. Semantic segmentation using regions and parts. In CVPR, 2012

2012
[30]

Semantic segmentation with second-order pooling

João Carreira, Rui Caseiro, Jorge Batista, and Cristian Sminchisescu. Semantic segmentation with second-order pooling. In Computer Vision–ECCV 2012, pages 430–443. Springer, 2012

2012
[31]

Probabilistic joint image segmentation and labeling

Adrian Ion, Joao Carreira, and Cristian Sminchisescu. Probabilistic joint image segmentation and labeling. In NIPS, pages 1827–1835, 2011

2011
[32]

Object recognition by sequential ﬁgure-ground ranking

João Carreira, Fuxin Li, and Cristian Sminchisescu. Object recognition by sequential ﬁgure-ground ranking. IJCV, 98(3):243–262, 2012

2012
[33]

Z. Li, E. Gavves, K. E. A. van de Sande, C. G. M. Snoek, and A. W. M. Smeulders. Codemaps segment, classify and search objects locally. In ICCV, 2013

2013
[34]

Simultaneous detection and segmentation

Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. Simultaneous detection and segmentation. In Computer Vision–ECCV 2014, 2014. 111

2014
[35]

Combining the best of graphical models and convnets for semantic segmentation

Michael Cogswell, Xiao Lin, Senthil Purushwalkam, and Dhruv Batra. Combining the best of graphical models and convnets for semantic segmentation. arXiv preprint, abs/1412.4313, 2014. URL

2014 arXiv
[36]

Class segmentation and object localization with superpixel neighborhoods

Brian Fulkerson, Andrea Vedaldi, and Stefano Soatto. Class segmentation and object localization with superpixel neighborhoods. In CVPR, 2009

2009
[37]

Context by region ancestry

Joseph J Lim, Pablo Arbeláez, Chunhui Gu, and Jitendra Malik. Context by region ancestry. In ICCV, 2009

2009
[38]

Are spatial and global constraints really necessary for segmentation? In ICCV, 2011

Aurélien Lucchi, Yunpeng Li, Xavier Boix, Kevin Smith, and Pascal Fua. Are spatial and global constraints really necessary for segmentation? In ICCV, 2011

2011
[39]

Learning hierarchical features for scene labeling

Clément Farabet, Camille Couprie, Laurent Najman, and Yann LeCun. Learning hierarchical features for scene labeling. IEEE TPAMI, 35(8), 2013

2013
[40]

P. H. O. Pinheiro and R. Collobert. Recurrent convolutional neural networks for scene labeling. In ICML, 2014

2014
[41]

Lin, Andrew Y

Richard Socher, Cliff C. Lin, Andrew Y . Ng, and Christopher D. Manning. Parsing Natural Scenes and Natural Language with Recursive Neural Networks. In ICML, 2011

2011
[42]

Deep and wide multiscale recursive networks for robust image labeling

Gary B Huang and Viren Jain. Deep and wide multiscale recursive networks for robust image labeling. arXiv preprint arXiv:1310.0354, 2013

2013 arXiv
[43]

Fully convolutional networks for semantic segmentation

Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. CVPR, 2015

2015
[44]

Hypercolumns for object segmentation and ﬁne-grained localization

Bharath Hariharan, Pablo Arbelaez, Ross Girshick, and Jitendra Malik. Hypercolumns for object segmentation and ﬁne-grained localization. CVPR, 2015. 112

2015
[45]

Slic superpixels compared to state-of-the-art superpixel methods

Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine SŸsstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE TPAMI, 2012

2012
[46]

Williams, John Winn, and Andrew Zisserman

Mark Everingham, S.M.Ali Eslami, Luc Van Gool, ChristopherK.I. Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospec- tive. International Journal of Computer Vision, 2014

2014
[47]

Imagenet large scale visual recognition challenge

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. arXiv:1409.0575, 2014

2014 arXiv
[48]

Structured output learning with high order loss functions

Daniel Tarlow and Richard S Zemel. Structured output learning with high order loss functions. In AISTATS, 2012

2012
[49]

Segmentation propagation in imagenet

Daniel Kuettel, Matthieu Guillaumin, and Vittorio Ferrari. Segmentation propagation in imagenet. In ECCV, 2012

2012
[50]

Semantic contours from inverse detectors

Bharath Hariharan, Pablo Arbelaez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In International Conference on Computer Vision (ICCV), 2011

2011
[51]

Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation

Jamie Shotton, John Winn, Carsten Rother, and Antonio Criminisi. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 2006. URL

2006
[52]

David G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91–110, November 2004. ISSN 0920-5691. doi: 10.1023/B: VISI.0000029664.99615.94

2004
[53]

Chatﬁeld, K

K. Chatﬁeld, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In BMVC, 2014. 113

2014
[54]

Simonyan and A

K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint , 2014

2014
[55]

Girshick, J

R. Girshick, J. Donohue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv preprint , 2014

2014
[56]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016

2016
[57]

Weinberger

Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. CVPR, 2017

2017
[58]

Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V . Le. Learning transferable architectures for scalable image recognition. arXiv preprint, abs/1707.07012, 2017

2017 arXiv
[59]

Caffe: Convolutional architecture for fast feature embedding

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint , 2014

2014
[60]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overﬁtting.Journal of Machine Learning Research, 15, 2014

2014
[61]

J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. arXiv preprint , 2015

2015
[62]

Hariharan, P

B. Hariharan, P. Aberláez an R. Girshick, and J. Malik. Hypercolumns for object segmentation and ﬁne-grained localization. arXiv preprint , 2015

2015
[63]

Imagenet classiﬁcation with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classiﬁcation with deep convolutional neural networks. In NIPS, 2012

2012
[64]

https://github.com/pytorch/pytorch

PyTorch. https://github.com/pytorch/pytorch. 114
[65]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015

2015
[66]

https://github.com/pytorch/vision

PyTorch torchvision. https://github.com/pytorch/vision
[67]

Multi-scale context aggregation by dilated convolutions

Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. ICLR, 2016

2016
[68]

Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs

Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. arXiv:1606.00915, 2016

2016 arXiv
[69]

Holschneider, R

M. Holschneider, R. Kronland-Martinet, J. Morlet, and Ph. Tchamitchian. A real-time algorithm for signal analysis with the help of the wavelet transform. In Jean-Michel Combes, Alexander Grossmann, and Philippe Tchamitchian, editors, Wavelets, pages 286–297, Berlin, Heidelberg...

1990
[70]

Semantic understanding of scenes through the ade20k dataset

Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ade20k dataset. International Journal on Computer Vision, 2018

2018
[71]

Re- thinking atrous convolution for semantic image segmentation

Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Re- thinking atrous convolution for semantic image segmentation. arXiv preprint , abs/1706.05587, 2017

2017 arXiv
[72]

Encoder-decoder with atrous separable convolution for semantic image seg- mentation

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image seg- mentation. arXiv preprint, abs/1802.02611, 2018

2018 arXiv
[73]

Multi-scale context intertwining for semantic segmentation

Di Lin, Yuanfeng Ji, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. Multi-scale context intertwining for semantic segmentation. In The European Conference on Computer Vision (ECCV), September 2018. 115

2018
[74]

Cordts, M

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3213–3223, June 2016

2016
[75]

Torchcv: A pytorch-based framework for deep learning in computer vision

Ansheng You and Yunhai Tong. Torchcv: A pytorch-based framework for deep learning in computer vision. , 2019

2019
[76]

H. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and R. M. Summers. Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5):1285–1298...

2016
[77]

Roth, Le Lu, Amal Farag, Hoo-Chang Shin, Jiamin Liu, Evrim B

Holger R. Roth, Le Lu, Amal Farag, Hoo-Chang Shin, Jiamin Liu, Evrim B. Turkbey, and Ronald M. Summers. Deeporgan: Multi-level deep convolutional networks for auto- mated pancreas segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro Frangi, editors...

2015
[78]

Roth, Jiamin Liu, Evrim Turkbey, and Ronald M

Amal Farag, Le Lu, Holger R. Roth, Jiamin Liu, Evrim Turkbey, and Ronald M. Summers. A bottom-up approach for pancreas segmentation using cascaded superpixels and (deep) image patch labeling. Trans. Img. Proc., 26(1):386–399, January 2017. ISSN 1057-7149. doi: 10.1109/TIP.2016...

2017
[79]

Training deep networks to be spatially sensitive

Nicholas Kolkin, Eli Shechtman, and Gregory Shakhnarovich. Training deep networks to be spatially sensitive. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017

2017
[80]

Shimoda and K

W. Shimoda and K. Yanai. Weakly-supervised segmentation by combining cnn feature 116 maps and object saliency maps. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 1935–1940, Dec 2016

2016
[81]

Learning representations for automatic colorization

Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Learning representations for automatic colorization. ECCV, 2016

2016
[82]

T. M. H. C. Santana, K. Nogueira, A. M. C. Machado, and J. A. dos Santos. Deep contextual description of superpixels for aerial urban scenes classiﬁcation. In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) , pages 3027–3031, July 2017

2017
[83]

A. M. El Amin, Q. Liu, and Y . Wang. Zoom out cnns features for optical remote sensing change detection. In 2017 2nd International Conference on Image, Vision and Computing (ICIVC), pages 812–817, June 2017

2017
[84]

Learning deep features for scene recognition using places database

Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. Learning deep features for scene recognition using places database. In NIPS, 2014

2014
[85]

The pascal visual object classes challenge: A retrospective

Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. IJCV, 111(1), 2015

2015
[86]

Microsoft coco: Common objects in context

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014

2014
[87]

Object detectors emerge in deep scene cnns

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene cnns. In ICLR, 2015

2015
[88]

Is object localization for free?-weakly-supervised learning with convolutional neural networks

Maxime Oquab, Léon Bottou, Ivan Laptev, and Josef Sivic. Is object localization for free?-weakly-supervised learning with convolutional neural networks. In CVPR, 2015. 117

2015
[89]

What’s the point: Semantic segmentation with point supervision

Olga Russakovsky, Amy L Bearman, Vittorio Ferrari, and Fei-Fei Li. What’s the point: Semantic segmentation with point supervision. arXiv:1506.02106, 2015

2015 arXiv
[90]

Hariharan, P

B. Hariharan, P. Aberláez an R. Girshick, and J. Malik. Hypercolumns for object segmentation and ﬁne-grained localization. In CVPR, 2015

2015
[91]

Mostajabi, P

M. Mostajabi, P. Yadollahpour, and G. Shakhnarovich. Feedforward semantic segmen- tation with zoom-out features. In CVPR, 2015

2015
[92]

Weakly- and semi-supervised learning of a deep convolutional network for semantic image segmentation

George Papandreou, Liang-Chieh Chen, Kevin P Murphy, and Alan L Yuille. Weakly- and semi-supervised learning of a deep convolutional network for semantic image segmentation. In ICCV, 2015

2015
[93]

Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation

Jifeng Dai, Kaiming He, and Jian Sun. Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In ICCV, 2015

2015
[94]

Weakly supervised semantic labelling and instance segmentation

Anna Khoreva, Rodrigo Benenson, Jan Hosang, Matthias Hein, and Bernt Schiele. Weakly supervised semantic labelling and instance segmentation. arXiv:1603.07485, 2016

2016 arXiv
[95]

Constrained convolutional neural networks for weakly supervised segmentation

Deepak Pathak, Philipp Krahenbuhl, and Trevor Darrell. Constrained convolutional neural networks for weakly supervised segmentation. In CVPR, 2015

2015
[96]

Self-taught object localization with deep networks

Alessandro Bergamo, Loris Bazzani, Dragomir Anguelov, and Lorenzo Torresani. Self-taught object localization with deep networks. arXiv:1409.3964, 2014

2014 arXiv
[97]

Weakly supervised object localization with multi-fold multiple instance learning

Ramazan Gokberk Cinbis, Jakob Verbeek, and Cordelia Schmid. Weakly supervised object localization with multi-fold multiple instance learning. arXiv:1503.00949, 2015

2015 arXiv
[98]

Schwing, and Raquel Urtasun

Jia Xu, Alexander G. Schwing, and Raquel Urtasun. Learning to segment under various forms of weak supervision. In CVPR, 2015. 118

2015
[99]

Fully convolu- tional multi-class multiple instance learning

Deepak Pathak, Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolu- tional multi-class multiple instance learning. In ICLR workshop, 2015

2015
[100]

From image-level to pixel-level labeling with convolutional networks

Pedro O Pinheiro and Ronan Collobert. From image-level to pixel-level labeling with convolutional networks. In CVPR, 2015

2015
[101]

Deconvolutional feature stacking for weakly- supervised semantic segmentation

Hyo-Eun Kim and Sangheum Hwang. Deconvolutional feature stacking for weakly- supervised semantic segmentation. arXiv:1602.04984, 2016

2016 arXiv
[102]

Seed, expand and constrain: Three principles for weakly-supervised image segmentation

Alexander Kolesnikov and Christoph H Lampert. Seed, expand and constrain: Three principles for weakly-supervised image segmentation. arXiv:1603.06098, 2016

2016 arXiv
[103]

Diverse M-Best Solutions in Markov Random Fields

Dhruv Batra, Payman Yadollahpour, Abner Guzman-Rivera, and Greg Shakhnarovich. Diverse M-Best Solutions in Markov Random Fields. In ECCV, 2012

2012
[104]

Stc: A simple to complex framework for weakly-supervised semantic segmentation

Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming-Ming Cheng, Yao Zhao, and Shuicheng Yan. Stc: A simple to complex framework for weakly-supervised semantic segmentation. arXiv:1509.03150, 2015

2015 arXiv
[105]

Adam: A method for stochastic optimization

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015

2015
[106]

Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR, 2015

2015
[107]

High-performance semantic segmentation using very deep fully convolutional networks

Zifeng Wu, Chunhua Shen, and Anton van den Hengel. High-performance semantic segmentation using very deep fully convolutional networks. arXiv:1604.04339, 2016

2016 arXiv
[108]

Context encoders: Feature learning by inpainting

Deepak Pathak, Philipp Krähenbühl, Jeff Donahue, Trevor Darrell, and Alexei Efros. Context encoders: Feature learning by inpainting. CVPR, 2016. 119

2016
[109]

Unsupervised learning of visual representations by solving jigsaw puzzles

Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. ECCV, 2016

2016
[110]

Colorful image colorization

Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. ECCV, 2016

2016
[111]

Colorization as a proxy task for visual understanding

Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Colorization as a proxy task for visual understanding. CVPR, 2017

2017
[112]

Split-brain autoencoders: Unsuper- vised learning by cross-channel prediction

Richard Zhang, Phillip Isola, and Alexei A Efros. Split-brain autoencoders: Unsuper- vised learning by cross-channel prediction. CVPR, 2017

2017
[113]

Adversarial feature learning

Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning. ICLR, 2017

2017
[114]

Very deep convolutional networks for large- scale image recognition

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large- scale image recognition. ICLR, 2015

2015
[115]

DeCAF: A deep convolutional activation feature for generic visual recognition

Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. DeCAF: A deep convolutional activation feature for generic visual recognition. ICML, 2014

2014
[116]

Generative adversarial nets

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NIPS, 2014

2014
[117]

Deep learning from temporal coherence in video

Hossein Mobahi, Ronan Collobert, and Jason Weston. Deep learning from temporal coherence in video. ICML, 2009

2009
[118]

Slow and steady feature analysis: Higher order temporal coherence in video

Dinesh Jayaraman and Kristen Grauman. Slow and steady feature analysis: Higher order temporal coherence in video. CVPR, 2016. 120

2016
[119]

Learning visual groups from co-occurrences in space and time

Phillip Isola, Daniel Zoran, Dilip Krishnan, and Edward H Adelson. Learning visual groups from co-occurrences in space and time. ICLR, workshop paper, 2016

2016
[120]

Unsupervised learning using sequential veriﬁcation for action recognition

Ishan Misra, C Lawrence Zitnick, and Martial Hebert. Unsupervised learning using sequential veriﬁcation for action recognition. ECCV, 2016

2016
[121]

Unsupervised learning of visual representations using videos

Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. ICCV, 2015

2015
[122]

Unsupervised learning of video representations using LSTMs

Nitish Srivastava, Elman Mansimov, and Ruslan Salakhutdinov. Unsupervised learning of video representations using LSTMs. ICML, 2015

2015
[123]

Learning features by watching objects move

Deepak Pathak, Ross Girshick, Piotr Dollár, Trevor Darrell, and Bharath Hariharan. Learning features by watching objects move. CVPR, 2017

2017
[124]

McDermott, William T

Andrew Owens, Jiajun Wu, Josh H. McDermott, William T. Freeman, and Antonio Torralba. Ambient sound provides supervision for visual learning. ECCV, 2016

2016
[125]

Learn- ing to poke by poking: Experiential learning of intuitive physics

Pulkit Agrawal, Ashvin Nair, Pieter Abbeel, Jitendra Malik, and Sergey Levine. Learn- ing to poke by poking: Experiential learning of intuitive physics. NIPS, 2016

2016
[126]

Combining self-supervised learning and imitation for vision-based rope manipulation

Ashvin Nair, Dian Chen, Pulkit Agrawal, Phillip Isola, Pieter Abbeel, Jitendra Malik, and Sergey Levine. Combining self-supervised learning and imitation for vision-based rope manipulation. ICRA, 2017

2017
[127]

Augmenting supervised neural networks with unsupervised objectives for large-scale image classiﬁcation

Yuting Zhang, Kibok Lee, and Honglak Lee. Augmenting supervised neural networks with unsupervised objectives for large-scale image classiﬁcation. ICML, 2016

2016
[128]

Top-down learning for structured labeling with convolutional pseudoprior

Saining Xie, Xun Huang, and Zhuowen Tu. Top-down learning for structured labeling with convolutional pseudoprior. ECCV, 2016

2016
[129]

Contextual priming for object detection

Antonio Torralba. Contextual priming for object detection. IJCV, 2003. 121

2003
[130]

Discriminative models for multi-class object layout

Chaitanya Desai, Deva Ramanan, and Charless Fowlkes. Discriminative models for multi-class object layout. IJCV, 2011

2011
[131]

Sudderth, Antonio Torralba, William T

Erik B. Sudderth, Antonio Torralba, William T. Freeman, and Alan S. Willsky. Learning hierarchical models of scenes, objects, and parts. ICCV, 2005

2005
[132]

Poselets: Body part detectors trained using 3d human pose annotations

Lubomir Bourdev and Jitendra Malik. Poselets: Body part detectors trained using 3d human pose annotations. ICCV, 2009

2009
[133]

Detecting people using mutually consistent poselet activations

Lubomir Bourdev, Subhransu Maji, Thomas Brox, and Jitendra Malik. Detecting people using mutually consistent poselet activations. ECCV, 2010

2010
[134]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classiﬁcation with deep convolutional neural networks. NIPS, 2012

2012
[135]

U-Net: Convolutional networks for biomedical image segmentation

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. MICCAI, 2015

2015
[136]

SegNet: A deep convolu- tional encoder-decoder architecture for image segmentation

Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. SegNet: A deep convolu- tional encoder-decoder architecture for image segmentation. PAMI, 2017

2017
[137]

Feed- forward semantic segmentation with zoom-out features

Mohammadreza Mostajabi, Payman Yadollahpour, and Gregory Shakhnarovich. Feed- forward semantic segmentation with zoom-out features. CVPR, 2015

2015
[138]

Tsung-Wei Ke, Michael Maire, and Stella X. Yu. Multigrid neural architectures.CVPR, 2017

2017
[139]

Indoor segmenta- tion and support inference from rgbd images

Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmenta- tion and support inference from rgbd images. In Proceedings of the 12th European Conference on Computer Vision - Volume Part V, ECCV’12, pages 746–760, Berlin, Heidelberg, 2012. ISBN 978-3-642-33714-7. 122

2012
[140]

Cohen, B

Peng Wang, Xiaohui Shen, Zhe Lin, S. Cohen, B. Price, and A. Yuille. Towards uniﬁed depth and semantic prediction from a single image. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2800–2809, June 2015

2015
[141]

Look deeper into depth: Monocular depth estimation with semantic booster and attention-driven loss

Jianbo Jiao, Ying Cao, Yibing Song, and Rynson Lau. Look deeper into depth: Monocular depth estimation with semantic booster and attention-driven loss. In The European Conference on Computer Vision (ECCV), September 2018

2018
[142]

Joint semantic segmentation and depth estimation with deep convolutional networks

Arsalan Mousavian, Hamed Pirsiavash, and Jana Kosecka. Joint semantic segmentation and depth estimation with deep convolutional networks. In 3DV, 2016

2016
[143]

Casas, and Montse PardÃ s

Xiao Lin, Dalila SÃ¡nchez-Escobedo, Josep R. Casas, and Montse PardÃ s. Depth esti- mation and semantic segmentation from a single rgb image using a hybrid convolutional neural network. Sensors, 19(8), 2019. ISSN 1424-8220

2019
[144]

Zamir, Alexander Sax, William Shen, Leonidas J

Amir R. Zamir, Alexander Sax, William Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. 123

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Pushmeet Kohli, L’ubor Ladicky, and Philip H. S. Torr. Robust higher order potentials for enforcing label consistency. In CVPR, 2008. ISBN 978-1-4244-2242-5. doi: http://doi.ieeecomputersociety.org/10.1109/CVPR.2008.4587417

arXiv 2008

[2] [2]

Shotton, J

J.D.J. Shotton, J. Winn, C. Rother, and A. Criminisi. Textonboost for image under- standing: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV, 81(1), 2009

2009

[3] [3]

Efﬁcient inference in fully connected crfs with gaussian edge potentials

Philipp Krähenbühl and Vladlen Koltun. Efﬁcient inference in fully connected crfs with gaussian edge potentials. Adv. Neural Inf. Process. Syst, 2011

2011

[4] [4]

Ladick`y, C

L. Ladick`y, C. Russell, P. Kohli, and P. H. S. Torr. Associative hierarchical CRFs for object class image segmentation. ICCV, 2009

2009

[5] [5]

Pylon model for semantic segmentation

Victor Lempitsky, Andrea Vedaldi, and Andrew Zisserman. Pylon model for semantic segmentation. In NIPS, pages 1485–1493, 2011

2011

[6] [6]

Discriminative re-ranking of diverse segmentations

Payman Yadollahpour, Dhruv Batra, and Gregory Shakhnarovich. Discriminative re-ranking of diverse segmentations. In CVPR, 2013

2013

[7] [7]

Diverse M-Best Solutions in Markov Random Fields

Dhruv Batra, Payman Yadollahpour, Abner Guzman-Rivera, and Greg Shakhnarovich. Diverse M-Best Solutions in Markov Random Fields. In ECCV, 2012. 108

2012

[8] [8]

Gonfaus, Joost van de Weijer, Andrew D

Xavier Boix, Josep M. Gonfaus, Joost van de Weijer, Andrew D. Bagdanov, Joan Serrat Gual, and Jordi Gonzàlez. Harmony potentials - fusing global and local scale for semantic image segmentation. IJCV, 96(1):83–102, 2012

2012