REVIEW 3 major objections 5 minor 50 references
Where Is My Mirror?
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read MirrorNet segments mirrors in ordinary photos by learning the contrast between a mirror and its surroundings.
desk verdict First mirror segmentation dataset and network, with a real but fixable comparison confound in the main evaluation table. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Contextual Contrasted Feature Extraction (CCFE) module. Each CCFE block computes $f_{\mathrm{local}}(F, \Theta_{\mathrm{local}}) - f_{\mathrm{context}}(F, \Theta_{\mathrm{context}})$, subtracting features from a standard $3\times3$ convolution (local content) from features of dilated $3\times3$ convolutions with rates $2$, $4$, $8$, $16$ (surrounding context), so the difference highlights dividing boundaries rather than the mirror content itself. Four such blocks are chained to enlarge receptive fields, their multi-scale outputs are fused through an attention module, and the resulting mirror maps are used as attention maps that suppress non-mirror features in shallower layers, refining the segmentation coarse-to-fine. Training uses the Lovász-hinge loss as a surrogate for IoU, and a fully connected CRF post-processes the final output.
What would settle it
Test MirrorNet on a set of mirrors whose reflected content is deliberately matched to the surrounding wall in colour, texture, and layout, with no semantic break at the frame; if IoU drops to near chance, the contextual-contrast cue has vanished and the method's general claim fails, a direction the paper's own failure cases already hint at.
Extended reading notes
Core claim
The paper's central claim is that mirror segmentation, previously handled only by manual labelling or hardware tags in 3D scanning, can be treated as an automatic binary labelling problem in ordinary RGB images. The load-bearing observation is that although a mirror reflects content similar to its surroundings, there is nonetheless a discontinuity at the mirror boundary that combines low-level colour/texture change with high-level semantic change, and MirrorNet is built to detect exactly this contextual contrast. The authors construct MSD, with 4,018 image/mask pairs split into 3,063 training and 955 test images, and train MirrorNet end-to-end with a Lovász-hinge loss and deep supervision. In their comparison on the MSD test set, MirrorNet achieves IoU 78.95, pixel accuracy 0.933, F-measure 0.857, MAE 0.065, and BER 6.39, improving on all compared baselines; qualitatively it segments mirrors at different sizes, handles partial occlusion, and distinguishes mirrors from paintings, windows, and doors.
Load-bearing premise
The whole method rests on the premise that a mirror's region is set off from its surroundings by a learnable contrast in colour, texture, or semantics; when that contrast is too weak, the cue vanishes and the network has nothing to detect.
Editorial extensions
If this is right
- A depth estimator can be rescued by first masking out mirrors and interpolating depth from surrounding pixels, fixing the wrong-depth predictions mirrors cause.
- Object detectors and instance segmenters can avoid hallucinating objects inside mirrors by treating the mirror mask as a region to ignore before inference.
- The MSD split provides a public benchmark for future mirror segmentation work, since all compared baselines are retrained on the same training set.
- The contrast-based method extends to mirror-like surfaces such as paintings and windows, exploiting the fact that mirrors reflect content semantically consistent with the rest of the scene.
Reading between the lines
- Inference: MirrorNet could be inserted as a preprocessing module in SLAM or visual odometry pipelines to remove mirror-produced phantom geometry, but the paper only demonstrates depth interpolation, not full system integration.
- Inference: The strong indoor bias of MSD (3,677 indoor versus 341 outdoor images) means the reported numbers may not transfer to outdoor glass facades, which the paper explicitly leaves to future work.
- Inference: A natural extension is to combine contextual contrast with geometric cues such as multi-view consistency or depth edges, which would address the low-contrast failure cases the authors acknowledge.
- Inference: The same local-minus-context formulation could be applied to other reflective or transparent surfaces, such as glass windows, where semantic consistency with the surroundings is preserved.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the first large-scale mirror segmentation dataset (MSD) with 4,018 manually annotated images and proposes MirrorNet, an encoder-decoder network whose Contextual Contrasted Feature Extraction (CCFE) module compares local features with dilated-context features to locate mirror boundaries. The method is trained with a Lovász-hinge loss and deep supervision, and the paper reports large margins over nine baselines on the MSD test set across five metrics, plus qualitative results on ADE20K and internet images.
Significance. If the reported results are taken at face value, this is a useful first dataset and a credible baseline for a previously unaddressed vision task. The paper explicitly credits prior work, the dataset construction is described in reasonable detail, and the component analysis in Table 2 gives some evidence that the CCFE module is beneficial. However, the headline comparison in Table 1 is weakened by a likely backbone-capacity confound and by asymmetric use of CRF post-processing, and the ablation results are reported without any measure of variance. The central claim that MirrorNet outperforms state-of-the-art methods therefore needs additional, controlled experiments before it can be considered established.
major comments (3)
- [§4.4, §5.1, Table 1] The comparison in Table 1 is confounded by backbone choice. Section 4.4 states that MirrorNet uses a pre-trained ResNeXt101 as its feature extraction network, while Section 5.1 says only that the baselines are run with their publicly available codes, which in practice use VGG16 or ResNet50/101 backbones. The paper does not re-backbone any baseline to ResNeXt101 and does not report parameter counts or FLOPs. The large margins in Table 1 may therefore be due largely to the stronger backbone rather than to the CCFE module. Please add a matched-capacity comparison, for example by re-backboning the strongest baselines with ResNeXt101 or by evaluating a MirrorNet variant with a VGG16/ResNet101 encoder, and report model sizes.
- [Table 1] CRF post-processing is applied only to R3Net, BDRAR, and MirrorNet, not to the other seven baselines. For MirrorNet itself, CRF changes IoU from 78.46 to 78.95 and MAE from 0.085 to 0.065, so this is not a negligible detail. Applying a generic post-processing step only to the proposed method and two selected baselines makes the comparison unfair. Please either apply the same CRF post-processing to all compared methods or report all results without it.
- [Table 2] The component analysis is reported without error bars or repeated runs. The differences between some configurations are small (e.g., 'basic + CCFE w/o contrasts' at 78.59 IoU versus 'basic + CCFE w/ 1B4C' at 76.36, and 'MirrorNet' at 78.95 versus 'basic + CCFE w/o contrasts' at 78.59), and it is not clear whether these differences are statistically meaningful. Please report means and standard deviations over at least three random seeds, or otherwise justify that a single run is sufficient.
minor comments (5)
- [§5.2, Figure 7] The ADE20K results are presented qualitatively only; no mirror ground truth or quantitative metrics are provided for that dataset, so these examples should be described as illustrative rather than as evidence of generalization.
- [§4.2] The attention module is cited as [37] without specifying the exact variant or how it is adapted to the CCFE module; one sentence describing the module's inputs and outputs would improve reproducibility.
- [§4.4, Eq. (2)] The balancing weights w_s are said to be empirically set to 1; please state whether this was a fixed a priori choice or a value selected on a validation split.
- [§5.1, Table 1] The 'Statistics' baseline is described only in the table caption; a sentence in Section 5.1 explaining exactly how the location-statistics mask is computed would help reproducibility.
- [§1, project page] The paper says the dataset will be made publicly available and gives a project page, but the download link for the dataset is not included in the manuscript; please provide the direct link for reproducibility.
Circularity Check
No significant circularity: the paper is a standard supervised-learning evaluation whose predictions are not derived from its inputs by construction.
full rationale
The paper's central claim is empirical: MirrorNet, trained on the newly collected MSD dataset, segments mirrors better than retrained baselines on a held-out test split. No equation in the paper defines the prediction target in terms of the fitted model or vice versa. Eq. (1) defines a feature-extraction operation (local features minus context features); it is an architectural choice, not a quantity that is fitted from ground truth and then reported as a prediction. The loss in Eq. (2) is a standard surrogate for IoU, and the evaluation metrics in Eqs. (3)-(4) are standard external measures. The MSD dataset is constructed by manual annotation, and the test split is disjoint from the training split by mirror type, so the test numbers are not forced by construction. The baselines are said to be retrained on the same training set, and while the backbone-capacity comparison may be a real methodological concern, that is a correctness/rigor issue rather than circularity. The acknowledged limitation in Section 6 about low contextual contrast is a scope assumption, not a tautology. Self-citations are not load-bearing: the only cited work involving a current author (DeshadowNet, [29]) is background on shadow removal and does not justify the central mirror-segmentation claim. The CCL block from [10] is cited as inspiration and explicitly distinguished from CCFE, but even if that influence were stronger, it would not make the evaluation circular. Overall, the derivation chain is self-contained in the sense required here: no fitted input is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known result is relabeled.
Assumptions & free parameters
free parameters (1)
- Deep supervision loss weights w_s =
1 for all four side outputs
assumptions (3)
- domain assumption Mirror regions exhibit detectable contextual contrast with their surroundings.
- domain assumption Features from an ImageNet-pretrained ResNeXt101 transfer to mirror segmentation.
- domain assumption Manual annotations of mirror regions are correct and consistent.
Cite this review
Pith. "Pith review of Where Is My Mirror?." pith.science (2026). https://pith.science/paper/YRBB2HGJ
@misc{pith2026190809101,
author = {Pith},
title = {Pith review of: Where Is My Mirror?},
year = {2026},
howpublished = {\url{https://pith.science/paper/YRBB2HGJ}},
note = {Machine review of arXiv:1908.09101}
}
read the original abstract
Mirrors are everywhere in our daily lives. Existing computer vision systems do not consider mirrors, and hence may get confused by the reflected content inside a mirror, resulting in a severe performance degradation. However, separating the real content outside a mirror from the reflected content inside it is non-trivial. The key challenge is that mirrors typically reflect contents similar to their surroundings, making it very difficult to differentiate the two. In this paper, we present a novel method to segment mirrors from an input image. To the best of our knowledge, this is the first work to address the mirror segmentation problem with a computational approach. We make the following contributions. First, we construct a large-scale mirror dataset that contains mirror images with corresponding manually annotated masks. This dataset covers a variety of daily life scenes, and will be made publicly available for future research. Second, we propose a novel network, called MirrorNet, for mirror segmentation, by modeling both semantical and low-level color/texture discontinuities between the contents inside and outside of the mirrors. Third, we conduct extensive experiments to evaluate the proposed method, and show that it outperforms the carefully chosen baselines from the state-of-the-art detection and segmentation methods.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
https://pytorch.org/
Pytorch. https://pytorch.org/
-
[2]
Frequency-tuned salient region detec- tion
Radhakrishna Achanta, Sheila Hemami, Francisco Estrada, and Sabine Susstrunk. Frequency-tuned salient region detec- tion. In CVPR, 2009
work page 2009
-
[3]
SegNet: A deep convolutional encoder-decoder architecture for scene segmentation
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. SegNet: A deep convolutional encoder-decoder architecture for scene segmentation. IEEE TPAMI, 2017
work page 2017
-
[4]
Maxim Berman, Amal Rannen Triki, and Matthew Blaschko. The lovsz-softmax loss: A tractable surrogate for the op- timization of the intersection-over-union measure in neural networks. In CVPR, 2018
work page 2018
-
[5]
Matterport3d: Learning from RGB- D data in indoor environments
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Hal- ber, Matthias Niebner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from RGB- D data in indoor environments. In 3DV, 2017
work page 2017
-
[6]
Masklab: Instance segmentation by refining object detection with semantic and direction features
Liang-Chieh Chen, Alexander Hermans, George Papan- dreou, Florian Schroff, Peng Wang, and Hartwig Adam. Masklab: Instance segmentation by refining object detection with semantic and direction features. In CVPR, 2018
work page 2018
-
[7]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Semantic image seg- mentation with deep convolutional nets and fully connected CRFs. In ICLR, 2015
work page 2015
-
[8]
Re- verse attention for salient object detection
Shuhan Chen, Xiuli Tan, Ben Wang, and Xuelong Hu. Re- verse attention for salient object detection. In ECCV, 2018
work page 2018
Show all 50 references
-
[9]
R3net: Recurrent residual refinement network for saliency detection
Zijun Deng, Xiaowei Hu, Lei Zhu, Xuemiao Xu, Jing Qin, Guoqiang Han, and Pheng-Ann Heng. R3net: Recurrent residual refinement network for saliency detection. In IJCAI, 2018
2018
-
[10]
Context contrasted feature and gated multi- scale aggregation for scene segmentation
Henghui Ding, Xudong Jiang, Bing Shuai, Ai Qun Liu, and Gang Wang. Context contrasted feature and gated multi- scale aggregation for scene segmentation. In CVPR, 2018
2018
-
[11]
Salient objects in clut- ter: Bringing salient object detection to the foreground
Deng-Ping Fan, Ming-Ming Cheng, Jiang-Jiang Liu, Shang- Hua Gao, Qibin Hou, and Ali Borji. Salient objects in clut- ter: Bringing salient object detection to the foreground. In ECCV, 2018
2018
-
[12]
Mask R-CNN
Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask R-CNN. In ICCV, 2017
2017
-
[13]
Deeply supervised salient ob- ject detection with short connections
Qibin Hou, Ming-Ming Cheng, Xiaowei Hu, Ali Borji, Zhuowen Tu, and Philip Torr. Deeply supervised salient ob- ject detection with short connections. In CVPR, 2017
2017
-
[14]
Direction-aware spatial context features for shadow detection
Xiaowei Hu, Lei Zhu, Chi-Wing Fu, Jing Qin, and Pheng- Ann Heng. Direction-aware spatial context features for shadow detection. In CVPR, 2018
2018
-
[15]
Efficient inference in fully connected CRFs with Gaussian edge potentials
Philipp Kr ¨ahenb¨uhl and Vladlen Koltun. Efficient inference in fully connected CRFs with Gaussian edge potentials. In NIPS, 2011
2011
-
[16]
A+D Net: Training a shadow detector with adversarial shadow attenuation
Hieu Le, Tomas Yago Vicente, Vu Nguyen, Minh Hoai, and Dimitris Samaras. A+D Net: Training a shadow detector with adversarial shadow attenuation. In ECCV, 2018
2018
-
[17]
Deep saliency with encoded low level distance map and high level features
Gayoung Lee, Yu-Wing Tai, and Junmo Kim. Deep saliency with encoded low level distance map and high level features. In CVPR, 2016
2016
-
[18]
The secrets of salient object segmentation
Yin Li, Xiaodi Hou, Christof Koch, James Rehg, and Alan Yuille. The secrets of salient object segmentation. In CVPR, 2014
2014
-
[19]
Proposal-free network for instance-level object segmentation
Xiaodan Liang, Liang Lin, Yunchao Wei, Xiaohui Shen, Jianchao Yang, and Shuicheng Yan. Proposal-free network for instance-level object segmentation. IEEE TPAMI, 2018
2018
-
[20]
Feature pyramid networks for object detection
Tsung-Yi Lin, Piotr Doll ´ar, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie. Feature pyramid networks for object detection. In CVPR, 2017
2017
-
[21]
Picanet: Learning pixel-wise contextual attention for saliency detec- tion
Nian Liu, Junwei Han, and Ming-Hsuan Yang. Picanet: Learning pixel-wise contextual attention for saliency detec- tion. In CVPR, 2018
2018
-
[22]
Sgn: Sequential grouping networks for instance segmentation
Shu Liu, Jiaya Jia, Sanja Fidler, and Raquel Urtasun. Sgn: Sequential grouping networks for instance segmentation. In ICCV, 2017
2017
-
[23]
Path aggregation network for instance segmentation
Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia. Path aggregation network for instance segmentation. In CVPR, 2018
2018
-
[24]
Parsenet: Looking wider to see better
Wei Liu, Andrew Rabinovich, and Alexander Berg. Parsenet: Looking wider to see better. arXiv:1506.04579, 2015
2015 arXiv
-
[25]
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015
2015
-
[26]
Conditional generative adversarial nets
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv:1411.1784, 2014
2014 arXiv
-
[27]
Shadow detection with condi- tional generative adversarial networks
Vu Nguyen, Tomas Yago Vicente, Maozheng Zhao, Minh Hoai, and Dimitris Samaras. Shadow detection with condi- tional generative adversarial networks. In ICCV, 2017
2017
-
[28]
Apriltag: A robust and flexible visual fiducial system
Edwin Olson. Apriltag: A robust and flexible visual fiducial system. In ICRA, 2011
2011
-
[29]
DeshadowNet: A multi-context em- bedding deep network for shadow removal
Liangqiong Qu, Jiandong Tian, Shengfeng He, Yandong Tang, and Rynson Lau. DeshadowNet: A multi-context em- bedding deep network for shadow removal. In CVPR, 2017
2017
-
[30]
Faster r-cnn: towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: towards real-time object detection with region proposal networks. IEEE TPAMI, 2017
2017
-
[31]
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015
2015
-
[32]
Indoor segmentation and support inference from RGBD images
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from RGBD images. In ECCV, 2012
2012
-
[33]
Large-scale training of shadow de- tectors with noisily-annotated shadow examples
Tom ´as Yago Vicente, Le Hou, Chen-Ping Yu, Minh Hoai, and Dimitris Samaras. Large-scale training of shadow de- tectors with noisily-annotated shadow examples. In ECCV, 2016
2016
-
[34]
Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal
Jifeng Wang, Xiang Li, and Jian Yang. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In CVPR, 2018
2018
-
[35]
Deep networks for saliency detection via local esti- mation and global search
Lijun Wang, Huchuan Lu, Ruan Xiang, and Ming Hsuan Yang. Deep networks for saliency detection via local esti- mation and global search. In CVPR, 2015
2015
-
[36]
Reconstruct- ing scenes with mirror and glass surfaces
Thomas Whelan, Michael Goesele, Steven Lovegrove, Ju- lian Straub, Simon Green, Richard Szeliski, Steven Butter- field, Shobhit Verma, and Richard Newcombe. Reconstruct- ing scenes with mirror and glass surfaces. ACM TOG, 2018
2018
-
[37]
Cbam: Convolutional block attention module
Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In ECCV, 2018
2018
-
[38]
Bridging category-level and instance-level semantic image segmentation
Zifeng Wu, Chunhua Shen, and Anton van den Hengel. Bridging category-level and instance-level semantic image segmentation. arXiv:1605.06885, 2016
2016 arXiv
-
[39]
Aggregated residual transformations for deep neural networks
Saining Xie, Ross Girshick, Piotr Doll ´ar, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In CVPR, 2017
2017
-
[40]
Holistically-nested edge de- tection
Saining Xie and Zhuowen Tu. Holistically-nested edge de- tection. In ICCV, 2015
2015
-
[41]
Saliency detection via graph-based man- ifold ranking
Chuan Yang, Lihe Zhang, Huchuan Lu, Xiang Ruan, and Ming-Hsuan Yang. Saliency detection via graph-based man- ifold ranking. In CVPR, 2013
2013
-
[42]
Multi-scale context aggrega- tion by dilated convolutions
Fisher Yu and Vladlen Koltun. Multi-scale context aggrega- tion by dilated convolutions. arXiv:1511.07122, 2015
2015 arXiv
-
[43]
Con- text encoding for semantic segmentation
Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, and Amit Agrawal. Con- text encoding for semantic segmentation. In CVPR, 2018
2018
-
[44]
Amulet: Aggregating multi-level convolu- tional features for salient object detection
Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, and Ruan Xiang. Amulet: Aggregating multi-level convolu- tional features for salient object detection. In ICCV, 2017
2017
-
[45]
Monocular object instance segmentation and depth ordering with cnns
Ziyu Zhang, Alexander Schwing, Sanja Fidler, and Raquel Urtasun. Monocular object instance segmentation and depth ordering with cnns. In ICCV, 2015
2015
-
[46]
Exfuse: Enhancing feature fusion for semantic segmentation
Zhenli Zhang, Xiangyu Zhang, Chao Peng, Xiangyang Xue, and Jian Sun. Exfuse: Enhancing feature fusion for semantic segmentation. In ECCV, 2018
2018
-
[47]
Icnet for real-time semantic segmentation on high-resolution images
Hengshuang Zhao, Xiaojuan Qi, Xiaoyong Shen, Jianping Shi, and Jiaya Jia. Icnet for real-time semantic segmentation on high-resolution images. In ECCV, 2018
2018
-
[48]
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017
2017
-
[49]
Scene parsing through ade20k dataset
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017
2017
-
[50]
Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection
Lei Zhu, Zijun Deng, Xiaowei Hu, Chi-Wing Fu, Xuemiao Xu, Jing Qin, and Pheng-Ann Heng. Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In ECCV, 2018
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.