REVIEW 3 major objections 4 minor 53 references
DSIC: Deep Stereo Image Compression
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A stereo pair can be compressed as one joint code, with the second image costing 30–50% fewer bits at low bitrates and up to 29% less joint bitrate, at equal or better quality.
desk verdict First credible end-to-end learned stereo image codec with a genuinely novel architecture, but the headline bitrate savings are partly a comparison artifact from the entropy-model upgrade and the second-image numbers are inferred, not directly measured. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the parametric skip function combined with a conditional entropy model. The skip function takes the previous feature maps from both images plus a global context read out of the first image's code, forms a $C \times H \times W$ cost volume normalized by softmax along the disparity axis ($C$ is the hand-set maximum disparity: 32 on NorthAmerica, 64 on Cityscapes), and produces the aligned feature via $g_{2,i} = \sum_{d=0}^{C} c_{d,i}\,h_{1,(i,d)}$, turning disparity estimation into a differentiable attention-like weighting rather than a single hard shift. The conditional entropy model then codes the second quantized latent against the first latent and a hyperprior using a Gaussian-mixture density, capturing the residual correlation that warping leaves behind. Together these two mechanisms carry the claimed bitrate savings.
What would settle it
Run the same architecture on rectified pairs with known ground-truth disparity while raising the fraction of pixels whose disparity exceeds $C$ (for example by placing objects very close to the camera), and compare joint bitrate against the same model with $C$ doubled or against a version with $C=0$; if the savings do not drop when true disparities exceed $C$, the disparity range is not the limiting assumption. A simpler variant: synthetically shift one image by a constant offset larger than $C$ and check whether the second-image bitrate returns to single-image levels.
Extended reading notes
Core claim
The central discovery is that inter-image redundancy in a stereo pair can be exploited end-to-end by two coupled mechanisms. First, a parametric skip function predicts, at each encoder/decoder level, a soft cost volume over a fixed disparity range and uses it to densely warp the first image's feature map toward the second: the warped feature at pixel $i$ is a weighted sum over disparities $d=0..C$ of the first image's features. Second, a conditional entropy model factors the joint bitstream as $p(\bar{y}_2|\bar{y}_1,\bar{z}_2)\,p(\bar{y}_1|\bar{z}_1)\,p(\bar{z}_2)\,p(\bar{z}_1)$, so the second image's code is coded relative to the first. The paper argues that this joint model lowers the total bitrate below independent coding while keeping or improving reconstruction quality, and supports it with comparisons against image codecs, video codecs, and a stereo residual-coding baseline.
Load-bearing premise
The bitrate savings rest on the assumption that the two images are rectified and that every transferable pixel's true disparity lies within the preset range $0..C$ (32 or 64 pixels); where that fails, the skip functions cannot pull the needed information across and the savings shrink toward single-image coding.
Editorial extensions
If this is right
- Storing a stereo pair jointly can roughly double the per-image savings relative to independent compression, so the meaningful headline number is the residual bitrate reduction for the second image.
- At low bitrates the second image costs 30–50% fewer bits than single-image coding, which makes the method most attractive for bandwidth-limited applications; at high bitrates the advantage narrows to 10–20%.
- The model needs no ground-truth disparity supervision: the cost volumes emerge from the rate-distortion objective, so the same architecture can be trained on any rectified stereo corpus.
- In ablations, disparity warping alone helps most at low bitrates, while the conditional entropy model helps across all bitrates; combining them beats either alone, though some of the gain is shared.
- Against MS-SSIM the joint model surpasses conventional codecs and motion-compensation-plus-residual baselines; on Cityscapes PSNR, BPG and MV-HEVC remain competitive, so the advantage is clearest on perceptual structure rather than raw MSE.
Reading between the lines
- The fixed disparity range $C$ is the method's implicit geometry prior; on scenes with large close-up disparities (e.g., pedestrians right beside the camera) the savings should shrink, and a systematic study of bitrate savings vs. baseline width or scene depth would map that boundary.
- The same machinery could be lifted to video by replacing the disparity axis with optical flow or time-varying displacement, since the cost volume and conditional entropy do not depend on the two frames being from a stereo camera.
- Because warp-based transfer and conditional entropy both consume the same inter-image correlation, their partial redundancy (which the paper observes as cannibalization) suggests a cleaner decomposition into shared vs. unique information could yield further gains.
- The paper reports a large compute cost (roughly four times the FLOPs of the single-image model, and 141 GFLOPS at 480x300), so a practical deployment would need sparse cost volumes or learned disparity priors; testing such approximations would show how much of the bitrate gain survives.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DSIC, a deep stereo image compression architecture that compresses a rectified stereo pair jointly. The first image is encoded with a single-image autoencoder; the second image stream receives multi-level, disparity-warped features from the first image through parametric skip functions, and a conditional entropy model (a Gaussian mixture model) models the second code given the first. The whole system is trained end-to-end with a rate-distortion loss. Experiments on a proprietary NorthAmerica driving dataset and on Cityscapes compare against Ballé's hyperprior model, BPG, JPEG2000, HEVC/MV-HEVC, and a stereo residual-coding baseline, reporting joint bitrate savings, inferred second-image savings, and ablations over skip functions, conditional entropy, and hyperpriors.
Significance. If the empirical claims hold, the paper would be an early demonstration that learned multi-image compression can exploit cross-view redundancy beyond single-image codecs, with real coded bitrates verified within 1-2% of the entropy estimate (Appendix D). The architecture is clearly described, the end-to-end training objective is standard, and the ablation study attempts to separate the contributions of the skip functions and the conditional entropy model. The main significance depends on whether the comparison against the single-image baseline isolates the stereo-specific gain, which is currently not established.
major comments (3)
- [Section 3.3, Eq. (14), Appendix C.3, Section 4.3] The comparison to Ballé's hyperprior baseline does not isolate the stereo-specific gain from an entropy-model capacity upgrade. In DSIC, both image codes are modeled with a Gaussian mixture model (Eqs. (12)-(14)), while the Ballé baseline uses a single Gaussian conditional density; a mixture distribution strictly has lower cross-entropy for the same latent codes. Therefore part of the reported 29% joint-bitrate reduction and the inferred 58% residual-bitrate reduction may come from the GMM entropy model for the first image, not from cross-image coding. The ablation in Fig. 9 does not include a control where the first image uses the same GMM-based hyperprior entropy model but no skip functions and no conditioning of the second code on the first. Please add such a control and report the rate-distortion curve against the full stereo model.
- [Section 4.3, Appendix A] The headline 'second image bitrate' savings are not directly measured. The paper reports joint bitrate savings and then assumes 'our stereo pair is compressed as well under our model as under a single-image setting' to double the joint savings into residual savings. This assumption is not validated and is in tension with Appendix A, which states that the first code may contain additional information to help compress the second code. The first image may therefore carry extra bits relative to a pure single-image code, and the second image may carry fewer, so the decomposition 'second-image savings = 2 × joint savings' is an extrapolation, not an observed quantity. Please report actual per-image bitrates for both images under the stereo model and under a matched single-image model, or revise the abstract and conclusion to state joint-bitrate savings without the inferred residual percentages.
- [Eq. (3), Section 4.2] The approach relies on a rectified-stereo assumption and a hand-fixed maximum disparity C (32 for NorthAmerica, 64 for Cityscapes, Section 4.2). If a pixel's true disparity exceeds C or the input is not rectified, the warp in Eq. (3) cannot transfer the needed information and the claimed savings degrade. The paper does not report any sensitivity analysis to C, nor does it state the range of disparities present in the test sets. Please add an experiment that varies C for a fixed model (or reports the fraction of disparity estimates at the boundary of the range) to bound the regime in which the claims hold.
minor comments (4)
- [Section 4.3] The sentence 'At a higher Ballé bitrate 0.734, our model achieves a 17.8% reduction in the residual' appears inconsistent with the preceding sentence, which reports joint-bitrate reductions; please clarify whether 17.8% is the joint or the residual reduction.
- [Section 5] There are typos: 'parameteric' should be 'parametric', and 'demostrate' should be 'demonstrate'.
- [Eq. (3)] The notation '(i,d) represents the pixel index that is d pixels right of pixel i' is ambiguous about the direction convention for disparity on rectified stereo pairs; please specify the sign convention for d.
- [Table 1] Table 1 varies resolution and camera baseline simultaneously, so the individual effect of baseline width cannot be separated from resolution; please state the intended confound or provide a version that holds resolution fixed.
Circularity Check
No significant circularity: the stereo bitrate claims are measured with real range-coded bitstreams against an external single-image baseline, and the GMM-vs-Gaussian entropy-model mismatch is a baseline-fairness issue, not a circular reduction.
full rationale
Walking the claimed derivation chain, the stereo codec is trained end-to-end on the rate-distortion objective of Eq. (15), where the rate is the cross-entropy under the learned conditional entropy model of Eqs. (5)-(14). This is standard learned-compression practice, and the paper does not stop at the training objective: Appendix D reports that its range coder encodes within 1-2% of the Shannon entropy bound, so the reported bits are actual coded bitstream lengths rather than fitted values relabeled as predictions. The central comparison in Section 4.3 is against Ballé et al. [5], an external single-image baseline, and the ablation studies in Figs. 5 and 9 isolate the contributions of DispSkip, CE0, and the hyperprior rather than asserting the stereo gain directly from the baseline. The only author-affiliated citation is [28], used as background on stereo matching, and it bears no weight in the rate-distortion derivation; no uniqueness theorem or ansatz is imported from the authors' prior work. The Section 4.3 statement that residual savings are double the joint savings rests on a stated assumption about the first-image bitrate; if the GMM entropy model lowers the first-image rate relative to Ballé's single-Gaussian hyperprior, the stereo-specific saving could be overstated, but that is a benchmark-fairness confound, not a circular argument. No equation in the paper reduces to its own input, and no fitted parameter is renamed as an independent prediction.
Assumptions & free parameters
free parameters (2)
- max_disparity C =
32 (NorthAmerica), 64 (Cityscapes)
- rate-distortion weight beta =
0.5 to 0.001 for lower to higher bitrate models
assumptions (3)
- domain assumption The joint distribution factorizes as p(z1)p(z2)p(y1|z1)p(y2|y1,z2) (Eq. 5), i.e., y2 is conditionally independent of z1 given y1, and z1,z2 are independent.
- domain assumption Input images are rectified stereo pairs with all correspondences within disparity range [0,C] (Eq. 3).
- domain assumption Each latent pixel's conditional distribution is a Gaussian mixture (Eq. 13) with parameters predicted by a neural network.
Cite this review
Pith. "Pith review of DSIC: Deep Stereo Image Compression." pith.science (2026). https://pith.science/paper/TPSJQHXA
@misc{pith2026190803631,
author = {Pith},
title = {Pith review of: DSIC: Deep Stereo Image Compression},
year = {2026},
howpublished = {\url{https://pith.science/paper/TPSJQHXA}},
note = {Machine review of arXiv:1908.03631}
}
read the original abstract
In this paper we tackle the problem of stereo image compression, and leverage the fact that the two images have overlapping fields of view to further compress the representations. Our approach leverages state-of-the-art single-image compression autoencoders and enhances the compression with novel parametric skip functions to feed fully differentiable, disparity-warped features at all levels to the encoder/decoder of the second image. Moreover, we model the probabilistic dependence between the image codes using a conditional entropy model. Our experiments show an impressive 30 - 50% reduction in the second image bitrate at low bitrates compared to deep single-image compression, and a 10 - 20% reduction at higher bitrates.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
-
[1]
Available at https://github.com/listenlink/3D-HEVC
Mv-hevc implementation. Available at https://github.com/listenlink/3D-HEVC
-
[2]
Soft-to-hard vector quantization for end-to-end learn- ing compressible representations
Eirikur Agustsson, Fabian Mentzer, Michael Tschannen, Lukas Cavigelli, Radu Timofte, Luca Benini, and Luc V Gool. Soft-to-hard vector quantization for end-to-end learn- ing compressible representations. In NeurIPS, 2017
work page 2017
-
[3]
Haluk Aydinoglu and Monson H. Hayes. Stereo image cod- ing: a projection approach. IEEE Transactions on Image Processing, 1998
work page 1998
-
[4]
Johannes Ball, Valero Laparra, and Eero P. Simoncelli. End- to-end optimized image compression. In ICLR, 2017. 4328
work page 2017
-
[5]
Variational image compression with a scale hyperprior
Johannes Ball, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with a scale hyperprior. In ICLR, 2018
work page 2018
-
[6]
Johannes Ball ´e, Valero Laparra, and Eero P. Simoncelli. Density modeling of images using a generalized normaliza- tion transformation. In ArXiv, 2015
work page 2015
-
[7]
PatchMatch: A randomized correspon- dence algorithm for structural image editing
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. PatchMatch: A randomized correspon- dence algorithm for structural image editing. InSIGGRAPH, 2009
work page 2009
-
[8]
Depth discontinuities by pixel-to-pixel stereo
Stan Birchfield and Carlo Tomasi. Depth discontinuities by pixel-to-pixel stereo. In ICCV, 1998
work page 1998
Show all 53 references
-
[9]
Pyramid stereo matching network
Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo matching network. In CVPR, 2018
2018
-
[10]
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016
2016
-
[11]
Multiview video compres- sion
Markus Flierl and Bernd Girod. Multiview video compres- sion. IEEE Signal Processing Magazine, 2007
2007
-
[12]
Motion and disparity compensated coding for multiview video.IEEE Transactions on Circuits and Systems for Video Technology, 2007
Markus Flierl, Aditya Mavlankar, and Bernd Girod. Motion and disparity compensated coding for multiview video.IEEE Transactions on Circuits and Systems for Video Technology, 2007
2007
-
[13]
Deepstereo: Learning to predict new views from the world’s imagery
John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. Deepstereo: Learning to predict new views from the world’s imagery. InCVPR, 2016
2016
-
[14]
Residual image coding for stereo image compression
Tamas Frajka and Kenneth Zeger. Residual image coding for stereo image compression. In ICIP, 2002
2002
-
[15]
Deep probabilistic video compression
Jun Han, Salvator Lombardo, Christopher Schroers, and Stephan Mandt. Deep probabilistic video compression. In ArXiv, 2018
2018
-
[16]
Computer Matching of Areas in Stereo Images
Marsha Jo Hannah. Computer Matching of Areas in Stereo Images. PhD thesis, Stanford, CA, USA, 1974
1974
-
[17]
Stereo processing by semiglobal matching and mutual information
Heiko Hirschmuller. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
2008
-
[18]
Batch normalization: Accelerating deep network training by reducing internal co- variate shift
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In ICML, 2015
2015
-
[19]
Spatial transformer networks
Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks. In NIPS, 2015
2015
-
[20]
Light field image compression using generative adversarial network based view synthesis
Chuanmin Jia, Xinfeng Zhang, Shanshe Wang, Shiqi Wang, Shiliang Pu, and Siwei Ma. Light field image compression using generative adversarial network based view synthesis. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2018
2018
-
[21]
Light field compression using depth image based view syn- thesis
Xiaoran Jiang, Mikael Le Pendu, and Christine Guillemot. Light field compression using depth image based view syn- thesis. In ICMEW, 2017
2017
-
[22]
End-to-end learning of geometry and context for deep stereo regression
Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry. End-to-end learning of geometry and context for deep stereo regression. In ICCV, 2017
2017
-
[23]
Multi-view video coding using view interpolation and reference picture selection
Masaki Kitahara, Hideaki Kimata, Shinya Shimizu, Kazuto Kamikura, Yoshiyuki Yashima, Kenji Yamamoto, Tomohiro Yendo, Toshiaki Fujii, and Masayuki Tanimoto. Multi-view video coding using view interpolation and reference picture selection. ICME, 2006
2006
-
[24]
Context-adaptive entropy model for end-to-end optimized image compression
Jooyoung Lee, Seunghyun Cho, and Seung-Kwon Beack. Context-adaptive entropy model for end-to-end optimized image compression. In ICLR, 2019
2019
-
[25]
Neural Video Compression using Spatio-Temporal Priors
Haojie Liu, Tong Chen, Ming Lu, Qiu Shen, and Zhan Ma. Neural Video Compression using Spatio-Temporal Priors. In ArXiv, 2019
2019
-
[26]
Jean loup Gailly and Mark Adler. zlib. https://www. zlib.net/, 1995–2017
1995
-
[27]
Michael E. Lukacs. Predictive coding of multi-viewpoint im- age sets. In ICASSP, 1986
1986
-
[28]
Schwing, and Raquel Urtasun
Wenjie Luo, Alexander G. Schwing, and Raquel Urtasun. Ef- ficient deep learning for stereo matching. CVPR, 2016
2016
-
[29]
Range encoding: an algorithm for removing redundancy from a digitised message
Gloria Mart ´ın. Range encoding: an algorithm for removing redundancy from a digitised message. In Video and Data Recording Conference, 1979
1979
-
[30]
View synthesis for multiview video compression
Emin Martinian, Alexander Behrens, Jun Xin, and Anthony Vetro. View synthesis for multiview video compression. In Picture Coding Symposium, 2006
2006
-
[31]
Conditional probability models for deep image compression
Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Conditional probability models for deep image compression. In CVPR, 2018
2018
-
[32]
Efficient compression of multi-view video exploiting inter-view dependencies based on h.264/mpeg4- avc
Philipp Merkle, Karsten Muller, Aljoscha Smolic, and Thomas Wiegand. Efficient compression of multi-view video exploiting inter-view dependencies based on h.264/mpeg4- avc. In ICME, 2006
2006
-
[33]
Joint au- toregressive and hierarchical priors for learned image com- pression
David Minnen, Johannes Ball, and George Toderici. Joint au- toregressive and hierarchical priors for learned image com- pression. In NIPS, 2018
2018
-
[34]
Moellenhoff and Mark W
Mark S. Moellenhoff and Mark W. Maier. Transform cod- ing of stereo image residuals. IEEE Transactions on Image Processing, 1998
1998
-
[35]
Real-time adaptive im- age compression
Oren Rippel and Lubomir Bourdev. Real-time adaptive im- age compression. In ICML, 2017
2017
-
[36]
Anderson, and Lubomir Bourdev
Oren Rippel, Sanjay Nair, Carissa Lew, Steve Branson, Alexander G. Anderson, and Lubomir Bourdev. Learned video compression. In ArXiv, 2018
2018
-
[37]
Ryan, R.T
Thomas W. Ryan, R.T. Gray, and Bobby R. Hunt. Prediction of correlation errors in stereo pair images. Optical Engineer- ing, 1980
1980
-
[38]
Learning conditional ran- dom fields for stereo
Daniel Scharstein and Chris Pal. Learning conditional ran- dom fields for stereo. In CVPR, 2007
2007
-
[39]
Claude E. Shannon. A mathematical theory of communica- tion. Bell System Technical Journal, 1948
1948
-
[40]
Mel Siegel, Priyan Gunatilake, Sriram Sethuraman, and An- gel G. Jordan. Compression of stereo image pairs and streams. In Stereoscopic Displays and Virtual Reality Sys- tems, 1994
1994
-
[41]
Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand
Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on Circuits and Systems for Video Technology, 2012. 4329
2012
-
[42]
Stereoscopic coding system, 1997
Katsumi Tahara. Stereoscopic coding system, 1997. US Patent 5,633,682
1997
-
[43]
Lossy image compression with compressive autoen- coders
Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszar. Lossy image compression with compressive autoen- coders. In ICLR, 2017
2017
-
[44]
O’Malley, Sung Jin Hwang, and Damien Vincent
George Toderici, Sean M. O’Malley, Sung Jin Hwang, and Damien Vincent. Variable rate image compression with re- current neural networks. In ICLR, 2016
2016
-
[45]
Full resolution image compression with recurrent neural net- works
George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, and Michele Covell. Full resolution image compression with recurrent neural net- works. In CVPR, 2017
2017
-
[46]
Simoncelli, and Alan C
Zhou Wang, Eero P. Simoncelli, and Alan C. Bovik. Multi- scale structural similarity for image quality assessment. In ACSSC, 2003
2003
-
[47]
Sullivan, Gisle Bjontegaard, and Ajay Luthra
Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra. Overview of the h.264/avc video coding stan- dard. IEEE Transactions on Circuits and Systems for Video Technology, 2003
2003
-
[48]
Stereo image compression with disparity compensation using the mrf model
Antonio Ortega Woontack Woo. Stereo image compression with disparity compensation using the mrf model. 1996
1996
-
[49]
Video compression through image interpolation
Chao-Yuan Wu, Nayan Singhal, and Philipp Krahenbuhl. Video compression through image interpolation. In ECCV, 2018
2018
-
[50]
Group normalization
Yuxin Wu and Kaiming He. Group normalization. In ECCV, 2018
2018
-
[51]
Deep3d: Fully automatic 2d-to-3d video conversion with deep con- volutional neural networks
Junyuan Xie, Ross Girshick, and Ali Farhadi. Deep3d: Fully automatic 2d-to-3d video conversion with deep con- volutional neural networks. In ECCV, 2016
2016
-
[52]
Stereo matching by training a convolutional neural network to compare image patches
Jure Zbontar and Yann LeCun. Stereo matching by training a convolutional neural network to compare image patches. JMLR, 2016
2016
-
[53]
Self- supervised learning for stereo matching with self-improving ability
Yiran Zhong, Yuchao Dai, and Hongdong Li. Self- supervised learning for stereo matching with self-improving ability. In CVPR, 2017. 4330 A. Additional Qualitative Results We showcase additional qualitative results comparing our stereo compression model against other baselines,...
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.