Pith. sign in

REVIEW 3 major objections 4 minor 34 references

Compressive Sampling for Array Cameras

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper's DLACS scheme replaces JPEG-style camera-head compression with fixed integer-mask downsampling and off-camera neural reconstruction, claiming a >20x cut in camera-head processing power per pixel.

desk verdict A genuinely new camera-head compression method with reproducible code, but the headline 20x power reduction is not supported by the paper's own 14.7x runtime measurement and no power measurements. read the letter →

arxiv 1908.10903 v1 pith:OHCNYYBK submitted 2019-08-28 eess.IV

Xuefei Yan , David J. Brady , Jianqiang Wang , Chao Huang , Zian Li , Songsong Yan , Di Liu , Zhan Ma This is my paper · ORCID
classification eess.IV
keywords compressivesamplingarraycamerasgigapixelimagingimagesignalprocessingpowerefficiencyintegermasksdeeplearningdecompressionrawBayercompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the size, weight and power of image processing electronics, not optics or sensors, is now the main obstacle to gigapixel-scale array cameras. Its proposed fix, Deep-Learning-Aided Compressive Sampling (DLACS), replaces the conventional camera-head image signal processing (ISP) pipeline with a blind downsampling step: a few low-bit-depth integer masks sum raw Bayer pixels as they are read out, and a neural network off-camera reconstructs the image. Because the camera head no longer runs demosaicing, color transforms, or DCT-based compression like JPEG, the paper estimates and measures that the per-pixel computation drops by more than 20x, and concludes that camera-head power per pixel drops by the same factor. The reconstruction quality at low compression ratios is close to JPEG in the reported tests, and combining DLACS with entropy coding reaches similar SSIM at comparable bit rates.

What carries the argument

The load-bearing object is the low-bit-depth integer mask set used as a strided convolutional kernel. Four masks of size $k_x \times k_y$ are applied with stride equal to the kernel size, so each set of $k_x k_y$ pixels is summed into four measurements with no overlap, in a row-buffer-friendly way. The same masks appear on the decode side as the kernel of a transposed convolution, and the surrounding deep networks are trained end-to-end to minimize mean-squared reconstruction error while the masks themselves are fixed after training and rounded to integers. This makes the camera-head operation blind and arithmetic-light: per pixel, four integer-integer multiplications, four integer-integer additions, and one integer division (reducible to a bit shift) replace JPEG's DCT coefficient analysis, demosaicing, and color conversion.

What would settle it

Measure the actual wall-plug power of a camera head running a conventional ISP (demosaicing plus JPEG encoding) and the same head running DLACS mask accumulation and quantization at equal frame rate and output bit depth on the same hardware and memory subsystem; if the measured power drop is much less than the 20x arithmetic ratio, the central claim is falsified.

Watch

Extended reading notes

Core claim

DLACS is a two-sided coding scheme. On the capture side, the raw Bayer frame is compressed by a strided convolutional layer whose kernel is a set of four low-bit-depth integer masks (the paper demonstrates $[8,8]$, $[16,16]$ and $[32,32]$ masks), giving $n_c$ eight-bit integer arrays per frame and a compression ratio of $n_c/(3 k_x k_y)$ relative to an eight-bit RGB frame. No content analysis is performed; demosaicing, white balance, tone and color adjustments are deferred. On the display side, a transposed-convolution layer preceded and followed by trained convolutional neural networks reconstructs the raw Bayer data. The paper claims this scheme uses at least about 20 times fewer operations per pixel than JPEG, since JPEG's $8\times 8$ DCT requires 64 float multiplications and 64 additions per Y, Cb and Cr pixel while DLACS uses four integer multiply-adds and one quantization step per pixel; a GPU runtime comparison gave about 14x faster encoding per pixel. From this the paper concludes a >20x reduction in camera-head processing power per pixel, because it assumes power scales with operations per pixel.

Load-bearing premise

The argument assumes camera-head processing power is directly proportional to the number of arithmetic operations per pixel, so eliminating 20x of operations removes 20x of power.

Editorial extensions

If this is right

  • Camera-head electronics for array cameras could be reduced to readout buffers that accumulate masked sums, making gigapixel-scale video capture feasible in size, weight and power.
  • Because DLACS produces small eight-bit integer arrays, existing codecs such as entropy coding, JPEG, H.264 and HEVC can operate on the compressed representation at much lower computational cost than on full frames.
  • For array-camera light fields in which most pixels are never displayed, the reconstruction network runs only for the pixels actually viewed, so the extra display-side computation is paid rarely.
  • Longer term, the authors expect moving the same mask accumulation into sensor readout could cut camera-head power per pixel by up to 100x by nearly eliminating the ISP.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: equating runtime ratio to power ratio holds only if power tracks arithmetic operation count; memory bandwidth and data movement often dominate real ISP power, so a wall-power measurement could show a smaller saving than 20x even with the same operation count.
  • Editorial inference: the fixed masks are scene-independent, but a natural extension is to adapt mask size, bit depth, or number of masks to scene statistics or camera calibration, trading a little readout logic for better reconstruction at equal power.
  • Editorial inference: because the reported quality trails JPEG2000 while approaching JPEG, the practical win is strongest where capture-side power is scarce and reconstruction quality at very low bit rate is acceptable, or if newer generative decoders are used to close the gap; a direct comparison with current learned codecs would test this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. DLACS is a coded-downsampling scheme for array cameras: raw-bayer (or RGB) frames are convolved with a small set of learned low-bit-depth integer masks and quantized to 8-bit values; decompression is performed by a deep neural network off the camera head. The authors compare reconstruction quality against JPEG, JPEG2000, and OpenCV down/up sampling at compression ratios from 1/48 to 1/768, report CUDA kernel runtimes on an Nvidia TX1 for the DLACS encoder versus a DCT function, and claim that the resulting reduction in operations per pixel yields a >20x reduction in camera-head processing power. The paper includes public code links, detailed network architectures, and quality tables on raw-bayer Mantis images and Kodak RGB images.

Significance. The idea of moving ISP cost from the camera head to display/cloud processing is attractive for gigapixel array cameras, and the learned-integer-mask formulation is simple and hardware-friendly. The paper provides public code, detailed architecture descriptions, and quantitative quality tables for both raw-bayer and RGB data. However, the headline quantitative claim is not established: no power is measured, the measured runtime ratio is about 14.7x rather than 20x, and the premise that power is directly proportional to arithmetic operations is asserted rather than demonstrated. The contribution is plausible and reproducible, but the paper needs either direct power measurements or substantially corrected claims.

major comments (3)
  1. [Abstract; §3 Conclusion; §2.2, Table 2.6] The Abstract and Conclusion state that DLACS reduces camera-head power by >20x, but the only quantitative experiment in §2.2/Table 2.6 is a GPU compute-time comparison: DCT costs 4323.42 ps/pixel versus 294.48 ps/pixel for the 8×8 DLACS mask set, i.e., about 14.7x, with similar values for the 16×16 and 32×32 mask sets. The paper's own text describes this as 'more than 14 times', not 20x. The theoretical factor of 48 in §2.2 assumes a naive 64-multiply-per-pixel DCT, which is not representative of optimized JPEG or hardware ISP implementations, and the derivation from 48 to 32 to 24 to 'at least≈20x' is not explained. The central >20x power-reduction claim therefore overstates the reported evidence.
  2. [§2; §1; §2.2, Table 2.6] The paper asserts that 'power expended in the camera head ISP process is directly proportional to the number of pixels processed and the number of times each pixel value is accessed and processed' (§2), but this is not measured or referenced. The TX1 test explicitly excludes memory transfer, storage, and display time and reports only CUDA computation time. Real camera-head power includes sensor readout, ADC, memory bandwidth, static leakage, and I/O, none of which are measured. Moreover, the paper's own numbers in §1 (sensor at 100 mW/Mpix and ISP at 200–1000 mW/Mpix) imply that even reducing ISP computation to zero would reduce total camera-head power by at most a factor of roughly 3–11x, from 300–1100 mW/Mpix to 100 mW/Mpix, which contradicts the >20x claim. The power conclusion requires either direct power measurements or a model that covers these components.
  3. [§2.2] The comparison to JPEG is incomplete: the 'at least≈20x simpler than JPEG' statement excludes entropy coding and bit-depth differences and counts only the DCT kernel. A complete JPEG encoder also includes color conversion, quantization, and entropy coding, and the DLACS comparison should either include those costs or restrict the claim to the DCT kernel. In addition, the theoretical ratio (64/4)×3 = 48 assumes DCT on all three YCbCr channels; the paper's own adjusted factors of 32 and 24 for subsampled chroma would give at most a 24x ratio for 4:2:0, not 'approximately 20x', without additional assumptions that are not stated.
minor comments (4)
  1. [Title page and throughout] There are numerous typographical errors, including 'Camputer Laboratory' on the title page, 'refering', 'has has', and 'intraframe' inconsistencies; these should be corrected.
  2. [§2.2] The phrase 'the standards of 4:2:1 and 4:2:0' appears to be a typo for '4:2:2 and 4:2:0'; please correct the chroma-subsampling terminology.
  3. [§2.1, Tables 2.1–2.5] The reconstruction-quality comparisons show that DLACS is generally lower in PSNR/SSIM than JPEG and JPEG2000 at the same compression ratio; this quality trade-off should be stated explicitly in the abstract and conclusion rather than only in the body text.
  4. [§2.1] Please clarify whether the three raw-bayer test images were captured with the same camera and session as the training crops, and whether the decompression network was retrained separately for each mask size; this is relevant for assessing generalization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DLACS is a learned codec benchmarked on held-out images, and the power claim follows from a stated proportionality premise rather than from a fitted input.

full rationale

The derivation chain is self-contained rather than circular. DLACS trains four integer masks and a decompression network by minimizing MSE on [128,128,1] crops of Mantis raw-bayer frames and is then evaluated on three Mantis scenes and on the external Kodak dataset; the test metrics are not used to fit the masks or the network, so the reconstruction results are genuine out-of-sample measurements, not a renamed training objective. The computational-complexity comparison is a benchmark: DLACS uses four integer multiply-adds plus one division per pixel versus a DCT operation, and the runtime ratio on an Nvidia TX1 (approximately 14.7x) is measured, not fitted. The conclusion that camera-head power falls by more than 20x is an extrapolation from the unverified premise that 'power expended in the camera head ISP process is directly proportional to the number of pixels processed and the number of times each pixel value is accessed and processed'; that premise is an external assumption, not a quantity fitted to the power claim, so the power statement is a validity or correctness risk rather than a circular reduction. The citations to the authors' earlier Mantis and AWARE work provide background hardware power data and training-image provenance; they do not supply a uniqueness theorem or an ansatz that is then relabeled as a prediction. Accordingly, no step reduces by construction to its inputs, and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on trained mask values and scaling constants, and on two domain assumptions: that local correlation permits blind downsampling, and that compute count translates directly into power draw. No new physical entities are introduced.

free parameters (3)
  • Learned integer masks (compression kernels) = Not listed in text; shown as images in Fig. 2.3 for [8,8] masks
    The mask values are trained by minimizing MSE between input raw-bayer and decompressed output, then scaled and rounded to integers. They determine both compression quality and the exact integer operations used.
  • scW (mask scaling constant) = Not given; chosen to minimize rounding error
    Chosen so that the float masks times scW match the rounded integer masks with minimum MSE. This constant is fitted to the trained masks.
  • Qscale (quantization constant) = Not given; chosen to minimize quantization MSE on training data
    Scales the compressed array to 8-bit integers; the constant is selected to minimize MSE on training data, so it is fitted.
assumptions (3)
  • domain assumption Image data is locally correlated, so downsampling with less than one measurement per pixel preserves enough information for reconstruction.
    Used in the introduction to justify local blind downsampling over quasi-random global sampling. This is the reason the method can work at all.
  • domain assumption Camera-head ISP power is proportional to the number and bit depth of arithmetic operations per pixel.
    Stated in Section 2 and used to convert operation count and runtime ratios into a power reduction claim. This is the key leap from computation time to electrical power.
  • domain assumption The deep neural network can reconstruct raw-bayer frames from the compressed 8-bit integer arrays with acceptable quality.
    The method depends on the trained decompression network producing useful images; this is demonstrated empirically but only on three test scenes, so it is an assumption about generalization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Compressive Sampling for Array Cameras." pith.science (2026). https://pith.science/paper/OHCNYYBK

@misc{pith2026190810903,
  author       = {Pith},
  title        = {Pith review of: Compressive Sampling for Array Cameras},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OHCNYYBK}},
  note         = {Machine review of arXiv:1908.10903}
}
read the original abstract

While design of high performance lenses and image sensors has long been the focus of camera development, the size, weight and power of image data processing components is currently the primary barrier to radical improvements in camera resolution. Here we show that Deep-Learning- Aided Compressive Sampling (DLACS) can reduce operating power on camera-head electronics by 20x. Traditional compressive sampling has to date been primarily applied in the physical sensor layer, we show here that with aid from deep learning algorithms, compressive sampling offers unique power management advantages in digital layer compression.

Figures

Figures reproduced from arXiv: 1908.10903 by the authors.

Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (3 more)
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p012_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p013_2.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 26 canonical work pages

  1. [1]

    Agustsson, M

    E. Agustsson, M. Tschannen, F. Mentzer, R. Timofte, and L. V an Gool , Gen- erative adversarial networks for extreme learned image compression , arXiv preprint arXiv:1804.02958, (2018)

    arXiv 2018

  2. [2]

    Ballé, D

    J. Ballé, D. Minnen, S. Singh, S. J. Hw ang, and N. Johnston , Variational image compression with a scale hyperprior, arXiv preprint arXiv:1802.01436, (2018)

    arXiv 2018

  3. [3]

    D. J. Brady , Coding for compressive imaging, May 12 2009. US Patent 7,532,772

    work page 2009

  4. [4]

    D. J. Brady, M. E. Gehm, R. A. Stack, D. L. Marks, D. S. Kittle, D. R. Golish, E. M. Vera, and S. D. Feller , Multiscale gigapixel photography, Nature, 486 (2012), pp. 386–389, https://doi.org/10.1038/nature11150

    doi:10.1038/nature11150 2012

  5. [5]

    D. J. Brady and N. Hagen , Multiscale lens design, Optics Express, 17 (2009), pp. 10659– 10674, https://doi.org/10.1364/oe.17.010659

    doi:10.1364/oe.17.010659 2009

  6. [6]

    D. J. Brady, A. Mrozack, K. MacCabe, and P. Llull , Compressive tomography, Ad- vances in Optics and Photonics, 7 (2015), pp. 756–813, https://doi.org/10.1364/aop.7. 000756

    work page doi:10.1364/aop.7 2015

  7. [7]

    D. J. Brady, W. B. Pang, H. Li, Z. Ma, Y. Tao, and X. Cao , Parallel cameras, Optica, 5 (2018), pp. 127–137, https://doi.org/10.1364/optica.5.000127

    work page doi:10.1364/optica.5.000127 2018

  8. [8]

    E. J. Candes and T. Tao , Near-optimal signal recovery from random projections: Universal encoding strategies?, IEEE Transactions on Information Theory, 52 (2006), pp. 5406–5425, https://doi.org/10.1109/TIT.2006.885507

    arXiv 2006

Show all 34 references
  1. [9]

    Chartrand, R

    R. Chartrand, R. G. Baraniuk, Y. C. Eldar, M. A. Figueiredo, and J. Tanner , Introduction to the issue on compressive sensing, IEEE Journal of Selected Topics in Signal Processing, 4 (2010), pp. 241–243

    2010

  2. [10]

    Code and data for CS on Kodak examples, https://github.com/saberbud/Kodak_RGB_CS

  3. [11]

    Code and data for CS on raw-bayer examples, https://github.com/saberbud/Raw_bayer_CS

  4. [12]

    DCT, https://github.com/saberbud/Conv2D_vs_DCT

    Code for convolution vs. DCT, https://github.com/saberbud/Conv2D_vs_DCT

  5. [13]

    D. L. Donoho , Compressed sensing, IEEE Transactions on Information Theory, 52 (2006), pp. 1289–1306, https://doi.org/10.1109/TIT.2006.871582

    2006

  6. [14]

    Duran-F aundez, V

    C. Duran-F aundez, V. Lecuire, and F. Lepage , Tiny block-size coding for energy- efficient image compression and communication in wireless camera sensor networks, Sig- nal Processing: Image Communication, 26 (2011), pp. 466 – 481, https://doi.org/https: //doi.org/10.1016/j.imag...

    2011 doi

  7. [15]

    M. Z. C. Iole Moccagatta , Computational complexity evaluation of jpeg 2000, Proc.SPIE, 4115 (2000), pp. 4115 – 4115 – 14, https://doi.org/10.1117/12.411572, https://doi.org/10. 1117/12.411572

    2000 doi

  8. [16]

    ISO, JPEG and JPEG2000, https://www.iso.org/ics/35.040.30/x/p/1/u/0/w/0/d/0

  9. [17]

    ITU-T, Series h: Audiovisual and multimedia systems, infrastructure of audiovisual services– coding of moving video, high efficiency video coding, h.264 (04/13) itu-t, recommendation h, https://www.itu.int/ITU-T/recommendations/rec.aspx?id=13189

  10. [18]

    ITU-T, Series h: Audiovisual and multimedia systems, infrastructure of audiovisual services– coding of moving video, high efficiency video coding, h.265, recommendation h, https: //www.itu.int/ITU-T/recommendations/rec.aspx?rec=11885

  11. [19]

    K. H. Jin, M. T. McCann, E. Froustey, and M. Unser , Deep convolutional neural network for inverse problems in imaging, IEEE Transactions on Image Processing, 26 (2017), pp. 4509–4522

    2017

  12. [20]

    Kulkarni, S

    K. Kulkarni, S. Lohit, P. Turaga, R. Ker viche, and A. Ashok , Reconnet: Non-iterative reconstruction of images from compressively sensed measurements, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 449–458

    2016

  13. [21]

    Kutyniok , Compressed Sensing: Theory and Applications, Cambridge University Press, 2012, https://doi.org/10.1017/CBO9780511794308

    G. Kutyniok , Compressed Sensing: Theory and Applications, Cambridge University Press, 2012, https://doi.org/10.1017/CBO9780511794308

    2012 doi

  14. [22]

    G. Lu, W. Ouyang, D. Xu, X. Zhang, C. Cai, and Z. Gao , Dvc: An end-to-end deep video compression framework, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 11006–11015

    2019

  15. [23]

    Lucas, M

    A. Lucas, M. Iliadis, R. Molina, and A. K. Katsaggelos , Using deep neural networks for inverse problems in imaging: beyond analytical methods, IEEESignalProcessingMagazine, 16 35 (2018), pp. 20–36

    2018

  16. [24]

    Lustig, D

    M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly , Compressed sensing mri, IEEE signal processing magazine, 25 (2008), p. 72

    2008

  17. [25]

    Mentzer, E

    F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. V. Gool , Practical full resolution learned lossless image compression, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 10629–10638

    2019

  18. [26]

    Minnen, J

    D. Minnen, J. Ballé, and G. D. Toderici , Joint autoregressive and hierarchical priors for learned image compression, in Advances in Neural Information Processing Systems, 2018, pp. 10771–10780

    2018

  19. [27]

    Mohan, P

    S. Mohan, P. P. Nikos, and J. B. Da vid , Compressive video sensors using multichannel imagers, Appl. Opt., 49 (2010), pp. B9–B17, https://doi.org/10.1364/AO.49.0000B9, http: //ao.osa.org/abstract.cfm?URI=ao-49-10-B9

    2010 doi

  20. [28]

    Mousa vi and R

    A. Mousa vi and R. G. Baraniuk , Learning to invert: Signal recovery via deep convolu- tional networks, in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2017, pp. 2272–2276

    2017

  21. [29]

    J. M. Nichols, K. P. Judd, C. C. Olson, K. Nov ak, J. R. W aterman, S. Feller, S. McCain, J. Anderson, and D. Brady , Range performance of the DARPA AWARE wide field-of-view visible imager, Appl. Opt., 55 (2016), pp. 4478–4484

    2016

  22. [30]

    N. P. Pitsianis, D. J. Brady, A. Portnoy, X. Sun, T. Suleski, M. A. Fiddy, M. R. Feldman, and R. D. TeKolste , Compressive imaging sensors, vol. 6232, Proceedings of SPIE, 2006

    2006

  23. [31]

    Robucci, L

    R. Robucci, L. K. Chiu, J. Gray, J. Romberg, P. Hasler, and D. Anderson , Com- pressive sensing on a cmos separable transform image sensor, in 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2008, pp. 5125–5128

    2008

  24. [32]

    Santa-Cruz, R

    D. Santa-Cruz, R. Grosbois, and T. Ebrahimi , Jpeg 2000 performance evaluation and as- sessment, Signal Processing: Image Communication, 17 (2002), pp. 113 – 130, https:// doi.org/https://doi.org/10.1016/S0923-5965(01)00025-X, http://www.sciencedirect.com/ science/article/pii/...

    2002 doi

  25. [33]

    G. K. W allace , The jpeg still picture compression standard, Commun. ACM, 34 (1991), pp. 30–44, https://doi.org/10.1145/103085.103089, http://doi.acm.org/10.1145/103085. 103089

    1991

  26. [34]

    X. Xie, Y. W ang, G. Shi, C. W ang, J. Du, and X. Han , Adaptive measurement network for cs image reconstruction, in CCF Chinese Conference on Computer Vision, Springer, 2017, pp. 407–417. 17

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.