Pith. sign in

REVIEW 4 major objections 6 minor 33 references

End-to-end image compression and reconstruction with ultrahigh speed and ultralow energy enabled by opto-electronic computing processor

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A fully integrated optoelectronic processor compresses and reconstructs images end-to-end at 49.5 ps per pixel and under 10.6 nJ per pixel, 2-3 orders of magnitude better than GPU-based models.

desk verdict Real integrated photonic compressors, but the headline end-to-end latency/energy figures rest on missing methods and are likely misattributed. read the letter →

arxiv 2507.22374 v1 pith:MSXZ6U7V submitted 2025-07-30 physics.optics eess.IV

Yuhang Wang , Ang Li , Yihang Shao , Qiang Li , Yang Zhao , Shilong Pan This is my paper
classification physics.opticseess.IV
keywords opto-electroniccomputingprocessorimagecompressionreconstructionsiliconphotonicsprogrammablephotonicmatrixLiPICO-Netcompressivesensingremoteimagery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single integrated opto-electronic processor can perform the entire image compression and reconstruction pipeline, from raw pixels to reconstructed image, without handing the work to an external computer. The authors build a 32x32 silicon photonic chip with modulators, detectors, and a programmable optical matrix, co-packaged with an FPGA, ADCs, DACs, and TIAs, and they pair it with a lightweight network called LiPICO-Net. They report end-to-end latency of 49.5 ps per pixel and energy below 10.6 nJ per pixel, which they say is two to three orders of magnitude better than ReconNet and DR2-Net running on an Nvidia RTX 4090, while keeping PSNR above 34 dB at 4x compression and above 17 dB at 256x compression. A demonstration on a 130-million-pixel aerial image is offered as evidence that the system can handle practical large-scale workloads where electronic processors would be too slow or too power-hungry.

What carries the argument

The load-bearing object is the 32x32 photonic computing chip (PCC): a silicon photonic matrix core built as an FFT-based mesh of 80 Mach-Zehnder interferometers, monolithically integrated with 32 high-speed modulators and 32 detectors, and co-packaged with TIAs, ADCs, DACs, and an FPGA. Compression is achieved by programming the phase shifters so that submatrices of the optical transmission matrix are partitioned and summed into compressive matrices of varying size, giving adjustable compression ratios. Reconstruction is carried out by LiPICO-Net, a lightweight network that assigns a 256x1024 fully connected layer to the FPGA and three convolutional layers to the photonic core, with the compression matrix and network parameters trained jointly to maximize PSNR.

What would settle it

Measure the end-to-end power and latency of the complete OECP package, with a calibrated power supply on the FPGA, DAC, ADC, and TIA stages and a high-bandwidth oscilloscope at the pixel stream in and out, and compare against the GPU pipeline running the same network. If the true per-pixel energy turns out to be in the microjoule range or the latency is more than a few hundred picoseconds per pixel, the central order-of-magnitude claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that end-to-end image compression and reconstruction can be realized in a fully integrated opto-electronic computing processor (OECP), not merely in simulation or with off-chip processing. The 32x32 photonic computing chip uses a programmable FFT-based mesh of Mach-Zehnder interferometers to multiply image data by trainable compressive matrices, with compression ratios adjustable from 2x to 256x by partitioning and summing submatrices. Reconstruction is done by LiPICO-Net, a network whose fully connected layer runs on the FPGA and whose three convolutional layers run on the photonic matrix; the compressive matrix and network weights are co-trained. The authors report that this system reaches 49.5 ps/pixel latency and 10.58 nJ/pixel energy consumption, orders of magnitude below GPU implementations of ReconNet and DR2-Net, and they validate it on a 130-million-pixel airborne image.

Load-bearing premise

The load-bearing premise is that the reported 49.5 ps/pixel and 10.58 nJ/pixel figures account for the full data path, including the FPGA fully connected layer and all electronic conversions, which the paper delegates to a supplementary section that the preprint does not include.

Editorial extensions

If this is right

  • Compression ratio becomes a runtime parameter: the same 32x32 chip can switch from 2x to 256x compression by reconfiguring the optical matrix, without rebuilding hardware.
  • Total multiply-accumulate cost drops to 1024 MAC/pixel at 4x compression, a 95% or greater reduction versus ReconNet and DR2-Net, which makes on-chip reconstruction feasible on the photonic hardware.
  • Only the fully connected layer needs retraining when the compression matrix changes, so adapting to a new compression ratio is cheap.
  • The 130-million-pixel aerial image demonstration implies that gigapixel-scale remote sensing or surveillance imagery can be compressed and reconstructed in real time where GPU-based models would exceed power or latency budgets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the per-pixel energy figure truly covers the whole data path, then scaling the same architecture to larger photonic matrices would likely push the advantage further, since the optical multiply itself consumes near-zero static power; this is a projection, not a claim of the paper.
  • The design suggests that other linear transforms used in compressive sensing, not just image compression, could be mapped onto the same programmable matrix and co-trained with a lightweight network, though the paper does not test this.
  • A useful benchmark for future work would be to measure the OECP against a GPU running the same co-trained network, not just ReconNet and DR2-Net, to separate hardware gains from network-efficiency gains; the paper does not provide that comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper reports an opto-electronic computing processor (OECP) for end-to-end image compression and reconstruction. A 32x32 silicon photonic chip with modulators, detectors, and a programmable FFT-based interferometer mesh performs compressive measurements, while a lightweight network (LiPICO-Net), split between an FPGA-executed fully connected layer and photonic convolutional layers, reconstructs the image. The authors report adjustable compression ratios from 2x to 256x, PSNR values above 34 dB at 4x and about 17 dB at 256x, a latency of 49.5 ps/pixel, and an energy consumption of 10.58 nJ/pixel, which they compare with GPU implementations of ReconNet and DR2-Net. The validation includes a 130-million-pixel aerial image.

Significance. If the headline numbers are correct, this would be an important demonstration: a monolithic photonic-electronic processor with adjustable compression, integrated control electronics, and a hardware-oriented reconstruction network, with orders-of-magnitude improvement in latency and energy over GPU baselines. The paper has tangible strengths: it reports measured modulator/detector bandwidths, a packaged chip with an FPGA, a concrete MAC/pixel reduction of over 95% versus ReconNet and DR2-Net, and a real ultra-large image test. However, the central performance comparison is an estimate rather than a direct measurement, and the estimation basis is deferred to a missing supplementary section. The significance therefore hinges on whether the stated budget actually includes the full electronic pipeline; until that is clarified, the 2-3 orders-of-magnitude claim should be treated as conditional.

major comments (4)
  1. [Hardware implementation and performance characterization] The headline latency and energy values (49.5 ps/pixel and 10.58 nJ/pixel) are introduced as 'based on the estimation methods for key performance metrics explained in supplementary information S3,' but S3 is not present in the arXiv posting. Because the abstract and conclusion make the 2-3 orders-of-magnitude claim on the basis of these two numbers, the paper as posted does not provide a verifiable derivation. Please include S3 in full, with an itemized budget that accounts for the photonic matrix core, the 256x1024 fully connected layer executed on the FPGA, DAC/ADC conversions, TIA amplification, and FPGA control overhead, and state clearly whether each number is measured or estimated.
  2. [Conclusion] There is a factor-of-1000 inconsistency in the reported energy: the abstract and the Results section state 10.58 nJ/pixel, while the conclusion states 10.58 pJ/pixel. Since the claimed advantage over GPU baselines depends directly on this value, the correct unit and a consistent set of numbers must be provided, along with the relationship to the 1.26 pJ/pixel compression-only figure.
  3. [Results (description of Fig. 3(b))] The sentence 'the mean PSNR based on physical implementation of LiPICO-Net on PCC (but image compression is still in electronics)' appears to say that the 'real measurement' results were obtained with electronic compression and photonic reconstruction. This contradicts the later claim that Fig. 4(f) reports end-to-end compression and reconstruction by the OECP. The manuscript should state unambiguously, for every reported PSNR, which parts of the pipeline were executed on the PCC and which on the FPGA/electronics.
  4. [Results (MAC/pixel calculation)] The reported 'end-to-end' latency must be consistent with the stated architecture. The MAC calculation gives 262,144 MACs per 32x32 block for the FPGA-executed FCL alone (256 MAC/pixel). A total latency of 49.5 ps/pixel would require the FPGA to sustain about 5.2x10^12 MAC/s for this layer before accounting for the three convolutional layers, DAC/ADC, TIA, and optical propagation. Unless S3 documents the FPGA clock rate, MAC parallelism, and pipeline accounting, the quoted 49.5 ps/pixel cannot be taken as an end-to-end number; if it is only the optical matrix-multiplication time, the 'end-to-end' wording in the abstract is overstated.
minor comments (6)
  1. [Results (PSNR definition)] Equations (1) and (2) are garbled in the preprint and need to be typeset properly so that the PSNR and MSE definitions are readable.
  2. [Principle and simulation] The text states that the PCC enables '2x to 1024x ratios,' while the abstract and conclusion claim '2x to 256x'; the maximum compression ratio should be stated consistently and explained in terms of the X/Y submatrix partition.
  3. [Hardware implementation and performance characterization] Table 1 is referenced in the text but its contents do not appear in the posted manuscript; the quantitative comparison table should be included so that the comparison with prior work is complete.
  4. [Results (Fig. 3(b) and Fig. 4(f))] PSNR values are reported as single means without per-image spread or error bars; given the small 11-image test set, a measure of variance would help assess claims such as 'PSNR above 34 dB at 4x' and 'minimum PSNR around 17 dB at 256x.'
  5. [Hardware implementation and performance characterization] The phase-shifter efficiency in Fig. 4(c) is printed with missing symbols ('?? < 3.8??'); the units (presumably mW/pi) should be restored, and the same applies to other corrupted symbols in the figure captions.
  6. [Results (training description)] The phrase 'training of this network is performed by backward propagation of PSNR' is imprecise; PSNR is not directly differentiable, and the actual training loss should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PSNR values are trained-network evaluations on held-out data, and the latency/energy figures are hardware estimates rather than self-referential derivations.

full rationale

I walked the paper's derivation chain. The reconstruction PSNR results come from co-training the compressive matrix and LiPICO-Net on the standard 91-image training set and evaluating on the 11-image test set; this is a fitted learning-based system tested on held-out data, not a parameter-free prediction, and the paper transparently labels it as simulation (AE) and real measurement (RM). The adjustable compression-ratio construction via submatrix partitioning and summation is a constructive definition of how 2x-256x ratios are obtained, not a tautology. The headline latency and energy figures are attributed to 'estimation methods' in supplementary S3, which is absent from the preprint; combined with the conclusion's pJ/nJ inconsistency, this makes the 49.5 ps/pixel and 10.58 nJ/pixel claims unverifiable from the manuscript alone and raises a correctness or scope concern about whether the FPGA-executed 256x1024 fully connected layer, DACs/ADCs, and TIAs are included. However, unverifiability and unit inconsistency are not circularity: no main-text equation reduces to its own input, and no load-bearing premise is justified solely by a self-citation. The GPU comparisons against ReconNet and DR2-Net are external benchmarks. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No ad hoc physical constants or invented entities are introduced. The trained compression matrix and network weights are model parameters, not free assumptions. The central hardware figures (49.5 ps/pixel, 10.58 nJ/pixel) depend on component performance measurements and on the assumption that the full digital and analog pipeline can be pipelined at the stated rate; these are supported only by references to the missing supplementary S3.

assumptions (4)
  • domain assumption The 32x32 FFT-based mesh can be reconfigured to implement any desired compressive matrix with negligible loss and high accuracy.
    Invoked in 'Principle and simulation' and 'Hardware implementation and performance characterization' when compressing with adjustable ratios; the paper states the PCC's transmission matrix can be dynamically reconfigured, but detailed calibration data are not shown.
  • domain assumption The optical matrix multiplication is linear and fast enough that the full data path can achieve 49.5ps/pixel end-to-end.
    The latency estimate in 'Hardware implementation and performance characterization' relies on component bandwidths (30 GHz modulators, 40 GHz detectors) but the pipeline overhead (DAC, TIA, ADC, FPGA) is not accounted for in the main text.
  • domain assumption The FPGA can execute the 256x1024 fully connected layer within the same latency and energy budget.
    LiPICO-Net uses an FPGA-executed FCL; the paper does not provide FPGA timing or power measurements.
  • domain assumption The GPU baseline comparison is fair and reproducible.
    The paper states ReconNet and DR2-Net were run on an RTX 4090, but no measurement details or scripts are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of End-to-end image compression and reconstruction with ultrahigh speed and ultralow energy enabled by opto-electronic computing processor." pith.science (2026). https://pith.science/paper/MSXZ6U7V

@misc{pith2026250722374,
  author       = {Pith},
  title        = {Pith review of: End-to-end image compression and reconstruction with ultrahigh speed and ultralow energy enabled by opto-electronic computing processor},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MSXZ6U7V}},
  note         = {Machine review of arXiv:2507.22374}
}
read the original abstract

The rapid development of AR/VR, remote sensing, satellite radar, and medical equipment has created an imperative demand for ultra efficient image compression and reconstruction that exceed the capabilities of electronic processors. For the first time, we demonstrate an end to end image compression and reconstruction approach using an optoelectronic computing processor,achieving orders of magnitude higher speed and lower energy consumption than electronic counterparts. At its core is a 32X32 silicon photonic computing chip, which monolithically integrates 32 high speed modulators, 32 detectors, and a programmable photonic matrix core, copackaged with all necessary control electronics (TIA, ADC, DAC, FPGA etc.). Leveraging the photonic matrix core programmability, the processor generates trainable compressive matrices, enabling adjustable image compression ratios (from 2X to 256X) to meet diverse application needs. Deploying a custom lightweight photonic integrated circuit oriented network (LiPICO-Net) enables high quality reconstruction of compressed images. Our approach delivers an end to end latency of only 49.5ps/pixel while consuming only less than 10.6nJ/pixel-both metrics representing 2-3 orders of magnitude improvement compared with classical models running on state-of-the-art GPUs. We validate the system on a 130 million-pixel aerial imagery, enabling real time compression where electronic systems falter due to power and latency constraints. This work not only provides a transformative solution for massive image processing but also opens new avenues for photonic computing applications.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 31 canonical work pages

  1. [28]

    Integrated photonic encoder for low power and high-speed image processing,

    X. Wang, et al., "Integrated photonic encoder for low power and high-speed image processing," Nature Communications 15, 4510 (2024)

    work page 2024

  2. [1]

    Multiscale gigapixel photography,

    D. J. Brady, et al., "Multiscale gigapixel photography," Nature 486, 386-389 (2012)

    work page 2012

  3. [2]

    Parallel cameras,

    D. J. Brady, et al., "Parallel cameras," Optica 5, 127-137 (2018)

    work page 2018

  4. [3]

    Mapping global floods with 10 years of satellite radar data,

    A. Misra, et al., "Mapping global floods with 10 years of satellite radar data," Nature Communications 16, 5762 (2025)

    work page 2025

  5. [4]

    Astronomical image denoising by self-supervised deep learning and restoration processes,

    T. Liu, et al., "Astronomical image denoising by self-supervised deep learning and restoration processes," Nature Astronomy, 1-8 (2025)

    work page 2025

  6. [5]

    Pathology-oriented multiplexing enables integrative disease mapping,

    M. Kuehl, et al., "Pathology-oriented multiplexing enables integrative disease mapping," Nature, 1-11 (2025)

    work page 2025

  7. [6]

    Neutralizing the impact of atmospheric turbulence on complex scene imaging via deep learning,

    D. Jin, et al., "Neutralizing the impact of atmospheric turbulence on complex scene imaging via deep learning," Nature Machine Intelligence 3, 876-884 (2021)

    work page 2021

  8. [7]

    Towards real-time photorealistic 3D holography with deep neural networks,

    L. Shi, et al., "Towards real-time photorealistic 3D holography with deep neural networks," Nature 591, 234-239 (2021)

    work page 2021

Show all 33 references
  1. [8]

    Interpretable socioeconomic status inference from aerial imagery through urban patterns,

    J. L. Abitbol and M. Karsai, "Interpretable socioeconomic status inference from aerial imagery through urban patterns," Nature Machine Intelligence 2, 684-692 (2020)

    2020

  2. [9]

    Image compression based on compressive sensing: End-to-end comparison with JPEG,

    X. Yuan and R. Haimi-Cohen, "Image compression based on compressive sensing: End-to-end comparison with JPEG," IEEE Transactions on Multimedia 22, 2889-2904 (2020)

    2020

  3. [10]

    Image data compression: A review,

    A. K. Jain, "Image data compression: A review," Proceedings of the IEEE 69, 349-389 (1981)

    1981

  4. [11]

    Memristor-based storage system with convolutional autoencoder-based image compression network,

    Y. Feng, et al., "Memristor-based storage system with convolutional autoencoder-based image compression network," Nature Communications 15, 1132 (2024)

    2024

  5. [12]

    The JPEG 2000 still image compression standard,

    A. Skodras, et al., "The JPEG 2000 still image compression standard," IEEE Signal processing magazine 18, 36-58 (2002)

    2002

  6. [13]

    A tutorial on modern lossy wavelet image compression: foundations of JPEG 2000,

    B. E. Usevitch, "A tutorial on modern lossy wavelet image compression: foundations of JPEG 2000," IEEE signal processing magazine 18, 22-35 (2002)

    2002

  7. [14]

    Learning content-weighted deep image compression,

    M. Li, et al., "Learning content-weighted deep image compression," IEEE transactions on pattern analysis and machine intelligence 43, 3446-3461 (2020)

    2020

  8. [15]

    End-to-end optimized image compression,

    J. Ballé, et al., "End-to-end optimized image compression," arXiv preprint arXiv:1611.01704 (2016)

    2016 arXiv

  9. [16]

    An on-chip photonic deep neural network for image classification,

    F. Ashtiani, et al., "An on-chip photonic deep neural network for image classification," Nature 606, 501-506 (2022)

    2022

  10. [17]

    Deep learning with coherent nanophotonic circuits,

    Y. Shen, et al., "Deep learning with coherent nanophotonic circuits," Nature photonics 11, 441-446 (2017)

    2017

  11. [18]

    Universal photonic artificial intelligence acceleration,

    S. R. Ahmed, et al., "Universal photonic artificial intelligence acceleration," Nature 640, 368-374 (2025)

    2025

  12. [19]

    Integrated photonic metasystem for image classifications at telecommunication wavelength,

    Z. Wang, et al., "Integrated photonic metasystem for image classifications at telecommunication wavelength," Nature communications 13, 2131 (2022)

    2022

  13. [20]

    Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit,

    T. Zhou, et al., "Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit," Nature Photonics 15, 367-373 (2021)

    2021

  14. [21]

    Parallel convolutional processing using an integrated photonic tensor core,

    J. Feldmann, et al., "Parallel convolutional processing using an integrated photonic tensor core," Nature 589, 52-58 (2021)

    2021

  15. [22]

    Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence,

    Z. Xu, et al., "Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence," Science 384, 202-209 (2024)

    2024

  16. [23]

    All-optical spiking neurosynaptic networks with self-learning capabilities,

    J. Feldmann, et al., "All-optical spiking neurosynaptic networks with self-learning capabilities," Nature 569, 208-214 (2019)

    2019

  17. [24]

    All-optical machine learning using diffractive deep neural networks,

    X. Lin, et al., "All-optical machine learning using diffractive deep neural networks," Science 361, 1004-1008 (2018)

    2018

  18. [25]

    Image sensing with multilayer nonlinear optical neural networks,

    T. Wang, et al., "Image sensing with multilayer nonlinear optical neural networks," Nature Photonics 17, 408-415 (2023)

    2023

  19. [26]

    Photonic machine learning with on-chip diffractive optics,

    T. Fu, et al., "Photonic machine learning with on-chip diffractive optics," Nature Communications 14, 70 (2023)

    2023

  20. [27]

    11 TOPS photonic convolutional accelerator for optical neural networks,

    X. Xu, et al., "11 TOPS photonic convolutional accelerator for optical neural networks," Nature 589, 44-51 (2021)

    2021

  21. [29]

    Reconnet: Non-iterative reconstruction of images from compressively sensed measurements,

    K. Kulkarni, et al., "Reconnet: Non-iterative reconstruction of images from compressively sensed measurements," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016), 449-458

    2016

  22. [30]

    Dr2-net: Deep residual reconstruction network for image compressive sensing,

    H. Yao, et al., "Dr2-net: Deep residual reconstruction network for image compressive sensing," Neurocomputing 359, 483-493 (2019)

    2019

  23. [31]

    Lossy image compression with compressive autoencoders,

    L. Theis, et al., "Lossy image compression with compressive autoencoders," arXiv preprint arXiv:1703.00395 (2017)

    2017 arXiv

  24. [32]

    Autoencoders,

    D. Bank, et al., "Autoencoders," Machine learning for data science handbook: data mining and knowledge discovery handbook, 353-374 (2023)

    2023

  25. [33]

    Design of optical neural networks with component imprecisions,

    M. Y.-S. Fang, et al., "Design of optical neural networks with component imprecisions," Optics express 27, 14009-14029 (2019)

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.