pith. sign in

arxiv: 1906.10045 · v1 · pith:DIQVIARGnew · submitted 2019-06-24 · 📡 eess.IV · cs.SY· eess.SY

A closed-loop all-electronic pixel-wise adaptive imaging system for high dynamic range video

Jie (Jack) Zhang , Jonathan P. Newman , Xiao Wang , Chetan Singh Thakur , John Rattray , Ralph Etienne-Cummings , Matthew A. Wilson This is my paper

Pith reviewed 2026-05-25 16:47 UTC · model grok-4.3

classification 📡 eess.IV cs.SYeess.SY
keywords high dynamic range videopixel-wise adaptive imagingCMOS image sensorclosed-loop exposure controlall-electronic adaptationmotion blur reductioncomputer vision tasks
0
0 comments X

The pith

A CMOS imaging system adapts each pixel's exposure and sampling rate in a closed electronic loop to capture high dynamic range video.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper describes an imaging system that uses a custom sensor where exposure can be set differently for each pixel. A controller works with the sensor in a feedback loop to adjust these settings on the fly using only electronic circuits. The adjustments aim to prevent areas from being too dark or too bright and to cut down on motion blur during video capture. Such adaptation matters because it lets the camera handle scenes with big differences in brightness and movement without extra hardware, leading to clearer videos for analysis.

Core claim

The system consists of a custom designed image sensor with pixel-wise exposure configurability and a real-time pixel exposure controller that operate in a closed-loop to sample, detect and optimize each pixel's exposure and sampling rate to minimize local region's underexposure, overexposure and motion blurring. Exposure control is implemented using all-integrated electronics without external optical modulation. This reduces overall system size and power consumption. Experiments under complex lighting and motion conditions demonstrate the benefit on computer vision tasks such as segmentation, motion estimation and object recognition.

What carries the argument

Pixel-wise exposure configurability in the image sensor combined with a real-time closed-loop exposure controller implemented in all-electronic hardware.

If this is right

  • Minimizes underexposure, overexposure, and motion blurring in local regions of video frames.
  • Reduces overall system size and power consumption by avoiding external optical modulation.
  • Improves performance of computer vision tasks including segmentation, motion estimation, and object recognition.
  • The sensor is realized in a standard 130nm CMOS process while the controller runs on a computer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Direct integration of the controller onto the sensor chip could lower latency further.
  • The same per-pixel feedback principle might apply to other sensor technologies facing dynamic range limits.
  • Additional tests with faster motion or wider brightness ranges could expose practical bounds on sampling rate adaptation.

Load-bearing premise

Real-time closed-loop control of per-pixel exposure and sampling rate can be achieved using only integrated electronics while handling complex lighting and motion conditions.

What would settle it

An experiment in which the system fails to reduce underexposure, overexposure or motion blur in complex scenes, or requires external optical components to function, would disprove the central claim.

Figures

Figures reproduced from arXiv: 1906.10045 by Chetan Singh Thakur, Jie (Jack) Zhang, John Rattray, Jonathan P. Newman, Matthew A. Wilson, Ralph Etienne-Cummings, Xiao Wang.

Figure 1
Figure 1. Figure 1: (a-d) Snapshot of videos acquired with frame based CI sensors. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pixel-wise image sensor architecture. The sensor is fabricated in a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Timing diagram during pixel pixel exposure and pixel charge transfer [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 2
Figure 2. Figure 2: To rapidly prototype different feedback control algorithms, we made use of the Bonsai visual programming language [17]. We developed Bonsai plugins on top of a data stream￾ing API [18]. Specifically, we developed a source plugin (PCECameraRead) to capture images from the camera and a sink (PCECameraWrite) to write pixel exposure times to the camera using the bi-directional PCIe link. Code for these plugins… view at source ↗
Figure 5
Figure 5. Figure 5: Chip micrograph and experimental setups: (a) chip micrograph (b) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pixel-wise exposure optimization for stationary scene using controller [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Pixel-wise exposure optimization for scenes with instantaneous motion using controller in PI mode. Two video examples are shown in (a) and (b). [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Pixel-wise exposure optimization for scenes with continuous motion using controller in OF mode. (a) Six frame from a video where image sensor [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
read the original abstract

We demonstrated a CMOS imaging system that adapts each pixel's exposure and sampling rate to capture high dynamic range (HDR) videos. The system consist of a custom designed image sensor with pixel-wise exposure configurability and a real-time pixel exposure controller. These parts operate in a closed-loop to sample, detect and optimize each pixel's exposure and sampling rate to minimize local region's underexposure, overexposure and motion blurring. Exposure control is implemented using all-integrated electronics without external optical modulation. This reduces overall system size and power consumption. The image sensor is implemented using a standard 130nm CMOS process while the exposure controller is implemented on a computer. We performed experiments under complex lighting and motion condition to test performance of the system, and demonstrate the benefit of pixel-wise adaptive imaging on the performance of computer vision tasks such as segmentation, motion estimation and object recognition.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript presents a CMOS imaging system for high dynamic range video that uses a custom 130 nm image sensor with per-pixel exposure configurability together with a real-time exposure controller. These components form a closed loop that samples each pixel, detects local underexposure/overexposure/motion blur, and optimizes exposure time and sampling rate on a per-pixel basis. The system is claimed to operate with all-integrated electronics and no external optical modulation; the sensor is fabricated in standard 130 nm CMOS while the controller runs on an external computer. Experiments under complex lighting and motion are stated to demonstrate advantages for downstream vision tasks such as segmentation, motion estimation, and object recognition.

Significance. A working per-pixel closed-loop exposure controller that truly runs on-chip would constitute a meaningful hardware advance for compact HDR video, directly addressing power and size constraints that currently limit adaptive imaging in embedded applications. The absence of any reported quantitative metrics (PSNR, dynamic-range improvement, latency, power figures, or task-specific accuracy gains) prevents evaluation of whether the claimed benefits are realized.

major comments (2)
  1. [Abstract] Abstract: the assertion that 'Exposure control is implemented using all-integrated electronics without external optical modulation' is directly contradicted by the subsequent statement that 'the exposure controller is implemented on a computer.' Because the central claim rests on an all-electronic, on-sensor closed loop, this inconsistency is load-bearing and must be resolved before the contribution can be assessed.
  2. [Abstract] Abstract: the manuscript states that 'experiments were performed under complex lighting and motion condition' yet supplies no quantitative results, implementation details, timing measurements, or performance metrics. Without these data the central empirical claim cannot be verified.
minor comments (1)
  1. [Abstract] The abstract contains a grammatical error ('The system consist of').

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address the two major points below and will revise the manuscript to improve clarity and add supporting data where feasible.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the assertion that 'Exposure control is implemented using all-integrated electronics without external optical modulation' is directly contradicted by the subsequent statement that 'the exposure controller is implemented on a computer.' Because the central claim rests on an all-electronic, on-sensor closed loop, this inconsistency is load-bearing and must be resolved before the contribution can be assessed.

    Authors: We agree the abstract wording creates ambiguity. 'All-integrated electronics' and 'without external optical modulation' are meant to contrast the system against approaches that rely on mechanical or optical components (e.g., variable ND filters); the sensor itself provides per-pixel electronic exposure control in 130 nm CMOS. The controller algorithm, however, runs off-chip on a computer in this prototype. We will revise the abstract to explicitly state that the closed-loop control is demonstrated with an external controller while the sensor remains fully electronic, removing any implication of an on-sensor implementation. revision: yes

  2. Referee: [Abstract] Abstract: the manuscript states that 'experiments were performed under complex lighting and motion condition' yet supplies no quantitative results, implementation details, timing measurements, or performance metrics. Without these data the central empirical claim cannot be verified.

    Authors: We acknowledge that the abstract contains no numerical metrics. The full manuscript reports experiments under the stated conditions and shows qualitative gains for segmentation, motion estimation, and recognition, but does not report PSNR, explicit dynamic-range numbers, latency, or power figures. In revision we will expand the abstract with any available quantitative task-specific accuracy improvements and add timing or implementation details to the main text; if certain standard metrics (e.g., PSNR) were not computed, we will state the evaluation approach used. revision: partial

Circularity Check

0 steps flagged

No circularity: paper describes experimental hardware system with no derivations or self-referential fitting

full rationale

The manuscript presents a physical CMOS imaging system and its experimental validation under complex lighting/motion conditions. No equations, parameter fitting, uniqueness theorems, or derivation chains appear in the provided text. The central claim concerns hardware behavior and performance on downstream tasks (segmentation, motion estimation, object recognition), which is evaluated externally via experiments rather than reduced to self-citation or input data by construction. The noted discrepancy between 'all-integrated electronics' and computer-based controller is a factual inconsistency, not a circular reduction of any claimed derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities are identifiable from the abstract, as the work describes a hardware system rather than a mathematical derivation.

pith-pipeline@v0.9.0 · 5707 in / 894 out tokens · 24807 ms · 2026-05-25T16:47:24.029732+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    Enhanced image capture through fusion,

    P. J. Burt and R. J. Kolczynski, “Enhanced image capture through fusion,” in 1993 (4th) International Conference on Computer Vision . IEEE, 1993, pp. 173–182. 8 ,XGSK K^VUY[XK RKTMZN ,XGSK /SGMK 6O^KR K^VUY[XK VGZZKXT ͣGͤ H SY SY Fig. 7. Pixel-wise exposure optimization for scenes with instantaneous motion using controller in PI mode. Two video examples a...

    work page 1993

  2. [2]

    Radiometric self calibration,

    T. Mitsunaga and S. K. Nayar, “Radiometric self calibration,” in Pro- ceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149) , vol. 1. IEEE, 1999, pp. 374–380

    work page 1999

  3. [3]

    On being ‘undigital’ with digital camera: extending dynamic range by combining differently exposed pictures,

    S. Mann and R. Picard, “On being ‘undigital’ with digital camera: extending dynamic range by combining differently exposed pictures,” MIT Media Lab Perceptual , vol. 1, p. 2, 1994

    work page 1994

  4. [4]

    A versatile hdr video production system,

    M. D. Tocci, C. Kiser, N. Tocci, and P. Sen, “A versatile hdr video production system,” in ACM Transactions on Graphics (TOG) , vol. 30, no. 4. ACM, 2011, p. 41

    work page 2011

  5. [5]

    Split aperture imaging for high dynamic range,

    M. Aggarwal and N. Ahuja, “Split aperture imaging for high dynamic range,” International Journal of Computer Vision , vol. 58, no. 1, pp. 7–17, 2004

    work page 2004

  6. [6]

    High dynamic range imaging: Spatially varying pixel exposures,

    S. K. Nayar and T. Mitsunaga, “High dynamic range imaging: Spatially varying pixel exposures,” in Computer Vision and Pattern Recognition,

    work page

  7. [7]

    IEEE Conference on , vol

    Proceedings. IEEE Conference on , vol. 1. IEEE, 2000, pp. 472– 479

    work page 2000

  8. [8]

    Generalized mosaicing,

    Y . Y . Schechner and S. K. Nayar, “Generalized mosaicing,” in Proceed- ings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 1. IEEE, 2001, pp. 17–24

    work page 2001

  9. [9]

    Adaptive dynamic range imaging: Optical control of pixel exposures over space and time,

    S. K. Nayar and V . Branzoi, “Adaptive dynamic range imaging: Optical control of pixel exposures over space and time,” in null. IEEE, 2003, p. 1168

    work page 2003

  10. [10]

    Compact all-cmos spatiotemporal compressive sensing video camera with pixel-wise coded exposure,

    J. Zhang, T. Xiong, T. Tran, S. Chin, and R. Etienne-Cummings, “Compact all-cmos spatiotemporal compressive sensing video camera with pixel-wise coded exposure,” Optics express , vol. 24, no. 8, pp. 9013–9024, 2016

    work page 2016

  11. [11]

    Exposure-programmable cmos pixel with selective charge storage and code memory for computational imaging,

    Y . Luo, D. Ho, and S. Mirabbasi, “Exposure-programmable cmos pixel with selective charge storage and code memory for computational imaging,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 65, no. 5, pp. 1555–1566, 2017

    work page 2017

  12. [12]

    Dual-tap pipelined-code-memory coded-exposure-pixel cmos image sensor for multi-exposure single-frame computational imaging,

    N. Sarhangnejad, N. Katic, Z. Xia, M. Wei, N. Gusev, G. Dutta, R. Gulve, H. Haim, M. M. Garcia, D. Stoppa et al. , “Dual-tap pipelined-code-memory coded-exposure-pixel cmos image sensor for multi-exposure single-frame computational imaging,” in 2019 IEEE International Solid-State Circuits Conference-(ISSCC) . IEEE, 2019, pp. 102–104

    work page 2019

  13. [13]

    A review of the pinned photodiode for ccd and cmos image sensors,

    E. R. Fossum, D. B. Hondongwa et al. , “A review of the pinned photodiode for ccd and cmos image sensors,” IEEE J. Electron Devices Soc, vol. 2, no. 3, pp. 33–43, 2014

    work page 2014

  14. [14]

    A wide dynamic range cmos image sensor with multiple exposure-time signal outputs and 12-bit column-parallel cyclic a/d converters,

    M. Mase, S. Kawahito, M. Sasaki, Y . Wakamori, and M. Furuta, “A wide dynamic range cmos image sensor with multiple exposure-time signal outputs and 12-bit column-parallel cyclic a/d converters,” Solid-State Circuits, IEEE Journal of , vol. 40, no. 12, pp. 2787–2795, 2005

    work page 2005

  15. [15]

    A high- speed low-noise cmos image sensor with 13-b column-parallel single- ended cyclic adcs,

    J.-H. Park, S. Aoyama, T. Watanabe, K. Isobe, and S. Kawahito, “A high- speed low-noise cmos image sensor with 13-b column-parallel single- ended cyclic adcs,” IEEE Transactions on Electron Devices , vol. 56, no. 11, pp. 2414–2422, 2009

    work page 2009

  16. [16]

    An fpga ip core for easy dma over pcie with windows and linux

    “An fpga ip core for easy dma over pcie with windows and linux.” [Online]. Available: http://www.xillybus.com/

    work page

  17. [17]

    Submillisecond latency closed-loop feedback with pcie prototype system,

    J. Newman, J. V oigts, and A. C. Lopez, “Submillisecond latency closed-loop feedback with pcie prototype system,” May

    work page

  18. [18]

    Available: http://www.open-ephys.org/blog/2016/5/6/ submillisecond-latency-closed-loop-feedback-with-pcie-prototype-system

    [Online]. Available: http://www.open-ephys.org/blog/2016/5/6/ submillisecond-latency-closed-loop-feedback-with-pcie-prototype-system

    work page 2016

  19. [19]

    Bonsai: an event-based framework for processing and controlling data streams,

    G. Lopes, N. Bonacchi, J. Fraz ˜ao, J. P. Neto, B. V . Atallah, S. Soares, L. Moreira, S. Matias, P. M. Itskov, P. A. Correia et al. , “Bonsai: an event-based framework for processing and controlling data streams,” Frontiers in neuroinformatics, vol. 9, p. 7, 2015

    work page 2015

  20. [20]

    librosa/librosa: 0.6.3,

    J. Newman, J. V oigts, J. Zhang, ckemere, PhilDakin, T. Manders, yaoyua, Z. Rosen, and open ephys, “jonnew/open-ephys-pcie: Release 1.0.0,” Jun. 2019. [Online]. Available: https://doi.org/10.5281/zenodo. 3254431

    work page doi:10.5281/zenodo 2019

  21. [21]

    Two-frame motion estimation based on polynomial expansion,

    G. Farneb ¨ack, “Two-frame motion estimation based on polynomial expansion,” in Scandinavian conference on Image analysis . Springer, 2003, pp. 363–370

    work page 2003

  22. [22]

    An iterative image registration technique with an application to stereo vision,

    B. D. Lucas, T. Kanade et al., “An iterative image registration technique with an application to stereo vision,” 1981

    work page 1981

  23. [23]

    Flownet: Learning optical flow with convolutional networks,

    A. Dosovitskiy, P. Fischer, E. Ilg, P. Hausser, C. Hazirbas, V . Golkov, P. Van Der Smagt, D. Cremers, and T. Brox, “Flownet: Learning optical flow with convolutional networks,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 2758–2766

    work page 2015

  24. [24]

    Learning a convolutional neural network for non-uniform motion blur removal,

    J. Sun, W. Cao, Z. Xu, and J. Ponce, “Learning a convolutional neural network for non-uniform motion blur removal,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2015, pp. 769–777

    work page 2015

  25. [25]

    Deep multi-scale convolutional neural network for dynamic scene deblurring,

    S. Nah, T. Hyun Kim, and K. Mu Lee, “Deep multi-scale convolutional neural network for dynamic scene deblurring,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 3883–3891

    work page 2017