Pith. sign in

REVIEW 4 major objections 5 minor 21 references

Foveated image processing for faster object detection and recognition in embedded systems using deep convolutional neural networks

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read This paper claims that foveated (nonuniform) image sampling lets a deep object detector run about four times faster on an embedded GPU while retaining 92% of centered-object recall, where uniform downsampling retains only 50%.

desk verdict A useful matched benchmark for foveated vs uniform downsampling on an embedded GPU, but the headline recall number is measured with an oracle fovea and needs a sensitivity analysis before being trusted. read the letter →

arxiv 1908.09000 v1 pith:YJ6WIWO3 submitted 2019-08-15 cs.CV eess.IV

classification cs.CVeess.IV
keywords foveatedimagingobjectdetectionimagesizereductionembeddedGPUconvolutionalneuralnetworksYOLOv3log-spacedresamplingrecallpreservation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes replacing uniformly downsampled input images with foveated images—dense sampling in a central region and log-spaced compression toward the periphery—to make deep neural object detection practical on embedded GPUs. The claim is that this preserves detection of the centered object even when the image is shrunk from 416×416 to 128×128 pixels, cutting the number of convolution operations enough to raise frame rate from 3.59 to 15.24 FPS (about a 4× speed-up). In their retrained detector, foveal recall at 128×128 remained at 92.0% of the full-size baseline, whereas uniform downsampling at the same size kept only 50.1% of baseline recall. A sympathetic reader would care because embedded robots and other low-power devices could run advanced detectors at interactive frame rates without giving up accurate detection of the object being attended to.

What carries the argument

The load-bearing object is the Cartesian log-spaced foveated transform: starting from a fovea center $(x_0,y_0)$, sample coordinates are placed at $x_k=\exp(k\Delta_x)$ and $y_k=\exp(k\Delta_y)$ in each quadrant, so sampling density is high near the center and decreases logarithmically toward the edges. This produces a smaller image that keeps the full field of view and, because it distorts the original uniform image less than a log-polar transform, allows the network to be initialized with weights pretrained on ordinary images. The argument then rests on retraining the YOLOv3 detector at eleven image sizes, measuring recall and precision separately for objects inside versus outside the fovea, and timing inference on an embedded GPU.

What would settle it

Measure foveal recall on the same 128×128 foveated detector when fovea locations come from an automatic saliency or tracking algorithm rather than from ground-truth object centers; if recall falls substantially below 32.38% (or below 92% of the 416×416 baseline), the claimed accuracy retention does not transfer to deployment.

Watch

Extended reading notes

Core claim

Working with a retrained single-shot detector (YOLOv3) evaluated on a standard multi-object benchmark, the paper reports that Cartesian log-spaced foveated resampling changes the speed-accuracy trade-off of input size reduction. At 416×416 pixels the detector reached 35.20% foveal recall at 3.59 FPS; at 128×128 pixels foveal recall was 32.38% (92.0% of baseline) at 15.24 FPS. Uniform downsampling to the same 128×128 size dropped recall to 17.33% (50.1% of baseline). The same qualitative pattern appeared in un-retrained YOLOv3 and Faster R-CNN with 80 classes, indicating that the foveal advantage is not an artifact of retraining on the transform. Peripheral recall at small sizes fell to about a third of baseline, so the gain is specific to a fovea centered on the object, and precision for foveated images rose as image size fell because the attended object filled more of the scene.

Load-bearing premise

The load-bearing premise is that the fovea sits on the object of interest: every foveal-recall number is measured with the fovea placed at the ground-truth object location, so real saliency or tracking errors are not included in the reported 92% figure.

Editorial extensions

If this is right

  • On a 256-core embedded GPU, the same YOLOv3 detector can run at roughly 15 FPS instead of about 3.6 FPS by taking 128×128 foveated input, while keeping centered-object recall within 8% of the full-size baseline.
  • At equal input size, foveated sampling beats uniform downsampling by a wide margin for centered objects (92.0% vs 50.1% of baseline recall), making it the better choice whenever a fovea can be placed on the target.
  • Peripheral detection at 128×128 drops to about 34% of baseline recall, so the method trades away wide-field awareness; it suits active-vision systems that redirect the fovea, not passive monitoring of the whole scene.
  • Because un-retrained YOLOv3 and Faster R-CNN show the same foveal advantage, the benefit appears to belong to the input representation rather than to a single network architecture.
  • Precision improves as foveated images shrink, so the recall loss at small sizes is partially offset by fewer false positives on the attended object.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 92%-of-baseline result is reproducible with imperfect fovea placement, the practical payoff is a cheap saliency or tracking front-end that keeps the fovea on the target, yielding close to full-size accuracy at a fraction of the compute.
  • The speedups from foveated input size reduction and from compact network architectures are independent, so combining them may push embedded detection well beyond the 15 FPS reported here.
  • A direct stress test would be to retrain with foveal centers jittered around ground truth; graceful degradation under such jitter would be evidence the method tolerates real saliency errors.
  • The peripheral loss suggests a two-stream hybrid—foveated small input for the attended object plus a very cheap uniform wide-field branch for peripheral alarms—as a way to recover scene awareness.
  • If the method is paired with a learned saliency module inside the network, the saliency computation and the detector could share convolutional features, making the overhead of fovea placement nearly free.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes foveated image sampling based on Cartesian log-spaced resampling to reduce CNN input size while preserving detection and recognition performance in the foveal region. The authors retrain YOLOv3 on a 20-class COCO subset at 11 image sizes from 416x416 to 96x96 pixels, using both foveated and uniform downsampling, and measure recall and precision separately for objects in the fovea and in the periphery. They also evaluate un-retrained YOLOv3 and Faster R-CNN with 80 classes, and report frame rates on an NVIDIA Jetson TX2 embedded GPU. The central result is that at 128x128 with foveated inputs, foveal recall is 32.38% versus 17.33% for uniform downsampling, i.e. 92.0% versus 50.1% of the 416x416 baseline, with frame rate rising from 3.59 to 15.24 FPS, a 4.2x speedup. Peripheral recall falls much more sharply, to 34.3% of baseline for foveated images at 128x128.

Significance. If the reported effect is robust, the paper demonstrates a simple, biologically motivated way to lower compute for embedded object detection without catastrophic loss of accuracy when the fovea is correctly placed. The physiological constants in Table 1 come from external data rather than being fitted to the COCO results, and the comparison across two CNN families (retrained YOLOv3 and un-retrained YOLOv3/Faster R-CNN) provides a useful generalization check. The paper does not provide code or detailed reproducibility information, and the headline result is conditional on ideal fovea placement; nevertheless, the core experiment is a legitimate empirical comparison and the 4x speedup figure is clear as an upper bound on CNN-stage frame rate.

major comments (4)
  1. [Section 3.1 and Table 2] The headline claim that foveal recall at 128x128 is 92.0% of the 416x416 baseline is measured only when the fovea is exactly centered on the object. Section 2.5 states 'it is assumed that the fovea is centered in the object', and Section 3.1 says a saliency step 'crudely aligns the fovea with a point of interest'. Since all foveated training and validation images are generated from ground-truth object locations (Section 2.1), the network never sees a misaligned fovea. Table 3 (right) shows that at 128x128, peripheral recall for foveated images is 34.3% of baseline, which is below the 45.3% obtained by uniform downsampling. The reported 92% is therefore an upper bound for any deployed system whose saliency can miss or mislocate objects. The central claim should be restated as conditional on accurate fovea placement, or the authors should add an experiment with perturbed fovea centers to quantify sensitivity to saliency error.
  2. [Section 3.1 and Table 2] The frame-rate speedup from 3.59 FPS to 15.24 FPS covers only YOLOv3 inference, not the complete pipeline shown in Figure 5, which includes a Vocus2 saliency stage. The saliency computation time on the Jetson TX2 is not reported. Since saliency must run before foveation in the proposed architecture, the end-to-end frame rate is lower than the reported value. The manuscript should either measure and include saliency runtime in the frame-rate comparison, or explicitly restrict the speedup claim to the CNN inference stage.
  3. [Section 2.5] The protocol for computing IoU between predictions and ground-truth boxes is not specified. Foveated sampling uses exponential spacing (Eqs. 1-4), so bounding boxes predicted in the foveated coordinate frame do not map to rectangles in the original image without an explicit inverse transform. The text only says that IoU is computed with a threshold of 0.5, but does not state whether predictions are first inverse-transformed to original coordinates, how non-rectangular regions are handled, or what rounding/clipping is used. Without this detail, the recall and precision values in Table 2 cannot be reproduced.
  4. [Section 3.1 and Table 2] The results are reported as single-point recall and precision values with no variance, confidence intervals, or number of training runs. Differences as small as 0.07 percentage points (e.g., foveal recall 35.20 at 416 vs 35.27 at 384) are unlikely to be meaningful given the stochasticity of CNN training. The claim that recall decreases only slightly from 416 to 128 needs a paired comparison or at least an indication of run-to-run variability; otherwise the 92.0% figure is a ratio of two point estimates with unknown noise.
minor comments (5)
  1. [Introduction] There is a missing citation marker in the phrase 'which accounts for just ∼2% of the visual field [?]' in Section 1; the reference needs to be filled in.
  2. [Section 3.2] The text refers to 'Table 3.2' and 'table 3.2' but the tables are numbered as Table 2 and Table 3; the cross-references should be corrected.
  3. [Figure 1 caption] The caption contains typographical errors: 'at it's original resolution' should be 'at its original resolution', and the text in Section 2 also has 'inreasingly' instead of 'increasingly'.
  4. [Section 2.3] The validation set is described only as 'a reserved validation data set of 6000 images'; please specify whether this is the standard COCO val split and how the 6000 images were selected or stratified across the 20 classes.
  5. [Section 2.1] The claim that 82,000 training images expand to 306,000 foveated images implies about 3.7 objects per image; the text should clarify that one foveated image is generated per object (with the fovea centered on that object), and that the uniform-sampling comparison copies each image by the same multiplicity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: foveation constants are external physiological data, and the 92% recall / 4x speed-up figures are measured outcomes under a stated oracle-centered evaluation, not derived quantities.

full rationale

The paper is an empirical comparison, not a derivation, so the circularity burden is effectively zero. The foveated resampling in Section 2.2 uses Cartesian log-spaced sample locations x_k = exp(k*Delta_x), whose spacing parameters are tied to the retino-cortical data in Table 1, reproduced from the external reference [19]; no parameter in the transform is fitted to the COCO detection results in this paper. The headline recall and frame-rate figures in Section 3.1 are measured outcomes on the Jetson TX2 and the validation set, not quantities derived from the transform equations: recall 32.38% (92.0% of baseline) and 15.24 FPS at 128x128 are reported from tables after retraining YOLOv3 at each size. The central limitation, that the fovea is assumed centered on the object (Section 2.5) and saliency alignment is deferred to future work (Section 4.2), is candidly stated and affects external validity, but it does not make the measured foveal recall circular, because the centered-fovea condition is an explicit evaluation protocol rather than a definition that forces the result. There are no load-bearing self-citations: the authors cite no prior work of their own as authority for any uniqueness claim, and the constants from [19] are independent physiological data. Thus no step reduces a prediction to a fitted input, and no known result is merely renamed.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No fitted constants enter the central comparison, and no new entities are introduced. The reasoning rests on the foveation model from physiological data, on transfer learning across distorted inputs, on an oracle centering assumption, and on the fairness of the matched per-object baseline.

free parameters (1)
  • Fovea center (x0, y0) = Ground-truth object center per image
    Section 2.2 centers the transform at (x0,y0); Section 2.5 assumes this is the object location. The paper does not evaluate other centering choices, so imperfect saliency is untested.
assumptions (4)
  • domain assumption Log-spaced Cartesian foveation is a sufficient proxy for human retinal sampling and for the relative performance of foveated CNNs.
    Section 2.2 introduces the transform and says it 'captures the key feature of densely sampling the fovea and compressing the periphery', citing physiological data from Table 1 [19] but no validation that this particular transform is representative.
  • domain assumption Transfer learning from weights pretrained on uniformly sampled images transfers to foveated images.
    Section 2.2 claims the Cartesian distortion is small enough to 'enable the use of transfer learning'; training convergence is not shown, so the claim rests on the similarity assumption.
  • domain assumption The fovea can be centered on objects of interest at test time by an external saliency or tracking step.
    Section 2.5 states 'it is assumed that the fovea is centered in the object'; Section 3.1 similarly assumes 'a saliency step has already been performed'. No experiments include imperfect alignment.
  • domain assumption Per-object copies of uniformly sampled images provide a matched baseline for foveated per-object images.
    Section 2.1 says the number of uniformly sampled images was matched to 306,000 by copying each image by the number of objects, but the paper never specifies how recall is tallied per copy in the validation set, so the baseline comparison depends on this unstated protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Foveated image processing for faster object detection and recognition in embedded systems using deep convolutional neural networks." pith.science (2026). https://pith.science/paper/YJ6WIWO3

@misc{pith2026190809000,
  author       = {Pith},
  title        = {Pith review of: Foveated image processing for faster object detection and recognition in embedded systems using deep convolutional neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YJ6WIWO3}},
  note         = {Machine review of arXiv:1908.09000}
}
read the original abstract

Object detection and recognition algorithms using deep convolutional neural networks (CNNs) tend to be computationally intensive to implement. This presents a particular challenge for embedded systems, such as mobile robots, where the computational resources tend to be far less than for workstations. As an alternative to standard, uniformly sampled images, we propose the use of foveated image sampling here to reduce the size of images, which are faster to process in a CNN due to the reduced number of convolution operations. We evaluate object detection and recognition on the Microsoft COCO database, using foveated image sampling at different image sizes, ranging from 416x416 to 96x96 pixels, on an embedded GPU -- an NVIDIA Jetson TX2 with 256 CUDA cores. The results show that it is possible to achieve a 4x speed-up in frame rates, from 3.59 FPS to 15.24 FPS, using 416x416 and 128x128 pixel images respectively. For foveated sampling, this image size reduction led to just a small decrease in recall performance in the foveal region, to 92.0% of the baseline performance with full-sized images, compared to a significant decrease to 50.1% of baseline recall performance in uniformly sampled images, demonstrating the advantage of foveated sampling.

Figures

Figures reproduced from arXiv: 1908.09000 by the authors.

Figure 1
Figure 1. (a) Representation of some key eccentricity values departing from the center of the fovea. (b) Curve representation of the logarithmic behaviour of the rows selected for the example image transformed from 2080 to 494 pixel per side. (d) NVIDIA Jetson TX2 for embedded systems (256 Pascal CUDA core GPU, dual-core Denver2 processer, quad-core ARM-A57 processor, 8GB memory, 7.5 Watts, 50×87 mm) mounted on a development … view at source ↗
Figure 2
Figure 2. Recall and precision performance for the YOLO network trained at different resolutions. Top row: objects in the fovea. Bottom row: objects in the periphery. Top left also shows the average frame rate for processing each image size. 128 × 128 pixels, recall performance for both foveal and uniformly sampled im￾ages decreased significantly, to 34.3% of baseline for foveated images and 45.3% of baseline for uniform imag… view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Precision and recall performance curves for the network at small resolutions (96x96, 128x128, 160x160 and 192x192). The foveal advantage is much more evident for the smaller networks, where the speed-up is also larger. In all cases, the performance is very similar betw…
Figure 5
Figure 5. Figure 5: Example of an image from the Coco dataset run through the Vocus2 bottom-up saliency algorithm [3], then taking the most salient point as the center of the fovea. In this example the original image is 640x480 pixels, which in this case is downsampled to 160x160 pixels …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 19 canonical work pages

  1. [1]

    PLoS Computational Biology 13(10), e1005743 (2017)

    Akbas, E., Eckstein, M.P.: Object detection through search with a foveated visual system. PLoS Computational Biology 13(10), e1005743 (2017)

  2. [2]

    In: Robot 2017: Third Iberian Robotics Conference

    Almeida, A.F., Figueiredo, R., Bernardino, A., Santos-Victor, J.: Deep networks for human visual attention: A hybrid model using foveal vision. In: Robot 2017: Third Iberian Robotics Conference. pp. 117–128 (2017)

  3. [3]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Frintrop, S., Werner, T., Martin Garcia, G.: Traditional saliency reloaded: A good old model in new shape. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 82–90 (2015)

  4. [4]

    IEEE Transactions on Image Processing 13(10), 1304–1318 (2004) 12 U

    Itti, L.: Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing 13(10), 1304–1318 (2004) 12 U. Jaramillo-Avila and S. R. Anderson

  5. [5]

    In: European Conference on Computer Vision

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: European Conference on Computer Vision. pp. 740–755. Springer (2014)

  6. [6]

    In: Proceedings of the IASTED International Conference on Signal Processing, Pattern Recognition, and Applications, SPPRA 2006

    Martinez, J., Altamirano, L.: A new foveal cartesian geometry approach used for object tracking. In: Proceedings of the IASTED International Conference on Signal Processing, Pattern Recognition, and Applications, SPPRA 2006. pp. 133–139. Innsbruck, Austria (2006)

  7. [7]

    In: NIPS-W (2017), accessed: 2018-10-20

    Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch. In: NIPS-W (2017), accessed: 2018-10-20

  8. [8]

    Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks

    Recasens, A., Kellnhofer, P., Stent, S., Matusik, W., Torralba, A.: Learning to zoom: a saliency-based sampling layer for neural networks. arXiv preprint arXiv:1809.03355 (2018)

Show all 21 references
  1. [9]

    [online] http:// pjreddie.com/darknet/ (2016), accessed: 2018-08-25

    Redmon, J.: Darknet: Open source neural networks in C. [online] http:// pjreddie.com/darknet/ (2016), accessed: 2018-08-25

  2. [10]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 779–788 (2016)

  3. [11]

    arXiv preprint arXiv:1804.02767 (2018)

    Redmon, J., Farhadi, A.: YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  4. [12]

    In: Advances in Neural Information Pro- cessing Systems

    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Pro- cessing Systems. pp. 91–99 (2015)

  5. [13]

    arXiv preprint arXiv:1709.05943 (2017)

    Shafiee, M.J., Chywl, B., Li, F., Wong, A.: Fast YOLO: A fast you only look once system for real-time embedded object detection in video. arXiv preprint arXiv:1709.05943 (2017)

  6. [14]

    Information Processing & Management 45(4), 427–437 (2009)

    Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Information Processing & Management 45(4), 427–437 (2009)

  7. [15]

    In: ICCV2017, the Interna- tional Conference on Computer Vision

    Tijtgat, N., Van Ranst, W., Volckaert, B., Goedem´ e, T., De Turck, F.: Embedded real-time object detection for a uav warning system. In: ICCV2017, the Interna- tional Conference on Computer Vision. pp. 2110–2118 (2017)

  8. [16]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 17(5), 500–511 (1995)

    Tong, F., Li, Z.N.: Reciprocal-wedge transform for space-variant sensing. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(5), 500–511 (1995)

  9. [17]

    Robotics and Autonomous Systems 58(4), 378–398 (2010)

    Traver, V.J., Bernardino, A.: A review of log-polar imaging for visual perception in robotics. Robotics and Autonomous Systems 58(4), 378–398 (2010)

  10. [18]

    Nature 341(6243), 643– 646 (1989)

    W¨ assle, H., Gr¨ unert, U., R¨ ohrenbeck, J., Boycott, B.B.: Cortical magnification factor and the ganglion cell density of the primate retina. Nature 341(6243), 643– 646 (1989)

  11. [19]

    International Journal of Man- Machine Studies 18(4), 361–389 (1983)

    Wilson, S.W.: On the retino-cortical mapping. International Journal of Man- Machine Studies 18(4), 361–389 (1983)

  12. [20]

    In: CVPR Workshops

    Wu, B., Iandola, F.N., Jin, P.H., Keutzer, K.: Squeezedet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In: CVPR Workshops. pp. 446–454 (2017)

  13. [21]

    Design Automation for Embedded Systems pp

    Zhang, X., Gao, T., Gao, D.: A new deep spatial transformer convolutional neural network for image saliency detection. Design Automation for Embedded Systems pp. 1–14 (2018)

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.