REVIEW 3 major objections 5 minor 36 references
Near-sensor Computing for Rapid Visuotactile Perception
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that a spectral Poisson solver, implemented as a fully streaming FPGA pipeline, reconstructs dense tactile depth maps with a fixed 0.211 ms latency, 0.17% error versus a double-precision reference, and triggers a…
desk verdict Solid, measured deterministic-latency hardware with an over-stated accuracy claim; the 0.17% figure is a fixed-point fidelity check, not a geometric accuracy validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the spectral Poisson solver: the observation that the discrete Laplacian is diagonalized by the discrete sine transform, turning the Poisson equation into $N^{2}$ independent scalar divisions in the frequency domain. In hardware this solver becomes a fixed-schedule streaming pipeline: gradient lookup, finite differencing, forward DST, point-wise spectral division, and inverse DST, with the 2D transform computed separably and each 1D transform implemented by a pipelined FFT core with input reordering and phase correction. This is what makes latency deterministic by construction rather than by measurement.
What would settle it
Take one of the 15 test indenters and scan its contact face with a calibrated laser profilometer to obtain physical ground truth, then compare the hardware depth maps against that scan; if the RMS deviation from the physical surface is much larger than 0.17% of peak depth, the hardware's advertised accuracy claim is unsupported.
Extended reading notes
Core claim
The central discovery is a complete mapping of the depth-reconstruction Poisson equation onto a single feed-forward hardware datapath. The solver diagonalizes the discrete Laplacian under homogeneous Dirichlet boundary conditions, so the 2D Poisson solve reduces to a forward discrete sine transform, an element-wise division in the spectral domain, and an inverse transform; the hardware implements these as pipelined streaming cores with 24-bit fixed-point arithmetic. Because the pipeline has no iteration or input-dependent control flow, every frame takes exactly 35,107 clock cycles at 128x128, which is 0.211 ms at 166 MHz, and the same datapath runs at 64x64 and 256x256 with quadratic latency scaling. The reconstructed depth maps agree with a double-precision software reference to 0.17% RMS of peak contact depth over 15 geometries, and an on-chip threshold comparator acting on the depth stream triggers finger withdrawal in 28.3±4.9 ms, about six times faster than a host-based loop using the same actuator.
Load-bearing premise
The accuracy and reflex claims stand on treating the double-precision software Poisson reconstruction as the true contact geometry; if that spectral model misrepresents real contacts, the hardware inherits the error.
Editorial extensions
If this is right
- Every 128x128 tactile frame can be reconstructed to depth in a fixed 0.211 ms, making tactile geometry available at the camera frame rate with no frame-to-frame jitter.
- A threshold comparator fed directly by the depth stream can trigger a protective reflex in 28.3 ms, below the reported human withdrawal-reflex range of 65–137 ms.
- The same streaming architecture scales to 64x64 and 256x256 with latency proportional to pixel count, so higher-resolution tactile depth remains deterministic.
- At 324 mW estimated in a 45 nm ASIC implementation, the reconstruction datapath is small and power-efficient enough to consider integrating into a fingertip-scale sensor.
Reading between the lines
- If the underlying spectral Poisson model is physically accurate for real contacts, the same architecture could be reused for any sensing modality that produces gradient fields and needs fast depth, not just visuotactile skins.
- The paper's accuracy comparison validates fixed-point arithmetic against floating-point software, not against physical ground truth; a profilometer-based validation would be the natural next test.
- Event-based tactile sensors and this dense-depth pipeline are complementary: events flag fast changes while this pipeline supplies continuous geometry, so fusing them could cover both slip transients and steady contact shape.
- Because only on-chip memory limits grid size, adding external memory could push the deterministic pipeline beyond 256x256 without changing the solver core.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents a near-sensor computing architecture for visuotactile sensors, mapping a spectral Poisson solver onto a fully streaming FPGA datapath. The authors report a measured reconstruction latency of 35,107 cycles (0.211 ms at 166 MHz) across 1,000 consecutive frames, a throughput of about 10,106 frames/s, power estimates of 347 mW (FPGA cores) and 324 mW (45 nm ASIC), a 0.17% RMS difference from a double-precision software reference across 15 contact geometries, and a contact-to-motion reflex loop of 28.3 ms versus 169.9 ms for a host-based baseline. The central technical claim is that moving the Poisson solve onto the sensor node yields deterministic sub-millisecond latency and enables faster robotic reflexes. The paper includes detailed methods for latency characterization, accuracy evaluation, ASIC synthesis, and reflex-loop timing.
Significance. The paper's main contribution is a credible streaming hardware implementation of a direct spectral Poisson solver with deterministic, on-chip measured latency and throughput near the theoretical limit. The latency measurements are machine-checked over 1,000 frames, and the resource-scaling analysis is informative and reproducible. If the accuracy and system-level claims were properly validated, the architecture would be a significant step for high-speed visuotactile perception. However, the current evidence does not support the advertised absolute geometric accuracy or an unambiguous 'six times faster' reflex comparison; these gaps require revision before the paper can be accepted.
major comments (3)
- [Methods, 'Reconstruction accuracy' (Fig. 3c); Discussion] The 0.17% RMS difference is computed after a per-frame scale and offset are fitted to the double-precision software reference, which implements the same spectral Poisson algorithm (Supplementary S1). This metric therefore validates fixed-point arithmetic fidelity, not physical depth accuracy; it does not bound absolute error in millimetres. The Discussion's claim that 'the sensor outputs calibrated depth in millimetres' is unsupported because no comparison to known ground-truth geometry (e.g., indenters with independently measured dimensions) is provided. Please either add such a validation or revise the abstract and Discussion to describe the result as implementation fidelity rather than geometric accuracy.
- [Methods, 'Reflex loop measurement' (Fig. 4c)] The near-sensor pathway uses the custom VBTS sensor, while the host baseline uses a GelSight Mini, a different sensor with a different frame rate and interface. The reported 6.0-fold contact-to-motion reduction therefore conflates sensor differences with the processing-location difference. To support the claim that near-sensor processing alone is responsible for the speedup, the comparison should use the same sensor with host-based reconstruction, or the latency contributions should be decomposed and reported separately (sensor capture, transfer, reconstruction, decision). As written, 'equivalent host-based loop using the same actuator' is misleading because the sensors differ.
- [Results, 'Deterministic latency and power efficiency' (Fig. 2e,f)] The power comparisons mix estimation and measurement. The 347 mW FPGA and 324 mW ASIC values are simulation/synthesis estimates for the reconstruction cores, whereas the CPU, GPU, and Jetson values are measured device or package powers. The text discloses the measurement boundaries, but the abstract's 'energy-efficient' claim rests on an apples-to-oranges comparison. Please state explicitly that the near-sensor core power is an estimate, provide an end-to-end measured power figure for the prototype (including sensor and I/O), or temper the efficiency claim accordingly.
minor comments (5)
- [Fig. 3 and text] The figure labels use 'float64' while the text uses 'double-precision'; please unify the terminology.
- [Methods, 'Reconstruction pipeline'] The text states 'no full-frame input buffering' but also describes the two transform passes as connected by 'full-frame matrix transpositions'; please clarify that the input is streamed while intermediate transposition buffers are full-frame.
- [Abstract and Fig. 1b] The abstract reports 347 mW, while Fig. 1b also lists a 324 mW ASIC estimate; please state which process technology the headline power figure refers to.
- [Results, 'Deterministic latency and power efficiency' (Fig. 2c)] The measured cycle counts scale by about 3.7x per grid-size doubling (9,421 to 35,107 to 135,448), described as 'approximately fourfold'; please report the expected O(N^2) values or explain the small deviation.
- [References] Reference [28] is a patent application; if it is the sole source for the underlying sensor framework, please add publicly available citations for the sensor design and calibration.
Circularity Check
No circularity in the derivation chain; the 0.17% figure is a fixed-point fidelity check against an independent software reference, not a circular prediction.
full rationale
The paper's central derivation is the spectral Poisson solve (Supplementary S1), which follows Strang's external textbook treatment: the discrete Laplacian is diagonalized by the DST and the depth is recovered by spectral division. No step in that derivation is defined in terms of the hardware result, and no fitted parameter is renamed as a prediction. Latency, throughput, power, and reflex-loop measurements are direct empirical measurements rather than quantities forced by construction. The reconstruction-accuracy comparison uses a double-precision software implementation of the same standard solver as a reference; this validates the fixed-point datapath's arithmetic fidelity, and the disclosed per-frame scale-and-offset alignment removes global gain/offset differences before computing the 0.17% RMS residual. That alignment and the software reference mean the figure does not by itself validate absolute physical depth in millimetres, but this is an external-validity limitation, not circularity: the hardware output is not derived from the fitted parameters, and the reference is an independent implementation of a standard model rather than a renamed version of the claimed result. Self-citations, such as ref. 28 for the architecture framework, are not load-bearing for the measured results.
Assumptions & free parameters
free parameters (1)
- Per-frame scale and offset alignment =
Not reported (fitted per frame)
assumptions (4)
- standard math Spectral diagonalization of the discrete Laplacian via DST (Strang)
- domain assumption Photometric stereo lookup table accurately maps tri-color intensities to surface gradients
- domain assumption DST-II with half-pixel-shifted grid matches the sensor pixel geometry
- domain assumption Power estimates from Vivado simulation and Design Compiler represent deployed power
Cite this review
Pith. "Pith review of Near-sensor Computing for Rapid Visuotactile Perception." pith.science (2026). https://pith.science/paper/UQBI6I6E
@misc{pith2026260805725,
author = {Pith},
title = {Pith review of: Near-sensor Computing for Rapid Visuotactile Perception},
year = {2026},
howpublished = {\url{https://pith.science/paper/UQBI6I6E}},
note = {Machine review of arXiv:2608.05725}
}
read the original abstract
Visuotactile sensors reconstruct dense contact geometry from measured surface gradients, but host-based processing increases power consumption and introduces data-transfer delays and variable scheduling latency, limiting the sensing and response speed of robotic systems. To address these limitations, we implement a near-sensor computing framework that includes a spectral Poisson solver as a fully streaming hardware pipeline. The computational core logic has an estimated power consumption of 347 mW and achieves high throughput without data-dependent branching or iterative convergence, thereby providing deterministic latency. Operating at 166 MHz, the pipeline produces the first depth value of each 128x128 frame 35,107 cycles after receiving the first input pixel, corresponding to a fixed latency of 0.211 ms. Across 15 contact geometries, the reconstructed depths differ from a double-precision reference by 0.17 % of the peak contact depth. On-chip decisions based on these reconstructions close a robot protective reflex loop in 28.3 +/- 4.9 ms, compared with 169.9 +/- 27.8 ms for an equivalent host-based loop using the same actuator. These results demonstrate that near-sensor reconstruction can provide accurate, energy-efficient, and deterministic tactile geometry on timescales suitable for rapid robotic contact responses.
Reference graph
Works this paper leans on
-
[1]
Yuan, W., Dong, S. & Adelson, E. H. GelSight: High-resolution robot tactile sensors for estimating geometry and force.Sensors17, 2762 (2017). RIMLAB 10 RIMLAB ROBOT INTERACTION AND MANIPULATION LABORATORY
work page 2017
-
[2]
Lambeta, M.et al.DIGIT: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation.IEEE Robotics and Automation Letters5, 3838–3845 (2020)
work page 2020
-
[3]
Taylor, I. H., Dong, S. & Rodriguez, A. GelSlim 3.0: High-resolution measurement of shape, force and slip in a compact tactile-sensing finger. In2022 International Conference on Robotics and Automation (ICRA), 10781–10787 (2022)
work page 2022
-
[4]
Li, S.et al.When vision meets touch: A contemporary review for visuotactile sensors from the signal processing perspective.IEEE Journal of Selected Topics in Signal Processing18, 267–287 (2024)
work page 2024
-
[5]
Luo, S.et al.Tactile robotics: An outlook.IEEE Transactions on Robotics41, 5564–5583 (2025)
work page 2025
- [6]
-
[7]
Lin, P.et al.Pp-tac: Paper picking using tactile feedback in dexterous robotic hands.arXiv preprint arXiv:2504.16649(2025)
arXiv 2025
- [8]
Show all 36 references
-
[9]
In2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3988–3993 (2014)
Li, R.et al.Localization and manipulation of small parts using GelSight tactile sensing. In2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3988–3993 (2014)
2014
-
[10]
& Rodriguez, A
Ma, D., Donlon, E., Dong, S. & Rodriguez, A. Dense tactile force estimation using GelSlim and inverse FEM. In2019 International Conference on Robotics and Automation (ICRA), 5418–5424 (2019)
2019
-
[11]
Lin, C., Zhang, H., Xu, J., Wu, L. & Xu, H. 9DTact: A compact vision-based tactile sensor for accurate 3D shape reconstruction and generalizable 6D force estimation.IEEE Robotics and Automation Letters9, 923–930 (2024)
2024
-
[12]
& Adelson, E
Dong, S., Yuan, W. & Adelson, E. H. Improved GelSight tactile sensor for measuring geometry and slip. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 137–144 (2017)
2017
-
[13]
Suresh, S.et al.Neuralfeels with neural fields: Visuotactile perception for in-hand manipulation.Science Robotics9, eadl0628 (2024)
2024
-
[14]
A., Kaess, M
Huang, H.-J., Mirzaee, M. A., Kaess, M. & Yuan, W. GelSLAM: A real-time, high-fidelity, and robust 3D tactile SLAM system.arXiv preprint(2026). ArXiv:2508.15990
2026
-
[15]
& Kim, S
Lee, Y ., Lin, T.-Y ., Alexiev, A. & Kim, S. High-bandwidth tactile-reactive control for grasp adjustment.arXiv preprint(2025). ArXiv:2509.15876
2025
-
[16]
Kota, R., Shah, K., Colgate, J. E. & Reardon, G. 3D Cal: An open-source software library for depth reconstruction on vision-based tactile sensors.IEEE Robotics and Automation Letters11, 6248–6255 (2026)
2026
-
[17]
& Peters, J
Funk, N., Helmut, E., Chalvatzaki, G., Calandra, R. & Peters, J. Evetac: An event-based optical tactile sensor for robotic manipulation.IEEE Transactions on Robotics40, 3812–3832 (2024)
2024
-
[18]
& Wang, S
Jiang, T., Zhang, C., Zhang, S., Cui, S. & Wang, S. SpikingTac: A miniaturized neuromorphic visuotactile sensor for high-precision dynamic tactile imprint tracking.arXiv preprint(2026). ArXiv:2602.23654
2026
-
[19]
& Chai, Y
Zhou, F. & Chai, Y . Near-sensor and in-sensor computing.Nature Electronics3, 664–671 (2020)
2020
-
[20]
Chen, Y .et al.Capacitive in-sensor tactile computing.Nature Communications16, 5691 (2025)
2025
-
[21]
Cho, J.-Y .et al.Tactile near-sensor computing systems incorporating hourglass-shaped microstructured capacitive sensors for bio-realistic energy efficiency.npj Flexible Electronics9, 34 (2025)
2025
-
[22]
& Xiao, C
Leng, Y ., Chen, Y ., Tang, Z. & Xiao, C. Touch-linked sleeve: A haptic interface for augmented tactile perception in robotic teleoperation. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), i–vii (IEEE, 2025)
2025
-
[23]
R., Reguly, I
Kamalakkannan, K., Mudalige, G. R., Reguly, I. Z. & Fahmy, S. A. High-level FPGA accelerator design for structured-mesh-based explicit numerical solvers. InIEEE International Parallel and Distributed Processing Symposium (IPDPS), 1087–1096 (2021). RIMLAB 11 RIMLAB ROBOT INTERA...
2021
-
[24]
& Marek, M
Tomczak, T., Ksi˛ e˙zyk, M., Hanke, J., Kostur, M. & Marek, M. Double-precision hardware accelerator for incompressible Navier–Stokes equations solver based on discontinuous Galerkin method.Advances in Engineering Software176, 103393 (2023)
2023
-
[25]
I., Damaj, I
Kasbah, S. I., Damaj, I. W. & Haraty, R. A. Multigrid solvers in reconfigurable hardware.Journal of Computational and Applied Mathematics213, 79–94 (2008)
2008
-
[26]
F., Reza, A
Siddiqui, M. F., Reza, A. W., Shafique, A., Omer, H. & Kanesan, J. FPGA implementation of real-time SENSE reconstruction using pre-scan and Emaps sensitivities.Magnetic Resonance Imaging44, 82–91 (2017)
2017
-
[27]
& Omer, H
Inam, O., Basit, A., Qureshi, M. & Omer, H. FPGA-based hardware accelerator for SENSE (a parallel MR image reconstruction method).Computers in Biology and Medicine117, 103598 (2020)
2020
-
[28]
Xiao, C., Zhu, Z., Zhang, R. & Hu, R. A real-time three-dimensional reconstruction system, design method and edge-computing hardware. Chinese patent application CN122176184A (2026)
2026
-
[29]
Thorell, O.et al.Experimental nerve block study on painful withdrawal reflex responses in humans.PLOS ONE19, e0309048 (2024)
2024
-
[30]
& Adelson, E
Wang, S., She, Y ., Romero, B. & Adelson, E. GelSight Wedge: Measuring high-resolution 3D contact geometry with a compact robot finger. InIEEE International Conference on Robotics and Automation (ICRA), 6468–6475 (2021)
2021
-
[31]
Strang, G.Computational Science and Engineering(Wellesley-Cambridge Press, 2007)
2007
-
[32]
A fast cosine transform in one and two dimensions.IEEE Trans
Makhoul, J. A fast cosine transform in one and two dimensions.IEEE Trans. Acoust. Speech Signal Process. 28, 27–34 (1980)
1980
-
[33]
Chen, Z.et al.Training tactile sensors to learn force sensing from each other.Nature Communications17 (2026)
2026
-
[34]
ArXiv:2607.28416
Lu, X.et al.FasTac: A curved multispectral vision-based tactile sensor for high-speed high-precision 3D shape and force perception.arXiv preprint(2026). ArXiv:2607.28416
2026 arXiv
-
[35]
A survey on optimized implementation of deep learning models on the NVIDIA Jetson platform
Mittal, S. A survey on optimized implementation of deep learning models on the NVIDIA Jetson platform. Journal of Systems Architecture97, 428–442 (2019). RIMLAB 12 RIMLAB ROBOT INTERACTION AND MANIPULATION LABORATORY Supplementary Information S1. Spectral diagonalisation of th...
2019
-
[36]
(Eq. (S4)). Our implementation instead uses the DST-II as defined in SciPy ( scipy.fft.dstn, type=2), which corresponds to a half-pixel-shifted (cell-centred) grid where the boundary condition is imposed at half-grid-spacings beyond the first and last pixel. On this grid, the ...
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.