REVIEW 3 major objections 7 minor 24 references
Justlookup: One Millisecond Deep Feature Extraction for Point Clouds By Lookup Tables
T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A lookup table can replace the point cloud network at inference: 1.5 ms on CPU.
desk verdict Neat and valid acceleration trick for point-wise point cloud networks, but the abstract's trio of 1.5 ms, 15 MB, and no degradation does not hold at 15 MB; no-degradation needs the 8 GB table. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key machinery is the factorization $F(X) = \max_{p} (h_1(x_p), \dots, h_m(x_p))$, where each $h_s: \mathbb{R}^3 \to \mathbb{R}$ is a 3-variable function implemented by the shared MLP. Because the input space is only 3-dimensional, the set $\{h_s\}$ can be sampled at voxel centers and stored as a 4-dimensional table indexed by voxel position and function index; quantization to 8-bit values shrinks the table without hurting accuracy. At test time each point produces its $m$ function values by memory lookups, and a max over points yields the global feature, followed by a small trained head. The table acts as a universal precomputation of the entire feature extractor, so network depth and width no longer affect inference cost.
What would settle it
Train PointWise on a fine-grained part-segmentation dataset and compare per-part IoU with PointNet; if the point-wise model underperforms by a large margin, the lookup table claim 'no performance degradation' fails at the architecture level, not the table level.
Extended reading notes
Core claim
The central discovery is that the deep 3-variable functions $h_s(x)$ induced by a point-wise network can be offline-encoded into a lookup table $T[i][j][k][s]$ with negligible loss in task accuracy. The paper constructs the table by subdividing $[-1,1]^3$ into $S^3$ voxels, evaluating each point-wise function at the voxel corner, quantizing the stored value to 8 bits, and then fine-tuning the network head so it is calibrated to the approximate features. At the default $S=200$ this delivers a 1.5 ms forward pass on CPU, and even at $S=25$ with 15 MB of memory the fine-tuned model reaches 88.86% overall accuracy on ModelNet40, only 0.4% below PointNet's published number. The paper further claims that the point-wise architecture implicitly captures local shape because max pooling grows each function's isosurface until it is tangent to the object at a critical point, so the absence of explicit neighborhoods is not a fatal limitation. The claim 'inference is only determined by memory access' holds regardless of how deep or wide the original network is.
Load-bearing premise
The load-bearing premise is that a point-wise network with no explicit neighborhood operations can be accurate enough on the task; if the underlying point-wise model is too weak, the lookup table simply encodes a weak function.
Editorial extensions
If this is right
- Inference cost for point-wise networks becomes independent of network depth and width, so one could train very large models and compress them into the same table.
- CPU-only and embedded devices can run point-cloud classification at real-time rates, since the 1.5 ms measurement is on a single CPU core without GPUs.
- The fine-tuning step turns the table's quantization error into a training signal, which may mean lookup-table models can be treated as a form of learned robust features.
- The same recipe applies to any task that max-pools functions of coordinates, such as implicit surface networks, as long as the per-point functions depend only on the point itself.
Reading between the lines
- The table can be viewed as a learned spatial basis; one likely extension is adaptive, density-aware voxelization, which would use more cells near object surfaces and fewer in empty space to cut memory further.
- The paper's geometric tangency argument is informal; a formal bound relating the number of functions, voxel size, and surface-curvature would make the interpretation testable and could guide how many channels are needed for local detail.
- Fine-tuning on table outputs may act as a regularizer, which could explain why Sampled PointWise_f sometimes beats PointWise; testing on small training sets would show whether the table softens overfitting.
- The method could port to other coordinate-based networks, such as NeRF or occupancy fields, when the network is a sum or max of per-coordinate computations, though memory requirements would need rethinking.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes JustLookup, a method to accelerate inference for point-wise deep networks on point clouds. The key observation is that PointNet-style architectures compute per-point features via a shared MLP which, for each output channel, is a function of the three input coordinates. The authors discretize the input volume into S^3 voxels and precompute the MLP outputs on the voxel grid, storing them in a lookup table indexed by quantized coordinates. During inference, per-point features are read from memory, max-pooled, and passed to a task head (classification or retrieval). To recover accuracy lost to approximation, the head is fine-tuned on the approximated features. Experiments on ModelNet10/40 and ShapeNet-Core55 report CPU inference at 1.5 ms for 1024 points, a 32x speedup over PointNet, with accuracy claims that vary between 'no performance degradation' and 'almost the same performance' across different configurations. The paper also offers a geometric interpretation of max pooling as level-set tangency to explain why point-wise networks can capture local structure.
Significance. The central idea—exploiting the three-dimensional input of point-wise networks to turn feature extraction into memory lookups—is genuinely original and potentially useful for CPU-bound or edge deployments of simple point cloud models. The paper provides a concrete memory/accuracy analysis (Eq. (5)-(6), Table 4) and the construction is straightforward to reproduce in principle. If the claims were stated accurately, the method would be a valuable engineering contribution. However, the significance is narrower than the headline suggests: the method applies only to point-wise architectures without transform networks, and the reported speedup and 'no degradation' claim conflate the lookup table's effect with architectural simplification. The geometric interpretation in Section 4 is informal and does not provide quantitative support, so the paper's primary value rests on the empirical speedup and accuracy trade-off.
major comments (3)
- [Abstract, Introduction, Section 5.5, Table 4] The abstract's phrase 'without any performance degradation' is not supported by the configuration that the paper advertises with small memory. Table 4 shows that the 15 MB table (S=25) yields 88.86% overall accuracy on ModelNet40, which is 0.34 points below PointNet's 89.2%, and Section 5.5 itself calls this 'merely drops 0.4%.' The no-degradation result relative to PointNet is only achieved at S=200, which requires 8000 MB. Please report the resolution/memory trade-off explicitly and revise the abstract and introduction so that the 'no degradation' claim is associated with the 8 GB configuration, not with the 15 MB configuration.
- [Table 1, Section 5.3] The claimed 32x speedup over PointNet is not an apples-to-apples comparison of the lookup-table method. PointNet's 49.15 ms CPU inference includes two T-Net modules, which the paper removes in its PointWise baseline; the lookup table is then applied to the resulting point-wise network. The paper's own PointWise baseline without any lookup table already runs in 17 ms (Table 1), so the speedup attributable to the lookup table is roughly 17/1.5 ≈ 11x, not 32x. The remaining factor of about 3 comes from removing the T-Nets, which are not part of the point-wise functions and are not accelerated by the proposed method. Please either benchmark against PointWise as the baseline or decompose the speedup into the contributions from architectural simplification and from lookup-table access.
- [Section 5.3, Section 5.5] The accuracy comparisons are made against PointNet, but the point-wise architecture without T-Nets is itself a weaker baseline: PointWise achieves 87.95% on ModelNet40 versus PointNet's 89.2%, a 1.3-point gap. The later claim that the fine-tuned sampled model at S=200 reaches 89.51% and 'maintains the same performance' as PointNet is relative to a different architecture. Reporting the accuracy delta relative to PointWise and to PointNet separately would make clear what the lookup-table approximation itself costs and what the architectural simplification costs.
minor comments (7)
- [Title] The title says 'One Millisecond' but the reported inference time is 1.5 ms; please adjust the title or the reported time for consistency.
- [Section 3, Eq. (5)] The quantization scheme uses per-channel MIN and MAX values over the volume, but the paper never states how these extrema are estimated (e.g., over a dense random grid or analytically). Please specify the procedure and whether the MIN/MAX values are recomputed after fine-tuning.
- [Section 5.1] Please clarify whether fine-tuning of the model M is performed by backpropagating through the lookup table (which would require differentiable rounding) or by using the original MLP with the approximate outputs; the current description is ambiguous.
- [Tables 2 and 3] The retrieval results report 'Ours' and 'Ours (Sampled PointWisef)' without specifying the voxel resolution S used for those experiments. Since accuracy depends on S, please state the configuration (and memory size) for each retrieval result.
- [Section 5.6] The statement that 'we can achieve the same accuracy as PointNet by only using 500 points' is missing the voxel resolution S used for that experiment; please add this detail to make the claim reproducible.
- [Section 4 and Figure 2] The geometric interpretation with level sets is presented as an explanation of why point-wise architectures work, but it is not formalized. I recommend clearly labeling it as intuition and, if possible, adding real examples of learned level sets instead of an idealized schematic.
- [Section 5.5, Figure 3(a)] The accuracy axis of Figure 3(a) starts at 82%, which visually exaggerates the differences among configurations. Please use a scale that does not distort the reported trade-offs.
Circularity Check
No circular derivation: the lookup table directly encodes the network's own 3-variable functions, and the reported speed and accuracy are measured results.
full rationale
The central construction is not circular. Equation (3) defines the lookup table as T[i][j][k][s] = h_s(iδ,jδ,kδ), i.e., the table is a sampled encoding of the point-wise network's own learned functions, not an independently fitted quantity that is later renamed as a prediction. The approximate function RF is then obtained by table lookup plus max pooling, and the fine-tuning step adapts the downstream head M to RF, which is a standard adaptation rather than a hidden reuse of the target result. The paper's claimed inference time (1.5 ms) and accuracy numbers are measured experiment outputs on ModelNet and ShapeNet, not quantities derived by construction from the method's assumptions. There is no load-bearing self-citation: the main architectural reference is to external work on PointNet, and the authors are not the authors of that cited work. Section 4's geometric tangency argument is informal and post-hoc, but it does not smuggle the conclusion into the derivation. One non-circular concern is that the abstract's conjunction of 15 MB memory and 'without any performance degradation' is not directly supported by Table 4, where the 15 MB configuration (S=25) gives 88.86% vs. PointNet's 89.2%; the no-degradation result is shown for the 8 GB S=200 configuration. That is an empirical support/consistency issue, not circularity. Overall, no step in the derivation reduces to its own inputs.
Assumptions & free parameters
free parameters (3)
- S (number of voxels per axis) =
200 for main experiments, 25 for low-memory version
- L (quantization bits per stored value) =
8
- Point-wise MLP architecture =
5 layers with sizes 64, 64, 64, 128, 1024
assumptions (5)
- standard math Voxelization of [-1,1]^3 into S^3 boxes covers the input space.
- standard math Uniform quantization of values to L bits introduces bounded error.
- domain assumption Per-point MLP with shared weights computes a set of 3-variable functions, and max pooling over points yields the global feature.
- domain assumption The point-wise architecture can implicitly learn local structure via max pooling.
- ad hoc to paper Fine-tuning the head on the approximate features recovers accuracy lost to quantization.
Cite this review
Pith. "Pith review of Justlookup: One Millisecond Deep Feature Extraction for Point Clouds By Lookup Tables." pith.science (2026). https://pith.science/paper/G6HWT2BX
@misc{pith2026190808996,
author = {Pith},
title = {Pith review of: Justlookup: One Millisecond Deep Feature Extraction for Point Clouds By Lookup Tables},
year = {2026},
howpublished = {\url{https://pith.science/paper/G6HWT2BX}},
note = {Machine review of arXiv:1908.08996}
}
read the original abstract
Deep models are capable of fitting complex high dimensional functions while usually yielding large computation load. There is no way to speed up the inference process by classical lookup tables due to the high-dimensional input and limited memory size. Recently, a novel architecture (PointNet) for point clouds has demonstrated that it is possible to obtain a complicated deep function from a set of 3-variable functions. In this paper, we exploit this property and apply a lookup table to encode these 3-variable functions. This method ensures that the inference time is only determined by the memory access no matter how complicated the deep function is. We conduct extensive experiments on ModelNet and ShapeNet datasets and demonstrate that we can complete the inference process in 1.5 ms on an Intel i7-8700 CPU (single core mode), 32x speedup over the PointNet architecture without any performance degradation.
Reference graph
Works this paper leans on
-
[1]
Justlookup: One Millisecond Deep Feature Extraction for Point Clouds By Lookup Tables
INTRODUCTION Learning representations directly from 3D point clouds is very attractive considering the intrinsic advantages of being less sensitive to pose and light changes. Deep neural models have demonstrated powerful ability of learning complicated functions on point cloud tasks [1, 2, 3, 4]. However, such ability often comes at the cost of high compu...
work page Pith review arXiv 1908
-
[2]
RELATED WORK 2.1. Deep Learning on 3D Point Cloud PointNet [1] is the pioneer to apply deep neural networks to directly process unordered point clouds. To address the prob- lem, it adopted spatial transform networks and a symmetry function to maintain the invariance of permutation. After that, many recent works mainly focus on how to efficiently cap- ture ...
-
[3]
PROPOSED METHOD The main idea of our method is to train a point-wise architec- ture for point cloud to get the point-wise functionF and then Shared 𝒉𝟏 𝒙𝟏 𝒉𝟐 𝒙𝟏 …… 𝒉𝟏𝟎𝟐𝟒 𝒙𝟏 𝒉𝟏 𝒙𝟐 𝒉𝟐 𝒙𝟐 ...... 𝒉𝟏𝟎𝟐𝟒 𝒙𝟐 …… …… …… 𝒉𝟏 𝒙𝒏 𝒉𝟐 𝒙𝒏 ...... 𝒉𝟏𝟎𝟐𝟒 𝒙𝒏 n Inputn 3 MLP(64,64,64,128,1024) 1024 Max pool 10241 MLP (512,256,k) Inputn 3 𝒉𝟏 𝒙 , 𝒉𝟐 𝒙 , … , 𝒉𝟏𝟎𝟐𝟒(𝒙) Lookup Table ...
-
[4]
However, it still gains impressive results on 3D vision tasks
GEOMETRIC INTERPRETATION One concern with the point-wise architecture is that it lacks of the ability of capturing the local structure as the key pro- cessing step is point-wise without using any neighborhood in- formation explicitly. However, it still gains impressive results on 3D vision tasks. x 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 y 1.00 0.75 ...
-
[5]
EXPERIMENT In this section, we first describe implementation details and datasets used for training and testing our method. Next, we compare our method with a number of state-of-the-art meth- ods on different benchmark datasets for 3D object classifica- tion and retrieval tasks. Finally, we provide detailed experi- ments under different numbers of voxels an...
work page 2003
-
[6]
It is not applicable for recently proposed deep architectures such as DGCNN [3] and SO-Net [4]
LIMITATION Currently, our method requires that the deep architecture must be point-wise. It is not applicable for recently proposed deep architectures such as DGCNN [3] and SO-Net [4]. Consid- ering its simplicity and speedup performance, how to mine more local information from the point-wise architecture re- mains to be an attractive and challenging prob...
-
[7]
CONCLUSION In this paper, we propose to apply a classical lookup table to speed up the inference process for a particular deep architec- ture for 3D point cloud tasks. This architecture implements a deep function from a set of 3-variable functions by max pooling operation. Our method can ensure the inference time be determined by the memory access no matt...
-
[8]
Pointnet: Deep learning on point sets for 3d classification and segmentation,
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” Proc. Computer Vi- sion and Pattern Recognition (CVPR), IEEE , vol. 1, no. 2, pp. 4, 2017
work page 2017
Show all 24 references
-
[9]
Pointnet++: Deep hierarchical feature learning on point sets in a metric space,
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” in Advances in Neural Information Processing Systems, 2017, pp. 5099–5108
2017
-
[10]
Dy- namic graph cnn for learning on point clouds,
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon, “Dy- namic graph cnn for learning on point clouds,” arXiv preprint arXiv:1801.07829, 2018
2018 arXiv
-
[11]
So-net: Self-organizing network for point cloud analysis,
Jiaxin Li, Ben M Chen, and Gim Hee Lee, “So-net: Self-organizing network for point cloud analysis,” in Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, 2018, pp. 9397–9406
2018
-
[12]
Deep com- pression: Compressing deep neural networks with prun- ing, trained quantization and huffman coding,
Song Han, Huizi Mao, and William J Dally, “Deep com- pression: Compressing deep neural networks with prun- ing, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149, 2015
2015 arXiv
-
[13]
Exploiting linear structure within convolutional networks for efficient evaluation,
Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in Advances in neural information processing systems , 2014, pp. 1269–1277
2014
-
[14]
Do deep nets really need to be deep?,
Jimmy Ba and Rich Caruana, “Do deep nets really need to be deep?,” in Advances in neural information pro- cessing systems, 2014, pp. 2654–2662
2014
-
[15]
Exploiting the panorama representation for convolutional neural network classification and re- trieval,
Konstantinos Sfikas, Theoharis Theoharis, and Ioannis Pratikakis, “Exploiting the panorama representation for convolutional neural network classification and re- trieval,” in Eurographics Workshop on 3D Object Re- trieval. The Eurographics Association, 2017, vol. 8
2017
-
[16]
Deeppano: Deep panoramic representation for 3- d shape recognition,
Baoguang Shi, Song Bai, Zhichao Zhou, and Xiang Bai, “Deeppano: Deep panoramic representation for 3- d shape recognition,” IEEE Signal Processing Letters , vol. 22, no. 12, pp. 2339–2343, 2015
2015
-
[17]
Pvnet: A joint convolutional network of point cloud and multi-view for 3d shape recognition,
Haoxuan You, Yifan Feng, Rongrong Ji, and Yue Gao, “Pvnet: A joint convolutional network of point cloud and multi-view for 3d shape recognition,” in 2018 ACM Multimedia Conference on Multimedia Confer- ence. ACM, 2018, pp. 1310–1318
2018
-
[18]
Seqviews2seqlabels: Learn- ing 3d global features via aggregating sequential views by rnn with attention,
Zhizhong Han, Mingyang Shang, Zhenbao Liu, Chi- Man V ong, Yu-Shen Liu, Matthias Zwicker, Junwei Han, and CL Philip Chen, “Seqviews2seqlabels: Learn- ing 3d global features via aggregating sequential views by rnn with attention,” IEEE Transactions on Image Processing, vol. 28,...
2019
-
[19]
Ensemble of panorama-based convolu- tional neural networks for 3d model classification and retrieval,
Konstantinos Sfikas, Ioannis Pratikakis, and Theoharis Theoharis, “Ensemble of panorama-based convolu- tional neural networks for 3d model classification and retrieval,”Computers & Graphics, vol. 71, pp. 208–218, 2018
2018
-
[20]
Gift: A real-time and scalable 3d shape search engine,
Song Bai, Xiang Bai, Zhichao Zhou, Zhaoxiang Zhang, and Longin Jan Latecki, “Gift: A real-time and scalable 3d shape search engine,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, 2016, pp. 5023–5032
2016
-
[21]
3d shapenets: A deep representation for volumetric shapes,
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao, “3d shapenets: A deep representation for volumetric shapes,” in Proceedings of the IEEE conference on com- puter vision and pattern recognition , 2015, pp. 1912– 1920
2015
-
[22]
Shrec16 track large-scale 3d shape retrieval from shapenet core55,
Manolis Savva, Fisher Yu, Hao Su, M Aono, B Chen, D Cohen-Or, W Deng, Hang Su, Song Bai, Xiang Bai, et al., “Shrec16 track large-scale 3d shape retrieval from shapenet core55,” in Proceedings of the eurographics workshop on 3D object retrieval , 2016
2016
-
[23]
Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints,
Asako Kanezaki, Yasuyuki Matsushita, and Yoshifumi Nishida, “Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints,” in Proceedings of IEEE International Con- ference on Computer Vision and Pattern Recognition (CVPR), 2018
2018
-
[24]
Multi-view convolutional neural networks for 3d shape recognition,
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller, “Multi-view convolutional neural networks for 3d shape recognition,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 945–953
2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.