REVIEW 3 major objections 5 minor 32 references
Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read By filtering unreliable depth data and pruning empty surface blocks, a live telepresence pipeline can stream one captured scene to more than 24 exploration clients at once.
desk verdict A solid systems paper with a genuine scalability win, but the >24-client claim is factually wrong for one scene and the benchmark clients omit the rendering pipeline; worth a revised version, not as printed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a three-part filtering pipeline around a sparse voxel-block truncated signed distance field (TSDF) scene representation. A depth-discontinuity filter discards depth samples whose values deviate strongly from their 7x7 neighborhood or whose neighborhood has too many missing measurements; a virtual downsampling step during voxel block allocation considers only every $c_a$-th pixel so fewer empty blocks are created; and a Marching Cubes voxel block pruning step on the server drops blocks whose MC index is $0$ or $255$ or whose TSDF fusion weight falls below a threshold, so only reliable surface blocks enter the stream. Together these filters shrink the streaming model and the update set, which is what raises client capacity.
What would settle it
Run the described server with 24 actual exploration clients that render the received blocks in VR on separate computers over a realistic network, and count how many can keep their local model within the one-second sync delay used in the paper; if that number is below 24, the claimed capacity does not transfer to real immersive use.
Extended reading notes
Core claim
The central claim is that filtering unreliable depth pixels before fusion, allocating voxel blocks from a virtually downsampled depth image, and pruning empty or low-weight Marching Cubes blocks in the server's streaming model collectively reduce the server's model size and the client request rate enough that the same pipeline that handled 3-5 clients can handle more than 24. In the evaluation, the full system lowered the required request rate to 12 Hz for most scenes and cut memory footprint by 25-60% compared with the base approach. The paper presents this as an optimization of an existing pipeline rather than a new reconstruction algorithm, and it demonstrates that each individual filter contributes a measurable gain in client count, latency, or visual quality.
Load-bearing premise
The measured client ceiling assumes that a benchmark client that discards every received block and runs on the same computer as all other benchmark clients behaves like a real exploration client that renders the scene on its own hardware.
Editorial extensions
If this is right
- More than 24 users can explore the same live-captured scene at once on consumer hardware, supporting classroom, entertainment, and remote-collaboration settings.
- The required client request rate drops to about 12 Hz for most tested scenes, so server bandwidth and per-request overhead are no longer the bottleneck they were at 100 Hz.
- Reconstructed models become noticeably more compact, with reported memory reductions of 25-60%, while visual artifacts around depth discontinuities and poorly observed regions are reduced.
- The same filters apply to standalone volumetric reconstruction, not just telepresence, improving runtime and memory there too.
- Because latency between server and client stays low across the capture sequence, reconnecting clients can catch up quickly after network interruptions.
Reading between the lines
- If real rendering clients behave like the benchmark discard clients, the 24-client number is likely optimistic; actual immersive users add GPU load and different request timing, so field deployments should expect a lower ceiling until re-measured.
- The pruning threshold and downsampling factor are fixed empirically in the paper; an adaptive policy that tunes $c_w$ and $c_a$ per scene or per depth range could preserve the scalability gains while recovering some of the holes that aggressive downsampling can cause at long range.
- The same block-level pruning idea could be applied to collaborative multi-client reconstruction, where submap fusion would benefit from ignoring unstable blocks without deleting geometry that later revisits might stabilize.
- A natural stress test is to combine this server-side pruning with a real VR client on a laptop-class GPU, measuring both network and render latency under the same 12 Hz request rate.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes three optimizations to a SLAMCast-style volumetric telepresence pipeline: filtering depth discontinuities, virtual downsampling of voxel block allocation, and pruning of empty or unstable Marching Cubes voxel blocks on the server. Combined, these are claimed to allow more than 24 concurrent exploration clients in the same live-captured scene, about a factor of 6 more than the prior SLAMCast system, without additional latency or hardware changes. The claims are evaluated on five datasets using benchmark clients that discard received data.
Significance. The engineering direction is valuable: the proposed filters are simple, described concretely, and the ablation (B, B+DDF, B+VBAD, B+MCVBP, Ours) shows a clear stepwise reduction in model size, bandwidth, and latency. If the scalability claim were validated with realistic clients, the system would be a practical contribution to group-scale VR telepresence on consumer hardware. The paper also demonstrates the benefit of the filters for standalone volumetric reconstruction. However, the evaluation methodology currently prevents the headline result from being accepted as stated.
major comments (3)
- [Abstract, Section 6.1, Table 1] The Abstract, Section 6.1, and Conclusion claim that the system supports 'more than 24 clients in all scenes', but Table 1 reports a maximum of 18 exploration clients for the copyroom dataset under Ours. The unqualified claim is therefore not supported by the data; the authors should either qualify the claim per dataset or explain why copyroom is exceptional.
- [Section 6.1, Evaluation setup] The scalability and latency results are obtained with benchmark clients that discard all received data and run on the same computer as the other benchmark clients. This removes client-side mesh integration, rendering, and display, as well as the realistic request timing induced by rendering load. Consequently, the measured maximum of more than 24 clients and the claim of 'without introducing further latency' apply only to non-rendering clients, not to the immersive VR clients the system is designed for. The authors should either measure with at least one realistic rendering client or provide evidence that the discard benchmark is a faithful proxy (e.g., compare request timing and client-side resource usage).
- [Section 6, Filter tuning] The filter thresholds (cd, ch, ca, cw) are 'determined empirically using several datasets' (Section 6), and the same datasets are used for the reported evaluation. Without a held-out validation set or a sensitivity analysis, the reported improvements may be optimistic and the transferability of the chosen thresholds is unclear. Please include a sensitivity analysis or validation on at least one held-out sequence.
minor comments (5)
- [Section 6.1] The sentence 'which allows for running all benchmark clients on a single computer without an overhead' is vague; please specify what overhead is avoided (e.g., no rendering, no mesh integration, no display costs).
- [Table 1 caption] The notation 'M MC/P MC A (M T SDF)' is not explained; please add a sentence clarifying that the two numbers under Model Size correspond to the MC model size and the update-set size, with the TSDF size in parentheses.
- [Section 6.2, Figures 4 and 5] The label 'S MC US' is used but not defined; please define it as the update set size P_MC_A to avoid confusion.
- [Section 4.1, Eq. (1)] The set-builder condition mixes an existential quantifier over neighbors with a condition on the whole neighborhood; please clarify the logical precedence or split the condition into two explicit clauses.
- [Section 6.3] The reported improvements of '10-40%' for runtime and '25-60%' for memory would be more useful if accompanied by a per-dataset table with exact values.
Circularity Check
No significant circularity: the paper reports empirical engineering optimizations measured against its own prior system as a baseline, with no load-bearing reduction of a derived claim to its inputs.
full rationale
The paper's contributions are three concrete algorithmic filters applied to reconstruction and server streaming components: depth-discontinuity filtering with user thresholds c_h and c_d, virtual downsampling at voxel block allocation with factor c_a, and pruning of empty or unstable Marching Cubes blocks with weight threshold c_w. Each contribution is evaluated empirically in Table 1 and Figures 4-6 against a Base variant described as equivalent to the authors' prior SLAMCast system. The reported maximum exploration-client counts are measurements taken under stated benchmark conditions, not quantities derived from the thresholds or from SLAMCast. The thresholds are admittedly tuned empirically ('The filter sizes and thresholds as described above were determined empirically using several datasets'), which is ordinary system parameter selection rather than a fitted parameter being renamed as a prediction. The comparison baseline [27] is the authors' own prior work, and the paper does rely on it as the foundation, but this is a self-citation that serves as an experimental baseline and implementation substrate, not as an unverified theorem that forces the outcome. The scalability headline is vulnerable to an evaluation-scope criticism: the benchmark clients discard received data and run on one machine, so the claimed 'more than 24 clients' may not generalize to rendering, immersive VR clients. That is an empirical validity concern about what was demonstrated, not a circularity in which a prediction is equivalent to an input by construction. No equation in the paper reduces a claimed result to its own definition, no fitted parameter is relabeled as a prediction, and no uniqueness or ansatz argument is imported from the authors' prior work to forbid alternatives. The analysis is therefore self-contained in the sense relevant to circularity: every quantitative claim is backed by direct measurement against a stated baseline, and the improvements are not forced by the way the system or metrics are defined.
Assumptions & free parameters
free parameters (5)
- cd =
0.2 m
- ch =
0.25
- ca =
4
- cw =
2.0
- Client request rate =
12 Hz to 200 Hz per dataset and variant
assumptions (6)
- domain assumption Depth pixels at strong discontinuities or with many missing neighbors are unreliable outliers and can be discarded without harming reconstruction.
- domain assumption Virtually downsampling the depth image during voxel block allocation, while fusing the full image, does not prevent allocation of the blocks needed for a complete model.
- domain assumption MC voxel blocks whose Marching Cubes index is 0 or 255, or whose TSDF fusion weight is below cw, can be ignored or pruned without losing reliable geometry.
- domain assumption A benchmark client that discards all received data and runs on one machine is a faithful proxy for real exploration clients.
- domain assumption The scene is static and captured by a single moving RGB-D camera, as in SLAMCast.
- standard math Volumetric TSDF fusion with voxel block hashing and Marching Cubes extraction works as described in the cited prior work.
Cite this review
Pith. "Pith review of Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence." pith.science (2026). https://pith.science/paper/CTM7X727
@misc{pith2026190803118,
author = {Pith},
title = {Pith review of: Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence},
year = {2026},
howpublished = {\url{https://pith.science/paper/CTM7X727}},
note = {Machine review of arXiv:1908.03118}
}
read the original abstract
Sharing live telepresence experiences for teleconferencing or remote collaboration receives increasing interest with the recent progress in capturing and AR/VR technology. Whereas impressive telepresence systems have been proposed on top of on-the-fly scene capture, data transmission and visualization, these systems are restricted to the immersion of single or up to a low number of users into the respective scenarios. In this paper, we direct our attention on immersing significantly larger groups of people into live-captured scenes as required in education, entertainment or collaboration scenarios. For this purpose, rather than abandoning previous approaches, we present a range of optimizations of the involved reconstruction and streaming components that allow the immersion of a group of more than 24 users within the same scene - which is about a factor of 6 higher than in previous work - without introducing further latency or changing the involved consumer hardware setup. We demonstrate that our optimized system is capable of generating high-quality scene reconstructions as well as providing an immersive viewing experience to a large group of people within these live-captured scenes.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
J. Chen, D. Bautembach, and S. Izadi. Scalable Real-time V olumetric Surface Reconstruction. ACM Trans. Graph., 32:113:1–113:16, 2013
work page 2013
-
[2]
A. Dai, M. Nießner, M. Zollh¨ofer, S. Izadi, and C. Theobalt. Bundle- Fusion: Real-time Globally Consistent 3D Reconstruction using On- the-fly Surface Reintegration. ACM Trans. Graph., 36(3):24, 2017
work page 2017
- [3]
-
[4]
A. J. Fairchild, S. P. Campion, A. S. Garc´ıa, R. Wolff, T. Fernando, and D. J. Roberts. A Mixed Reality Telepresence System for Collaborative Space Operation. IEEE Trans. on Circuits and Systems for Video Technology, 27(4):814–827, 2016
work page 2016
- [5]
-
[6]
S. Golodetz, T. Cavallari, N. A. Lord, V . A. Prisacariu, D. W. Murray, and P. H. S. Torr. Collaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation. IEEE Trans. on Visualiza- tion and Computer Graphics , 24(11):2895–2905, Nov 2018
work page 2018
- [7]
- [8]
Show all 32 references
-
[9]
Izadi et al
S. Izadi et al. KinectFusion: Real-time 3D Reconstruction and Inter- action Using a Moving Depth Camera. In Proc. of the ACM Symp. on User Interface Software and Technology, pp. 559–568, 2011
2011
-
[10]
Jones et al
B. Jones et al. RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector-camera Units. In Proc. of the Annual Symp. on User Interface Software and Technology, pp. 637–644, 2014
2014
-
[11]
K¨ahler, V
O. K¨ahler, V . A. Prisacariu, and D. W. Murray. Real-Time Large-Scale Dense 3D Reconstruction with Loop Closure. In European Conference on Computer Vision, pp. 500–516, 2016
2016
-
[12]
K¨ahler, V
O. K¨ahler, V . A. Prisacariu, C. Y . Ren, X. Sun, P. Torr, and D. Murray. Very High Frame Rate V olumetric Integration of Depth Images on Mobile Devices. IEEE Trans. on Visualization and Computer Graphics, 21(11):1241–1250, 2015
2015
-
[13]
K¨ahler, V
O. K¨ahler, V . A. Prisacariu, J. P. C. Valentin, and D. W. Murray. Hierar- chical V oxel Block Hashing for Efficient Integration of Depth Images. In IEEE Robotics and Automation Letters , pp. 1(1):192–197, 2016
2016
-
[14]
Keller, D
M. Keller, D. Lefloch, M. Lambers, S. Izadi, T. Weyrich, and A. Kolb. Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion. In Proc. of Joint 3DIM/3DPVT Conference, p. 8, 2013
2013
-
[15]
W. E. Lorensen and H. E. Cline. Marching Cubes: A High Resolution 3D Surface Construction Algorithm. In Proc. of the 14th Annual Conf. on Computer Graphics and Interactive Techniques , pp. 163–169, 1987
1987
-
[16]
Maier, R
R. Maier, R. Schaller, and D. Cremers. Efficient Online Surface Correc- tion for Real-time Large-Scale 3D Reconstruction. In British Machine Vision Conference (BMVC), 2017
2017
-
[17]
Maimone, J
A. Maimone, J. Bidwell, K. Peng, and H. Fuchs. Enhanced personal autostereoscopic telepresence system using commodity depth cameras. Computers & Graphics, 36(7):791 – 807, 2012
2012
-
[18]
Maimone and H
A. Maimone and H. Fuchs. Real-time volumetric 3D capture of room- sized scenes for telepresence. In Proc. of the 3DTV-Conference, 2012
2012
-
[19]
Molyneaux, S
D. Molyneaux, S. Izadi, D. Kim, O. Hilliges, S. Hodges, X. Cao, A. Butler, and H. Gellersen. Interactive Environment-Aware Handheld Projectors for Pervasive Computing Spaces. In Proc. of the Int. Conf. on Pervasive Computing, pp. 197–215, 2012
2012
-
[20]
Mossel and M
A. Mossel and M. Kr¨oter. Streaming and exploration of dynamically changing dense 3d reconstructions in immersive virtual reality. In Proc. of IEEE Int. Symp. on Mixed and Augmented Reality , pp. 43–48, 2016
2016
-
[21]
R. A. Newcombe et al. KinectFusion: Real-Time Dense Surface Mapping and Tracking. In Proc. of IEEE Int. Symp. on Mixed and Augmented Reality. IEEE, 2011
2011
-
[22]
R. A. Newcombe, D. Fox, and S. M. Seitz. DynamicFusion: Recon- struction and tracking of non-rigid scenes in real-time. In IEEE Conf. on Computer Vision and Pattern Recognition, pp. 343–352, 2015
2015
-
[23]
Nießner, M
M. Nießner, M. Zollh¨ofer, S. Izadi, and M. Stamminger. Real-time 3D Reconstruction at Scale Using V oxel Hashing. ACM Trans. Graph., 32(6):169:1–169:11, 2013
2013
-
[24]
Orts-Escolano et al
S. Orts-Escolano et al. Holoportation: Virtual 3D Teleportation in Real-time. In Proc. of the Annual Symp. on User Interface Software and Technology, pp. 741–754, 2016
2016
-
[25]
Reichl, J
F. Reichl, J. Weiss, and R. Westermann. Memory-Efficient Interac- tive Online Reconstruction From Depth Image Streams. Computer Graphics F orum, 35(8):108–119, 2016
2016
-
[26]
Roth and M
H. Roth and M. V ona. Moving volume kinectfusion. In Proc. of the British Machine Vision Conference, pp. 112.1–112.11, 2012
2012
-
[27]
Stotko, S
P. Stotko, S. Krumpen, M. B. Hullin, M. Weinmann, and R. Klein. SLAMCast: Large-Scale, Real-Time 3D Reconstruction and Stream- ing for Immersive Multi-Client Live Telepresence. IEEE Trans. on Visualization and Computer Graphics, 25(5):2102–2112, 2019
2019
-
[28]
Vasudevan, G
R. Vasudevan, G. Kurillo, E. Lobaton, T. Bernardin, O. Kreylos, R. Ba- jcsy, and K. Nahrstedt. High-Quality Visualization for Geographically Distributed 3-D Teleimmersive Applications. IEEE Trans. on Multime- dia, 13(3):573–584, 2011
2011
-
[29]
Whelan, H
T. Whelan, H. Johannsson, M. Kaess, J. J. Leonard, and J. McDonald. Robust Real-Time Visual Odometry for Dense RGB-D Mapping. In IEEE Int. Conf. on Robotics and Automation , pp. 5724–5731, 2013
2013
-
[30]
Whelan, M
T. Whelan, M. Kaess, M. Fallon, H. Johannsson, J. Leonard, and J. McDonald. Kintinuous: Spatially Extended KinectFusion. In RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras , 2012
2012
-
[31]
Whelan, M
T. Whelan, M. Kaess, H. Johannsson, M. Fallon, J. J. Leonard, and J. McDonald. Real-time large-scale dense RGB-D SLAM with volu- metric fusion. The Int. Journal of Robotics Research , 34(4-5):598–626, 2015
2015
-
[32]
Zhou and V
Q.-Y . Zhou and V . Koltun. Dense Scene Reconstruction with Points of Interest. ACM Trans. Graph., 32(4):112, 2013
2013
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.