A Real-time 3D Desktop Display

Enrique Canessa; Livio Tenze

REVIEW 2 major objections 6 minor 13 references

A Real-time 3D Desktop Display

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The extended altiro3D library converts live 2D camera or desktop video into Native light-field streams for a Looking Glass Portrait at 10 frames per second on a laptop without CUDA.

desk verdict A useful, honest engineering extension of altiro3D to live streams, with a credible 10 fps claim but no evidence yet that the depth-based light fields look good on typical desktop content. read the letter →

arxiv 2506.08064 v1 pith:VW5I7UHP submitted 2025-06-09 cs.GR cs.CV

Livio Tenze , Enrique Canessa This is my paper

classification cs.GRcs.CV

keywords 3Dcomputervisionlight-fielddisplaymonoculardepthestimationMiDaSreal-timevideoprocessinglenticulardesktopGUIaltiro3D

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper extends the altiro3D C++ library so that a normal 2D camera feed, a video file, or any selected region of a computer desktop can be converted in real time into a Native light-field stream and shown on a lenticular, glasses-free 3D display such as the Looking Glass Portrait. The load-bearing result is measured in Section 4: on a Dell laptop with an i9-10885H CPU and a GTX 1650 Ti, the altiro3Drt tool acquires frames from the embedded camera at about 10 Hz (roughly 100 ms per frame) using the small MiDaS version 2.1 neural network, with no CUDA cores. If this holds, live 3D video conferencing, YouTube playback, and general desktop use become possible without per-application software. The authors claim the system is device-agnostic and works on standard Linux PCs, with a Windows adaptation also provided.

What carries the argument

The central object is the MiDaS CNN for monocular depth estimation, a convolutional neural network trained on large RGB datasets to predict a per-pixel depth map from a single image; it is the component that turns a flat frame into a depth-bearing scene. The other load-bearing mechanism is the pixel- and device-based Lookup Table (LUT) that maps the assembled Quilt of multiple views into the Native format of the lenticular display, without which the real-time constraint would not be met. The OpenCV parallel_for construct parallelizes the Quilt-to-Native mapping, and a multi-threaded ScreenRecorder class handles SNAP acquisition from the desktop.

What would settle it

Run the SNAP mode on a desktop filled with a plain, textureless application window (for example, a blank text editor or spreadsheet) and inspect the Native output on the Looking Glass display; if the rendered parallax shows severe artifacts such as bent text or warped window edges, or if the MiDaS depth map is essentially noise for such content, then the real-time 3D desktop display claim would not hold for common desktop use.

Watch

Extended reading notes

Core claim

The central claim of the paper is that real-time 2D-to-3D conversion is achievable for live video sources by combining monocular depth estimation with an optimized light-field mapping pipeline. Given a single RGB frame acquired from a USB camera (REAL mode) or a snapshot of a desktop region (SNAP mode), the extended altiro3D library runs the MiDaS convolutional neural network to produce a depth map, generates multiple viewpoints with either a geometric or a fast algorithm, assembles them into a Quilt, and maps the Quilt to the Native format of a Looking Glass Portrait using a precomputed lookup table. The profiling results show that the MiDaS inference is the dominant cost, and that on the tested hardware the whole chain fits within about 100 ms per frame without using CUDA. The authors therefore assert that altiro3Drt can acquire frames from the embedded camera in real-time with a framerate of 10 Hz while running a MiDaS small network.

Load-bearing premise

The pipeline assumes that the MiDaS depth map is accurate enough for the desktop content being rendered; for flat UI windows, icons, and text the depth estimates may be noisy or wrong, and the paper provides no evaluation of depth accuracy or end-to-end visual quality.

Editorial extensions

If this is right

A user can select any desktop region with the provided GUI and stream it live to a Looking Glass Portrait, so video calls, web browsers, and other running applications become 3D without modification.
Because the bottleneck is MiDaS inference, using a faster inference engine such as OpenVINO or a lighter depth model should push the frame rate above 10 Hz on the same hardware.
The system is device-agnostic, so the same code path can feed other lenticular or free-view displays once their calibration and LUT are provided.
The Windows build produced through the Docker cross-compilation pipeline means the real-time 3D desktop feature is not confined to Linux.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The depth accuracy of MiDaS on flat UI elements is an untested risk; a natural next experiment is to compare MiDaS depth on synthetic screenshots with known ground-truth window ordering, or to run a user study on perceived depth quality.
The 10 Hz figure is measured on a specific laptop with the small MiDaS network; other machines or larger Quilt masks may drop below real-time, so a portable benchmark across CPUs and GPUs would clarify how general the result is.
Because the pipeline is built around a LUT, changing the display geometry only requires regenerating the map, suggesting a path to supporting multiple lenticular displays with different view counts from the same codebase.
The paper does not measure end-to-end latency from camera to screen, so it is left implicit whether the 100 ms per frame processing time translates into perceptible lag for interactive desktop use.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a solid second-version engineering paper from the altiro3D authors. What is actually new is the real-time acquisition path—live webcam and desktop region capture, converted to Looking Glass Native light field streams at about 10 fps on a consumer laptop without CUDA, plus a Qt GUI and cross-platform builds. The code is public and the profiling data (Table 3) is consistent with the claimed bottleneck being MiDaS inference. The performance number is internally plausible and not the weak link.

The real gap is visual quality. The paper states that MiDaS produces an 'accurate depth map' (Sec. 3.1) but reports no depth error, no comparison to ground truth, and no artifact or user evaluation. Desktop content is mostly flat UI elements, text, and icons; monocular depth on such content is not obviously reliable, and artifacts get amplified by parallax view synthesis and the lenticular display. The abstract promises 'realistic 3D experiences'—that claim is unsupported by the evidence presented. This is an evidence gap, not an internal inconsistency. The architecture could work; we just don't know.

Secondary issues: the performance is a single-point measurement on one laptop (i9-10885H, GTX 1650 Ti) with a decimation factor of 8 and the MiDaS small model. The paper says 'any standard PC' in the final remarks, but only one configuration is tested. The CUDA comparison (Table 2) is useful but no error bars or repeats. Self-citations to the authors' prior work are appropriate here—the LUT and view synthesis are genuinely from those papers, and the new real-time result is measured directly.

For a reader: this is a useful reference implementation for anyone building a real-time 2D-to-light-field pipeline on consumer hardware. The paper is short, clear, and honest about its scope in the final remarks. It deserves a serious referee—the engineering is real and the missing evaluation is fixable. I'd ask the authors to add a qualitative/quantitative visual quality check on representative desktop content (browser, file manager, video) and to report timing across at least two hardware configs and a couple of decimation settings.

Final: send it to review, but require the visual validation before acceptance. It's not a breakthrough, but it's a legitimate systems contribution.

Referee Report

2 major / 6 minor

Summary. The paper presents an extended version of the altiro3D C++ library, adding real-time processing of 2D video streams (from USB cameras, video files, or a selected desktop region) and converting them into Native-format light fields for a lenticular display such as the Looking Glass Portrait. The pipeline uses MiDaS for monocular depth estimation, OpenCV-based view synthesis, a precomputed lookup table for quilt-to-native mapping, and OpenCV's parallel_for for speed. A Qt GUI (altiro3Dlive) simplifies selecting a screen region and configuring the output, and a Docker-based cross-compilation environment produces Windows installers. The central reported result is that altiro3Drt achieves approximately 100 ms/frame (10 Hz) on a Dell laptop with an i9-10885H and GTX 1650 Ti, without CUDA, with profiling showing MiDaS inference as the dominant bottleneck.

Significance. If the reported performance and output quality hold, the system would be a practical, low-cost way to turn ordinary 2D desktop content or webcam video into a real-time light-field stream on a commercial display. The paper's strengths include releasing the source code, providing a Dockerized build for cross-compilation, reporting a concrete profiling table (Table 3), and identifying the exact bottleneck (MiDaS inference). These are useful, reproducible contributions. The main weakness is that the paper measures only throughput, not the quality of the resulting depth maps or the visual fidelity of the light-field output; because depth errors are amplified by parallax view synthesis and the lenticular display, the end-to-end 'realistic 3D' claim is currently unsupported.

major comments (2)

[Section 4, Table 2] The central real-time claim rests on a single measurement on a single hardware configuration: one Dell laptop (i9-10885H, GTX 1650 Ti) with '~100 ms/frame' and no repeated runs, standard deviation, or frame-count details. Section 5 then generalizes to 'any standard PC.' To support the load-bearing real-time claim, the paper should either provide repeated measurements with error bars across at least a second hardware configuration, or explicitly restrict the claim to the tested machine and configuration.
[Section 3.1 and Section 5] No evidence is provided that MiDaS depth estimates are accurate for desktop UI content. The workflow in Section 3.1 states that MiDaS 'produce[s] an accurate depth map of every acquired image,' and Section 5 claims 'realistic 3D experiences' for YouTube videos and video conferencing, but the paper reports no depth-error metric, no comparison against ground truth or alternative depth methods, and no qualitative or user evaluation of the displayed light field. Since flat windows, text, icons, and browser pages are far from MiDaS's natural-image training distribution, and since parallax synthesis and the lenticular display amplify depth artifacts, this missing evaluation is a load-bearing gap in the end-to-end claim, even though the raw throughput measurement itself is plausible.

minor comments (6)

[Section 3.2] The phrase 'Native encoded flames' should read 'Native encoded frames' (typo).
[Section 1] The phrase 'wear a ad-hoc device' should be 'wear an ad-hoc device'.
[Section 4] The term 'decimation factor' appears in Table 2 but is never defined in the text; please clarify what it means for input resolution and how a factor of 8 was chosen.
[Section 4] The profiling table (Table 3) is reported for a single run on Ubuntu 23.10/24.04; please state how many frames were profiled and whether the reported exclusive/inclusive CPU seconds are representative across runs.
[Section 3.3] The GUI description would benefit from a small workflow example (e.g., a screenshot showing the selected desktop region and the resulting Native-format output) to help readers see the end-to-end effect without running the software.
[Section 5] The statement that the system 'performs satisfactorily on any standard PC running Linux' is stronger than the evidence in Section 4; please replace it with a claim scoped to the tested hardware or add supporting measurements.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the real-time throughput claim rests on direct measurements, not on self-citation or fitted inputs.

full rationale

The paper's derivation chain is acquisition → MiDaS depth estimation → view synthesis (FAST/REAL) → LUT-based Quilt-to-Native mapping → display. The central claim, 10 Hz real-time conversion on a GTX 1650 Ti without CUDA, is supported by directly measured per-frame timings (Section 4, Table 2) and by profiling data (Table 3) that identify MiDaS inference as the bottleneck. No parameter is fitted to a target output and then presented as a prediction. The self-citations to the authors' earlier altiro3D paper [2] and Morpholo paper [4] describe the view-synthesis and LUT components, but these are implementation components with available source code; the current paper independently measures the end-to-end pipeline, so the self-citations are not load-bearing in the sense of making the result true by definition. MiDaS itself is cited to external work [3]. The absence of depth-accuracy or end-to-end visual-quality evaluation is a genuine evidence gap, but it is a completeness or validation issue, not circularity. Therefore no circular step can be exhibited, and the score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper's central result is an engineering throughput measurement; it introduces no new mathematical model or physical entity. The main exogenous assumptions are the accuracy of the MiDaS depth backbone, the correctness of the Looking Glass calibration JSON, and the ability of FFmpeg screen capture to keep up. The free parameters are user-chosen configuration settings that materially affect the measured frame rate.

free parameters (3)

Decimation factor = 8 (used in reported tests)
Sub-sampling factor selected by the user; it directly trades throughput against output resolution and is part of the configuration that yields ~100 ms/frame.
Quilt mask (views grid) = 6 x 8 (48 views)
Device-dependent number of views packed into the quilt; larger masks increase view synthesis and mapping cost.
MiDaS model variant = small v2.1
The small network is chosen for speed; a larger MiDaS model would increase inference time, so the 10 fps claim is tied to this variant.

assumptions (3)

domain assumption MiDaS estimates an accurate enough depth map for arbitrary 2D content, including desktop UI.
The entire pipeline begins with the MiDaS depth map (Section 3.1); no evaluation of depth quality on screen-captured content is provided.
domain assumption The Looking Glass Portrait calibration JSON correctly defines the lenticular mapping, making the generated LUT valid.
The GUI constructs the LUT from calibration JSON pasted by the user (Section 3.3), and the paper does not analyze calibration errors.
domain assumption Screen capture via FFmpeg can deliver frames without significant drops at the target rate.
Real-time SNAP acquisition is assumed to keep up; no frame-drop statistics are reported (Section 3.2).

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Real-time 3D Desktop Display." pith.science (2026). https://pith.science/paper/VW5I7UHP

@misc{pith2026250608064,
  author       = {Pith},
  title        = {Pith review of: A Real-time 3D Desktop Display},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VW5I7UHP}},
  note         = {Machine review of arXiv:2506.08064}
}

read the original abstract

A new extended version of the altiro3D C++ Library -- initially developed to get glass-free holographic displays starting from 2D images -- is here introduced aiming to deal with 3D video streams from either 2D webcam images or flat video files. These streams are processed in real-time to synthesize light-fields (in Native format) and feed realistic 3D experiences. The core function needed to recreate multiviews consists on the use of MiDaS Convolutional Neural Network (CNN), which allows to extract a depth map from a single 2D image. Artificial Intelligence (AI) computing techniques are applied to improve the overall performance of the extended altiro3D Library. Thus, altiro3D can now treat standard images, video streams or screen portions of a Desktop where other apps may be also running (like web browsers, video chats, etc) and render them into 3D. To achieve the latter, a screen region need to be selected in order to feed the output directly into a light-field 3D device such as Looking Glass (LG) Portrait. In order to simplify the acquisition of a Desktop screen area by the user, a multi-platform Graphical User Interface has been also implemented. Sources available at: https://github.com/canessae/altiro3D/releases/tag/2.0.0

Figures

Figures reproduced from arXiv: 2506.08064 by the authors.

**Figure 1.** Diagram of the processes involved in the altiro3D Library –for details see [2]. The system is now able to acquire frames from a USB camera, to produce the views and to map the output directly in the Native display format. Now the altiro3D system can manage multiple displays: a new feature of the current system is the possibility to generate a video stream (altiro3Drt, altiro3Dlive and altiro3Dlive-cli) from a region… view at source ↗

**Figure 2.** Main GUI with two tabs: setup and processing. (a) Setup tab. (b) Processing tab. The GUI provides users an easy method to setup (Fig. 2a) and to manage the real-time conversion (Fig. 2b). In the setup tab Fig. 2a the user can choose between • Generate MAP from calibration • Use existing MAP file If the user selects the first choice, a dialog is shown (Fig. 3a) which can be filled-in to create the LUT map file for fu… view at source ↗

**Figure 3.** Menus of the altiro3Dlive GUI. (a) Generate MAP from calibration dialog. (b) Use existing MAP file dialog. (c) Video properties dialog. Instead, with the dialog in Fig. 3b, the user can re-use an existing MAP file: the user has to choose the right Quilt mask (used when that MAP file has been generated). The system will use that file in the processing tab. The processing tab (Fig. 2b) is used to configure the acquisi… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Processing tab in action with multiple displays. (a) Screen 1 and (b) Screen 2 with altiro3Dlive. The user can move the yellow region within the screen using drag and drop or he can manually setup the position, the size of the yellow region and the screen’s number with…

**Figure 5.** Figure 5: REAL (left) and SNAP (right) acquisition mode. (a) altiro3Drt in action: the stream from camera feeds the external holographic display. (b) altiro3Dlive in action: a region of the right screen is converted to the Native format and the real-time stream feeds the LFD. On…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

[1]

Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements

Ban S. et al., "Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements", Biosensors 12 (2022) 1039-1084. doi: 10.3390/bios12111039

work page doi:10.3390/bios12111039 2022
[2]

altiro3d: scene representation from single image and novel view synthesis

Tenze L., Canessa E., "altiro3d: scene representation from single image and novel view synthesis", Int. j. inf. tecnol. 16, 33–42 (2024). doi: 10.1007/s41870-023-01590-3 10 of 10

work page doi:10.1007/s41870-023-01590-3 2024
[3]

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross- dataset Transfer

Ranftl R. et al., "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross- dataset Transfer" IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2022) 1623-1637. doi: 10.1109/TPAMI.2020.3019967 –see also MiDaS github repository: https://github.com/intel-isl/MiDaS (Last visited 04/06/2025)

work page arXiv 2022
[4]

Morpholo: A Hologram Generator Algorithm

Canessa E., Tenze L., "Morpholo: A Hologram Generator Algorithm", Electronic Imaging 53 (2020) 53-1-53-5. doi: 10.2352/ISSN.2470-1173.2020.2.SDA-053

arXiv 2020
[5]

Looking Glass (LG) Portrait Device: https://lookingglassfactory.com/looking-glass-portrait (Last visited 04/06/2025)

work page 2025
[6]

3D User Interfaces: New Directions and Perspectives

Bowman D.A. et al., "3D User Interfaces: New Directions and Perspectives", IEEE Computer Graphics and Applications 28 (2008) 20-36. doi: 10.1109/MCG.2008.109

work page doi:10.1109/mcg.2008.109 2008
[7]

3D Displays: Their Evolution, Inherent Challenges & Future Perspectives

Pan X. et al., "3D Displays: Their Evolution, Inherent Challenges & Future Perspectives", Proceed. Future Technologies Conference (FTC), Lecture Notes in Networks and System 3 (2021) 397-415. doi: 10.1007/978-3- 030-89912-7_31

doi:10.1007/978-3- 2021
[8]

Screen Recorder Qt

Cheinasso F., Comparetto A., Cannarella A., The project "Screen Recorder Qt": https://github.com/cheina97/ScreenRecorderQT.git (Last visited 04/06/2025)

work page 2025

Show all 13 references

[9]

Docker Community Edition: https://docs.docker.com/get-docker/ (Last visited 04/06/2025)

2025
[10]

MinGW-264 –a complete runtime environment for GCC and LLVM for 32 and 64 bit Windows: https://www.mingw-w64.org/ (Last visited 04/06/2025)

2025
[11]

Nullsoft Scriptable Install System: https://nsis.sourceforge.io/Main_Page (Last visited 04/06/2025)

2025
[12]

GNU Gprofng: https://sourceware.org/binutils/docs-2.42/gprofng.html (Last visited 04/06/2025)

2025
[13]

OpenVINO toolkit: https://www.intel.com/content/www/us/en/developer/tools/openvino- toolkit/overview.html (Last visited 04/06/2025)

2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements

Ban S. et al., "Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements", Biosensors 12 (2022) 1039-1084. doi: 10.3390/bios12111039

work page doi:10.3390/bios12111039 2022

[2] [2]

altiro3d: scene representation from single image and novel view synthesis

Tenze L., Canessa E., "altiro3d: scene representation from single image and novel view synthesis", Int. j. inf. tecnol. 16, 33–42 (2024). doi: 10.1007/s41870-023-01590-3 10 of 10

work page doi:10.1007/s41870-023-01590-3 2024

[3] [3]

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross- dataset Transfer

Ranftl R. et al., "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross- dataset Transfer" IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2022) 1623-1637. doi: 10.1109/TPAMI.2020.3019967 –see also MiDaS github repository: https://github.com/intel-isl/MiDaS (Last visited 04/06/2025)

work page arXiv 2022

[4] [4]

Morpholo: A Hologram Generator Algorithm

Canessa E., Tenze L., "Morpholo: A Hologram Generator Algorithm", Electronic Imaging 53 (2020) 53-1-53-5. doi: 10.2352/ISSN.2470-1173.2020.2.SDA-053

arXiv 2020

[5] [5]

Looking Glass (LG) Portrait Device: https://lookingglassfactory.com/looking-glass-portrait (Last visited 04/06/2025)

work page 2025

[6] [6]

3D User Interfaces: New Directions and Perspectives

Bowman D.A. et al., "3D User Interfaces: New Directions and Perspectives", IEEE Computer Graphics and Applications 28 (2008) 20-36. doi: 10.1109/MCG.2008.109

work page doi:10.1109/mcg.2008.109 2008

[7] [7]

3D Displays: Their Evolution, Inherent Challenges & Future Perspectives

Pan X. et al., "3D Displays: Their Evolution, Inherent Challenges & Future Perspectives", Proceed. Future Technologies Conference (FTC), Lecture Notes in Networks and System 3 (2021) 397-415. doi: 10.1007/978-3- 030-89912-7_31

doi:10.1007/978-3- 2021

[8] [8]

Screen Recorder Qt

Cheinasso F., Comparetto A., Cannarella A., The project "Screen Recorder Qt": https://github.com/cheina97/ScreenRecorderQT.git (Last visited 04/06/2025)

work page 2025