Pith. sign in

REVIEW 4 major objections 5 minor 51 references

MARMOT: Masked Autoencoder for Modeling Transient Imaging

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MARMOT is a masked autoencoder pretrained on a million synthetic non-line-of-sight transients that completes missing measurements from arbitrary sparse scans and transfers its features to reconstruction, classification, albedo, and depth…

desk verdict Solid empirical paper: standard MAE applied to NLOS transients with a large synthetic dataset; the arbitrary-masking claim runs ahead of the evidence. read the letter →

arxiv 2506.08470 v1 pith:SL53FGKN submitted 2025-06-10 cs.CV

Siyuan Shen , Ziheng Wang , Xingyue Peng , Suan Xia , Ruiqian Li , Shiying Li , Jingyi Yu This is my paper
classification cs.CV
keywords maskedautoencodertransientimagingnon-line-of-sightself-supervisedpretrainingcompletionNLOSreconstructionsyntheticdatasettransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Transient imaging records photon arrival times for light that has bounced off hidden objects, and recovering those objects from sparse scans is a hard inverse problem. This paper proposes MARMOT, a masked autoencoder that learns to fill in missing transients in a self-supervised way. Pretrained on TransVerse, a synthetic dataset of one million confocal transients rendered from 500K 3D models, MARMOT reconstructs the full transient field from as little as 5% of the measurements. The paper claims that the completed transients and the encoder's latent features transfer to NLOS reconstruction, classification, albedo estimation, and depth estimation, outperforming prior physics-based and learning-based methods. If true, this would give the NLOS community a reusable pretrained model and dataset, reducing the need for dense and regular scanning.

What carries the argument

The central object is the scanning pattern mask (SPM): a random masking of the three-dimensional spatial-temporal transient volume that mimics arbitrary scanning patterns across the relay surface. MARMOT is a Transformer-based encoder-decoder where the encoder sees only the unmasked tokens (for example, 5% of the full grid), the decoder appends a learned mask token to the encoded features and regresses the missing transients, and training minimizes an $\ell^2$ loss on masked positions, following the masked-autoencoder paradigm. The other load-bearing component is TransVerse, a synthetic dataset of one million confocal transients rendered with a graphics-library-based renderer and a SPAD noise model, which provides the diverse pretraining signal that the paper argues transfers to real and structured measurements.

What would settle it

Take a real NLOS capture on a non-planar relay wall or of a specular object, both of which are absent from TransVerse, mask 95% of the scanning points, run MARMOT, and measure completion fidelity (for example, PSNR against a full dense capture). If the completed transients degrade to the level of simple interpolation, the pretraining benefit is specific to the simulator's scene distribution rather than to general transient structure.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a Transformer masked autoencoder, trained only on synthetic confocal transients with a random scanning-pattern mask, learns generalizable representations of NLOS light transport. The encoder operates on the unmasked subset, which is functionally equivalent to arbitrary sampling, and the decoder predicts the full transient set using a learned mask token; the self-supervised objective is mean squared error over the masked transients. Because high masking ratios such as 95% preserve performance, the model is robust to sparse and irregular scan patterns, and its features support downstream tasks via frozen-encoder transfer or lightweight finetuning. The paper reports that MARMOT outperforms two learned baselines on classification, albedo, and depth metrics, and that physics-based reconstructors such as LCT, f-k, and PF produce sharper shapes when applied to MARMOT's completed transients than when applied to raw sparse measurements or to the outputs of SOCR and CC-SOCR.

Load-bearing premise

The load-bearing premise is that TransVerse's synthetic confocal transients, rendered on planar relay surfaces with diffuse materials and a SPAD noise model, are representative enough of real NLOS transients that features learned on them transfer to real data and to structured scanning patterns beyond random masks.

Editorial extensions

If this is right

  • Physics-based NLOS reconstructors that require dense, regular grids (LCT, f-k, PF) could operate on 5% of the scanning points after MARMOT completion, drastically cutting acquisition time.
  • Learned downstream systems can start from a pretrained encoder instead of training from scratch, reducing the data and compute needed for new NLOS tasks such as classification, albedo, and depth estimation.
  • The masking scheme subsumes arbitrary scanning patterns, so scanners no longer need uniform grids; a single MARMOT model handles dense or sparse and regular or irregular sampling.
  • TransVerse itself becomes a public resource: a million-transient synthetic dataset for pretraining and benchmarking future transient-imaging models.
  • The pretraining objective is modality-agnostic, so the same recipe could transfer to other time-resolved imaging modalities beyond NLOS.
  • Completed transients from MARMOT can be fed to any existing reconstruction algorithm without retraining, making the model a plug-in enhancement for current NLOS systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 95% masking robustness suggests transients are highly spatially redundant; one could push to even higher masks or learn a content-aware mask to minimize acquisition time further, which the paper does not test.
  • Because the paper only pretrains on planar relay surfaces and diffuse materials, extending TransVerse with non-planar and specular scenes would likely improve transfer to those real-world settings; a direct experiment would compare completion PSNR with and without such scene types.
  • MARMOT's encoder features are shown to transfer to classification and 2D estimation; a natural extension is using them for temporal or intensity-based tasks such as motion estimation or material classification, which the paper does not explore.
  • The diffusion-based decoder for albedo and depth is conditioned on MARMOT features; combining this with the completed transients rather than raw full transients could enable joint completion-plus-estimation pipelines for extremely sparse captures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents MARMOT, a masked autoencoder pretrained on TransVerse, a large synthetic dataset of 500K Objaverse objects rendered as confocal NLOS transients. The model is trained to reconstruct masked transients from a small unmasked subset and is then adapted to downstream tasks including transient completion for NLOS reconstruction, albedo estimation, depth estimation, and classification. The authors claim that MARMOT accurately predicts complete transients under arbitrary masking patterns at high masking ratios such as 95%, and that it outperforms existing methods on several NLOS tasks. Experiments are conducted on synthetic TransVerse and public datasets (Zaragoza, Bike, Human) and on one real dataset (Stanford f-k statue), with quantitative results reported for completion, albedo, depth, and classification.

Significance. If the claimed capabilities hold, MARMOT would be a useful contribution: it introduces a large-scale pretraining paradigm for transient imaging, provides a dataset (TransVerse) that could benefit the NLOS community, and demonstrates that masked autoencoding transfers to multiple downstream NLOS tasks. The idea of leveraging the spatial-temporal redundancy of transients via random masking is plausible and the results in Table 6 show large quantitative gains on synthetic completion over optimization-based baselines. However, the significance is tempered by the fact that the central generalization claim (arbitrary masking patterns) is not experimentally established, and several quantitative comparisons show mixed or weak results across metrics. The paper is a solid technical proposal but needs substantially stronger empirical evidence to support its broad claims.

major comments (4)
  1. [Sec. 4.1, Fig. 7, Fig. 14, Table 6] The central claim that MARMOT handles 'arbitrary masking patterns' is not supported by the evidence. Pretraining (Sec. 3.3 and Suppl. B) uses only random masks at a fixed 95% ratio. The only quantitative evaluation on structured SPMs is on a single synthetic bunny from Zaragoza (Fig. 14/Table 6), with no error bars or statistics, and the real f-k statue experiment (Fig. 7) is purely qualitative. To support the 'arbitrary' claim, the authors should evaluate MARMOT on a variety of structured scanning patterns (e.g., stripes, spirals, low-discrepancy sequences) with quantitative completion metrics, and compare against the same model tested with random masks to demonstrate that performance is pattern-agnostic.
  2. [Sec. 4.1, Fig. 5, Table 6] No simple interpolation baseline is included. In Fig. 5, the baseline applies f-k to masked measurements after 'interpolation,' but the interpolation method is not specified and no standard interpolation (e.g., linear, cubic, nearest-neighbor) is compared as a completion step before reconstruction. Without an interpolation baseline in Table 6, it is unclear whether MARMOT's gains come from its learned completion or merely from any reasonable imputation of missing data. Adding such baselines is essential for isolating the contribution of the pretrained model.
  3. [Sec. 5.3, Table 3] The claim that MARMOT outperforms baselines in depth estimation is contradicted by the metric breakdown. For depth, MARMOT has the best ED and PSNR, but the worst cosine similarity (0.8982 vs. NLOST 0.9277 and AGK 0.9141) and only a marginal SSIM advantage over AGK (0.9117 vs. 0.9094). This mixed result does not support the unqualified statement in Sec. 5.3 that 'MARMOT outperforms the baseline methods as well.' The authors should either report per-metric significance tests, temper the claim, or explain why CS is less relevant for depth.
  4. [Sec. 6, Fig. 7] The synthetic-to-real transfer is demonstrated only qualitatively on a single real dataset (f-k statue). Given that pretraining uses synthetic TransVerse data, the practical value of MARMOT depends on real-data performance. The paper should provide at least one quantitative real-data evaluation where ground truth is available, or, if reconstruction metrics are not computable, report additional real scenes and a clear protocol for judging reconstruction quality. Without this, the generalization claim to real NLOS systems remains unsubstantiated.
minor comments (5)
  1. [Abstract and Sec. 1] Typos: 'pretrianed' should be 'pretrained'; 'Delvlin et al.' should be 'Devlin et al.'
  2. [Sec. 3.2, Eq. (1)] The notation τ∗j for the convolution with temporal jitter is undefined; please specify the convolution kernel and the meaning of b (ambient/dark-count bias) more precisely.
  3. [Sec. 5.1 and Sec. 5.2] Cross-references are wrong: 'Table 5.1' and 'Table 5.2' should be 'Table 1' and 'Tables 2–3.' Also, in Sec. 5.1 the text says 'finetune only the decoder,' while Suppl. D.1 says the encoder is frozen and only pooling and linear layers are optimized; please reconcile the wording.
  4. [Sec. 3.2] The dataset description says '500K 3D models from Objaverse' but also 'TransVerse includes one million transients'; clarify whether each object yields multiple transients and how the two numbers relate.
  5. [Supplementary, Figs. 10–14] Several captions contain typos: 'origin' should be 'original,' 'prvoide' should be 'provide,' 'beseline' should be 'baseline,' and 'overwall' should be 'overall.' Please proofread the supplement.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the pretraining/evaluation loop is standard self-supervised reconstruction, and self-citations are not load-bearing.

full rationale

Walking the derivation chain, the central objective is Eq. (2), a standard masked-MSE reconstruction loss, and the model is pretrained on random 95% masks from TransVerse and then evaluated on held-out TransVerse samples as well as on external public data (Zaragoza, Bike, Human) and the real Stanford f-k dataset. No reported prediction is produced by refitting a parameter to the same quantity it is asked to predict: transient completion is evaluated on inputs not used for training, and the downstream tasks use separately trained heads. The self-citations ([12,17,18,51]) appear as baselines, simulator components, or generic reconstruction tools, not as uniqueness theorems or as the sole support for the paper's claims, so they do not constitute a circular chain. The paper's Sec. 4 statement that experiments validate prediction under 'arbitrary masking patterns' is broader than the evidence (random masks at fixed ratios plus four structured SPMs on one real scene), and Sec. 6 concedes limitations (planar relay surface, confocal setup, diffuse materials only); these are correctness/generalization risks rather than cases where a result reduces by construction to its input. The shared synthetic simulator for pretraining and some evaluation is an in-distribution test, but the external and real-data evaluations provide independent evidence, so the circularity score is low.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. The central claim rests on domain assumptions about the forward model and data redundancy; free parameters are the masking ratio and architecture choices.

free parameters (3)
  • Masking ratio = 0.95
    Chosen by hand; the authors observe that MARMOT works well when masking 95% of the transients (Sec 3.1). The central completion results depend on this value.
  • Encoder depth and width = 24 blocks, 1024 channels
    Selected from ablations in the supplement (Table 4-5); not fitted to a physical model but chosen by architecture search.
  • Decoder depth and width = 8 blocks, 512 channels
    Selected from ablations in the supplement (Table 4-5); the asymmetric design is a design choice.
assumptions (3)
  • domain assumption Confocal NLOS transients are spatially and temporally redundant enough that 5 percent of samples can predict the full field.
    Used to justify the 95% masking ratio (Sec 3.1). Validated only empirically on synthetic data in this paper.
  • domain assumption The GL-based simulator plus SPAD noise model (Eq. 1) produces transients representative of real NLOS measurements.
    TransVerse is generated with this simulator (Sec 3.2). Transfer to real data is only shown qualitatively on one statue (Sec 4.1, Fig 7).
  • domain assumption Planar relay surface and confocal configuration hold for the intended applications.
    Stated as a limitation in Sec 6; non-planar surfaces and non-confocal setups require pre-calibration and introduce additional error.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MARMOT: Masked Autoencoder for Modeling Transient Imaging." pith.science (2026). https://pith.science/paper/SL53FGKN

@misc{pith2026250608470,
  author       = {Pith},
  title        = {Pith review of: MARMOT: Masked Autoencoder for Modeling Transient Imaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SL53FGKN}},
  note         = {Machine review of arXiv:2506.08470}
}
read the original abstract

Pretrained models have demonstrated impressive success in many modalities such as language and vision. Recent works facilitate the pretraining paradigm in imaging research. Transients are a novel modality, which are captured for an object as photon counts versus arrival times using a precisely time-resolved sensor. In particular for non-line-of-sight (NLOS) scenarios, transients of hidden objects are measured beyond the sensor's direct line of sight. Using NLOS transients, the majority of previous works optimize volume density or surfaces to reconstruct the hidden objects and do not transfer priors learned from datasets. In this work, we present a masked autoencoder for modeling transient imaging, or MARMOT, to facilitate NLOS applications. Our MARMOT is a self-supervised model pretrianed on massive and diverse NLOS transient datasets. Using a Transformer-based encoder-decoder, MARMOT learns features from partially masked transients via a scanning pattern mask (SPM), where the unmasked subset is functionally equivalent to arbitrary sampling, and predicts full measurements. Pretrained on TransVerse-a synthesized transient dataset of 500K 3D models-MARMOT adapts to downstream imaging tasks using direct feature transfer or decoder finetuning. Comprehensive experiments are carried out in comparisons with state-of-the-art methods. Quantitative and qualitative results demonstrate the efficiency of our MARMOT.

Figures

Figures reproduced from arXiv: 2506.08470 by the authors.

Figure 1
Figure 1. Concept of MARMOT. We resort to a masked autoencoder for modeling transients. In particular in non-line-of-sight (NLOS) scenarios, properties of hidden objects are measured through spherical light trajectories. MARMOT encodes sparse, non-uniform transients (e.g., red scanning points on the wall) into latent features and reconstructs the full transient field via a decoder. Using the features and the recovered transie… view at source ↗
Figure 2
Figure 2. (a) Schematic of confocal NLOS imaging. A laser is directed towards a relay wall where light propagates in spherical wavefronts. After hitting the object, light bounces off back toward the wall. A time-resolved detector records transients, which indicates photon counts over time bins. (b) Scanning points and transients By scanning the points on the wall, transients are collected. (c) Exemplars of TransVerse dataset.… view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Recovered Transients Transients with SPM (a) Pretrain (b) Downstream Tasks [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Reconstruction results on different datasets. We select various hidden objects from different datasets, including TransVerse, Zaragoza [48], Bike [16], and Human [17]. We then mask 95% of the transients for each object under random SPMs and use MARMOT to predict the fu…
Figure 7
Figure 7. Figure 7: LCT MARMOT (LCT) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: 3D reconstruction results. We mask transients with SPM and re￾cover them via MARMOT, then recon￾struct 3D models using a neural-field method [51]. In summary, extensive experimental results for NLOS reconstruction validate that MARMOT can accurately pre￾dict the comple…
Figure 9
Figure 9. Figure 9: Downstream task: imaging & depth estimation (a) Imaging (b) Depth Estimation [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Visualization of origin and recovered transients. We display the original fully sampled transients slices alongside the transients slices recovered using our method. Each group of slices was masked according to the pattern shown in the first column and then reconstruc…
Figure 11
Figure 11. Figure 11: Visualization of TransVerse reconstruction results. We showcase the reconstruction results of transients from the TransVerse dataset. For each set of results, the left side displays recon￾structions from the complete original data, while the right side shows reconstru…
Figure 12
Figure 12. Figure 12: Visualization of TransVerse reconstruction results. We showcase the reconstruction results of transients from the TransVerse dataset. For each set of results, the left side displays recon￾structions from the complete original data, while the right side shows reconstru…
Figure 13
Figure 13. Figure 13: Visualization of reconstruction results across different datasets. We showcase the reconstruction results of transients from the different datasets. For each set of results, the left side displays reconstructions from the complete original data, while the right side s…
Figure 14
Figure 14. Figure 14: Comparison between MARMOT and baseline methods on synthetic dataset. We exploit the transients of a hidden bunny from the Zaragoza dataset [48]. We mask the original transients using four SPMs and recover the missing transients using MARMOT and beseline methods: SOCR …
Figure 15
Figure 15. Figure 15: Qualitative comparison for albedo estimation task. We compare MARMOT with LCT [10], f-k [11], PF [26], NLOST [33] and AGK [17]. GTs are rendered using original meshes of motorbikes. MARMOT achieves reconstruction accuracy comparable to baseline methods. 20 [PITH_FULL…
Figure 16
Figure 16. Figure 16: Visualization of Diffusion Pipeline Architecture for Albedo and Depth Estimation. (a) shows the overwall diffusion process. (b) shows the architechture of the denoise network. Detailed parameters of the network are in Table. 7 21 [PITH_FULL_IMAGE:figures/full_fig_p02…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 42 canonical work pages

  1. [1]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2023

  2. [2]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,”Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020

    2020

  3. [3]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,”arXiv preprint arXiv:2010.02502, 2020

    arXiv 2010

  4. [4]

    USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysis

    J. Jiao, J. Zhou, X. Li, M. Xia, Y . Huang, L. Huang, N. Wang, X. Zhang, S. Zhou, Y . Wang et al., “Usfm: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis,”arXiv preprint arXiv:2401.00153, 2023

    work page Pith review arXiv 2023

  5. [5]

    Foundation model for cancer imaging biomarkers,

    S. Pai, D. Bontempi, I. Hadzic, V . Prudente, M. Sokaˇc, T. L. Chaunzwa, S. Bernatz, A. Hosny, R. H. Mak, and H. J. Birkbak, Nicolai andJ. W. L. Aerts, “Foundation model for cancer imaging biomarkers,”Nature Machine Intelligence, vol. 6, pp. 354–367, 2024

    work page 2024

  6. [6]

    Segment anything in medical images,

    J. Ma, Y . He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nature communications, vol. 15, p. 654, 2024

    work page 2024

  7. [7]

    Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration,

    C. Ma, W. Tan, R. H. He, and B. Yan, “Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration,”Nature Methods, vol. 35, pp. 1–10, 2024

    work page 2024

  8. [8]

    Looking around the corner using transient imaging,

    A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” inIEEE 12th International Conference on Computer Vision. IEEE, 2009, pp. 159–166

    work page 2009

Show all 51 references
  1. [9]

    Recov- ering three-dimensional shape around a corner using ultrafast time-of-flight imaging,

    A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recov- ering three-dimensional shape around a corner using ultrafast time-of-flight imaging,”Nature communications, vol. 3, no. 1, p. 745, 2012

    2012

  2. [10]

    Confocal non-line-of-sight imaging based on the light-cone transform,

    M. O’Toole, D. B. Lindell, and G. Wetzstein, “Confocal non-line-of-sight imaging based on the light-cone transform,”Nature, vol. 555, no. 7696, pp. 338–341, 2018. 10

    2018

  3. [11]

    Wave-based non-line-of-sight imaging using fast fk migration,

    D. B. Lindell, G. Wetzstein, and M. O’Toole, “Wave-based non-line-of-sight imaging using fast fk migration,”ACM Transactions on Graphics (ToG), vol. 38, no. 4, pp. 1–13, 2019

    2019

  4. [12]

    Non-line-of-sight imaging via neural transient fields,

    S. Shen, Z. Wang, P. Liu, Z. Pan, R. Li, T. Gao, S. Li, and J. Yu, “Non-line-of-sight imaging via neural transient fields,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 7, pp. 2257–2268, 2021

    2021

  5. [13]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inProceedings of NAACL-HLT, 2019, pp. 4171–4186

    2019

  6. [14]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009

    2022

  7. [15]

    Objaverse: A universe of annotated 3d objects,

    M. Deitke, D. Schwenk, J. Salvador, L. Weihs, O. Michel, E. VanderBilt, L. Schmidt, K. Ehsani, A. Kembhavi, and A. Farhadi, “Objaverse: A universe of annotated 3d objects,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 142– 13 153

    2023

  8. [16]

    Learned feature embeddings for non-line-of-sight imaging and recognition,

    W. Chen, F. Wei, K. N. Kutulakos, S. Rusinkiewicz, and F. Heide, “Learned feature embeddings for non-line-of-sight imaging and recognition,”ACM Transactions on Graphics, vol. 39, no. 6, pp. 1–18, 2020

    2020

  9. [17]

    Enhancing non-line-of-sight imaging via learnable inverse kernel and attention mechanisms,

    Y . Yu, S. Shen, Z. Wang, B. Huang, Y . Wang, X. Peng, S. Xia, P. Liu, R. Li, and S. Li, “Enhancing non-line-of-sight imaging via learnable inverse kernel and attention mechanisms,” inIEEE/CVF International Conference on Computer Vision, 2023, pp. 10 563–10 573

    2023

  10. [18]

    Onsite non-line-of-sight imaging via online calibration,

    Z. Pan, R. Li, T. Gao, Z. Wang, S. Shen, P. Liu, T. Wu, J. Yu, and S. Li, “Onsite non-line-of-sight imaging via online calibration,”IEEE Photonics Journal, vol. 14, no. 5, pp. 1–11, 2022

    2022

  11. [19]

    Non-line-of-sight imaging using a time-gated single photon avalanche diode,

    M. Buttafava, J. Zeman, A. Tosi, K. Eliceiri, and A. Velten, “Non-line-of-sight imaging using a time-gated single photon avalanche diode,”Optics express, vol. 23, no. 16, pp. 20 997–21 011, 2015

    2015

  12. [20]

    Efficient non-line-of-sight imaging from transient sinograms,

    M. Isogawa, D. Chan, Y . Yuan, K. Kitani, and M. O’Toole, “Efficient non-line-of-sight imaging from transient sinograms,” inEuropean conference on computer vision. Springer, 2020, pp. 193–208

    2020

  13. [21]

    Non-line-of-sight imaging with arbitrary illumination and detection pattern,

    X. Liu, J. Wang, L. Xiao, Z. Shi, X. Fu, and L. Qiu, “Non-line-of-sight imaging with arbitrary illumination and detection pattern,”Nature Communications, vol. 14, no. 1, p. 3230, 2023

    2023

  14. [22]

    Analysis of feature visibility in non-line-of-sight mea- surements,

    X. Liu, S. Bauer, and A. Velten, “Analysis of feature visibility in non-line-of-sight mea- surements,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 10 140–10 148

    2019

  15. [23]

    Self-calibrating, fully differentiable nlos inverse rendering,

    K. Choi, I. Kim, D. Choi, J. Marco, D. Gutierrez, and M. H. Kim, “Self-calibrating, fully differentiable nlos inverse rendering,” inSIGGRAPH Asia 2023 Conference Papers, 2023, pp. 1–11

    2023

  16. [24]

    Fpga accelerator for real-time non- line-of-sight imaging,

    Z. Liao, D. Jiang, X. Liu, A. Velten, Y . Ha, and X. Lou, “Fpga accelerator for real-time non- line-of-sight imaging,”IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, no. 2, pp. 721–734, 2021

    2021

  17. [25]

    Non-line-of-sight surface reconstruction using the directional light-cone transform,

    S. I. Young, D. B. Lindell, B. Girod, D. Taubman, and G. Wetzstein, “Non-line-of-sight surface reconstruction using the directional light-cone transform,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 1407–1416

    2020

  18. [26]

    Non-line-of-sight imaging using phasor-field virtual wave optics,

    X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. Huu Le, A. Jarabo, D. Gutierrez, and A. Velten, “Non-line-of-sight imaging using phasor-field virtual wave optics,”Nature, vol. 572, no. 7771, pp. 620–623, 2019

    2019

  19. [27]

    Fast back-projection for non-line of sight reconstruc- tion,

    V . Arellano, D. Gutierrez, and A. Jarabo, “Fast back-projection for non-line of sight reconstruc- tion,”Open Express, vol. 25, no. 10, pp. 11 574–11 583, 2017. 11

    2017

  20. [28]

    Nlos-neus: Non-line-of-sight neural implicit surface,

    Y . Fujimura, T. Kushida, T. Funatomi, and Y . Mukaigawa, “Nlos-neus: Non-line-of-sight neural implicit surface,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10 532–10 541

    2023

  21. [29]

    Deep non-line-of-sight reconstruc- tion,

    G. J. Chopite, M. B. Hullin, M. Wand, and J. Iseringhausen, “Deep non-line-of-sight reconstruc- tion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 960–969

    2020

  22. [30]

    Non-line-of-sight reconstruction with signal–object collaborative regularization,

    X. Liu, J. Wang, Z. Li, Z. Shi, X. Fu, and L. Qiu, “Non-line-of-sight reconstruction with signal–object collaborative regularization,”Light: Science & Applications, vol. 10, no. 1, p. 198, 2021

    2021

  23. [31]

    Compressed sensing for active non-line-of-sight imaging,

    J.-T. Ye, X. Huang, Z.-P. Li, and F. Xu, “Compressed sensing for active non-line-of-sight imaging,”Optics Express, vol. 29, no. 2, pp. 1749–1763, 2021

    2021

  24. [32]

    Few-shot non-line-of-sight imaging with signal-surface collaborative regularization,

    X. Liu, J. Wang, L. Xiao, X. Fu, L. Qiu, and Z. Shi, “Few-shot non-line-of-sight imaging with signal-surface collaborative regularization,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 303–13 312

    2023

  25. [33]

    Nlost: Non-line-of-sight imaging with transformer,

    Y . Li, J. Peng, J. Ye, Y . Zhang, F. Xu, and Z. Xiong, “Nlost: Non-line-of-sight imaging with transformer,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 313–13 322

    2023

  26. [34]

    Physics to the rescue: Deep non-line-of-sight reconstruction for high-speed imaging,

    F. Mu, S. Mo, J. Peng, X. Liu, J. H. Nam, S. Raghavan, A. Velten, and Y . Li, “Physics to the rescue: Deep non-line-of-sight reconstruction for high-speed imaging,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022

    2022

  27. [35]

    Non-line-of-sight imaging with signal superresolution network,

    J. Wang, X. Liu, L. Xiao, Z. Shi, L. Qiu, and X. Fu, “Non-line-of-sight imaging with signal superresolution network,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 420–17 429

    2023

  28. [36]

    Deep non-line-of-sight imaging from under- scanning measurements,

    Y . Li, Y . Zhang, J. Ye, F. Xu, and Z. Xiong, “Deep non-line-of-sight imaging from under- scanning measurements,” inAdvances in Neural Information Processing Systems, 2023, pp. 1–12

    2023

  29. [37]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” inInternational Conference on Learning Representations, 2015

    2015

  30. [38]

    Deep Residual Learning for Image Recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778

    2016

  31. [39]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” inAdvances in Neural Information Processing Systems, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds., vol. 25. Curran Associates, Inc., 2012. [Online]. A...

    2012

  32. [40]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inInternational Conference on Learning ...

    2021

  33. [42]

    Bevt: Bert pretraining of video transformers,

    R. Wang, D. Chen, Z. Wu, Y . Chen, X. Dai, M. Liu, Y .-G. Jiang, L. Zhou, and L. Yuan, “Bevt: Bert pretraining of video transformers,” 2022

    2022

  34. [43]

    Chatgpt,

    OpenAI ChatGPT, “Chatgpt,” 2023. [Online]. Available: https://www.openai.com/chatgpt

    2023

  35. [44]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training,

    Z. Tong, Y . Song, J. Wang, and L. Wang, “Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training,” inAdvances in neural information processing systems, vol. 35, 2022, pp. 10 078–10 093. 12

    2022

  36. [45]

    Videomae v2: Scaling video masked autoencoders with dual masking,

    L. Wang, B. Huang, Z. Zhao, Z. Tong, Y . He, Y . Wang, Y . Wang, and Y . Qiao, “Videomae v2: Scaling video masked autoencoders with dual masking,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 549–14 560

    2023

  37. [46]

    Masked autoencoders as spatiotemporal learners,

    C. Feichtenhofer, Y . Li, K. Heet al., “Masked autoencoders as spatiotemporal learners,” in Advances in neural information processing systems, vol. 35, 2022, pp. 35 946–35 958. [47]An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, 2021

    2022

  38. [48]

    A dataset for benchmarking time-resolved non-line-of-sight imaging,

    M. Galindo, J. Marco, M. O’Toole, G. Wetzstein, D. Gutierrez, and A. Jarabo, “A dataset for benchmarking time-resolved non-line-of-sight imaging,” 2019. [Online]. Available: https://graphics.unizar.es/nlos

    2019

  39. [49]

    A theory of fermat paths for non-line-of-sight shape reconstruction,

    S. Xin, S. Nousias, K. N. Kutulakos, A. C. Sankaranarayanan, S. G. Narasimhan, and I. Gkioulekas, “A theory of fermat paths for non-line-of-sight shape reconstruction,” inPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 6800–6809

    2019

  40. [50]

    Beyond volumetric albedo–a surface optimization framework for non-line-of-sight imaging,

    C.-Y . Tsai, A. C. Sankaranarayanan, and I. Gkioulekas, “Beyond volumetric albedo–a surface optimization framework for non-line-of-sight imaging,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 1545–1555

    2019

  41. [51]

    Omni-line-of-sight imaging for holistic shape reconstruction,

    B. Huang, X. Peng, S. Shen, S. Xia, R. Li, Y . Yu, Y . Wang, S. Gao, W. Chen, S. Liet al., “Omni-line-of-sight imaging for holistic shape reconstruction,”arXiv preprint arXiv:2304.10780, 2023

    2023 arXiv

  42. [52]

    Model-guided iterative diffusion sampling for nlos reconstruction,

    X. Su, Y . Hong, J.-t. Ye, F. Xu, and X. Yuan, “Model-guided iterative diffusion sampling for nlos reconstruction,”IEEE Journal of Selected Topics in Quantum Electronics, 2024. 13 Supplementary Material for MARMOT: Masked Autoencoder for Modeling Transient Imaging A Transients...

    2024

  43. [512]

    We vary the encoder’s depth and width, the scores of four metrics show that 16 or 24 are optimal for the depth, 512 and 1024 for the width, respectively

    Four metrics are exploited to assess the impact, including Euclidean Distance (ED), Cosine Similarity (CS), Structural Similarity Index (SSIM), and Peak Signal-to-Noise Ratio (PSNR). We vary the encoder’s depth and width, the scores of four metrics show that 16 or 24 are optim...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.