From Single Images to Motion Policies via Video-Generation Environment Representations

Matthew Johnson-Roberson; Tianyi Zhang; Weiming Zhi; Ziyong Ma

REVIEW 3 major objections 4 minor 38 references

From Single Images to Motion Policies via Video-Generation Environment Representations

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that VGER, from a single RGB image, builds a dense 3D scene representation and a metric-modulated motion policy that produces smooth, collision-free trajectories in real time.

desk verdict Clever and coherent pipeline for single-image reactive motion policy, but the numbers currently compare against itself; needs independent ground truth before the safety claim holds. read the letter →

arxiv 2505.19306 v1 pith:5AZ5AZGN submitted 2025-05-25 cs.RO cs.CVcs.GRcs.LG

Weiming Zhi , Ziyong Ma , Tianyi Zhang , Matthew Johnson-Roberson This is my paper

classification cs.ROcs.CVcs.GRcs.LG

keywords single-image3Dreconstructionvideogenerationimplicitunsigneddistancefieldmetric-modulatedmotionpolicycollision-freereactiveDUSt3Rmulti-scalenoisecontrastivesampling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

A single RGB image is enough, the paper argues, to build both a dense 3D model of a scene and a reactive motion policy that steers a robot around obstacles in that scene. The proposed pipeline, VGER, conditions a video-generation model on the input image to synthesize a moving-camera video, fuses sampled frames with a 3D foundation model into a dense point cloud, and trains an implicit unsigned distance field with multi-scale noise contrastive sampling. That distance field is embedded into a metric-modulated dynamical system, so the robot's nominal motion is warped by the environment's geometry in real time. The paper reports that this avoids the frustum-shaped depth artifacts that make monocular depth maps unusable for collision avoidance, and that generated trajectories match ground-truth motion more closely than trajectories built from a state-of-the-art depth estimator.

What carries the argument

The load-bearing object is the metric-modulated velocity update $\dot{x} = G(x)^{-1} g_{\mathrm{base}}(x)$, with $G(x) = I + f_{\mathrm{blow}}(x) u(x) u(x)^{\top}$, where $u(x) = \nabla f_{\theta}(x) / \|\nabla f_{\theta}(x)\|$ and $f_{\mathrm{blow}}(x) = k (f_{\theta}(x) + \epsilon)^{-4} \exp(-\beta f_{\theta}(x))$. Because $G(x)$ is a rank-one update of the identity, the Sherman-Morrison formula turns the inversion into the explicit update $\dot{x} = [I - (f_{\mathrm{blow}}(x)/(1+f_{\mathrm{blow}}(x))) u(x) u(x)^{\top}] g_{\mathrm{base}}(x)$, which is what lets the policy run in under a millisecond per step. Upstream of this, the representation is carried by $f_{\theta}$, an implicit unsigned distance field trained with multi-scale noise contrastive sampling: surface points are perturbed with Gaussian noise whose standard deviation is drawn log-uniformly, and the network, using SIREN periodic activations, regresses true distances while surface and Eikonal losses keep the zero level set on the point cloud and the gradient norm near one.

What would settle it

Run VGER on a single image of a scene and compare its reconstruction against laser-scanned or multi-view ground truth specifically in the regions occluded from the input; if the Chamfer distance in those regions is no better than a hallucination baseline, or if a policy trajectory aimed through a reconstructed region collides when executed on the real scene, the central claim fails.

Watch

Extended reading notes

Core claim

VGER's central claim is that a motion policy can be extracted from a single image without ever predicting per-pixel depth. The method generates a video of a camera moving through the scene, conditioned on the input image, via a pre-trained video generator; frames of that video are fed to a pre-trained 3D reconstruction model to obtain a dense point cloud that includes geometry occluded in the original view. A network with SIREN periodic activations is trained as an implicit unsigned distance field $f_{\theta}(x)$ by sampling surface points perturbed with Gaussian noise at many scales, together with surface and Eikonal losses, so that $f_{\theta}$ approximates the distance to the nearest surface everywhere. The paper then builds a Riemannian metric $G(x) = I + f_{\mathrm{blow}}(x) u(x) u(x)^{\top}$ from the gradient of $f_{\theta}$ and generates motion as $\dot{x} = G(x)^{-1} g_{\mathrm{base}}(x)$, where $g_{\mathrm{base}}$ is any nominal dynamical system; using the Sherman-Morrison identity turns the metric inversion into a rank-one correction. Evaluations on six indoor and outdoor scenes report lower normalized Chamfer distance to multi-view ground truth than DepthAnything-V2, VGGT, or DUSt3R from a single image, and lower normalized Fréchet distance between generated and ground-truth trajectories.

Load-bearing premise

The video generator's synthesized views correctly encode the real 3D geometry of the scene, including regions occluded in the input image, so the point cloud built from them is a trustworthy basis for avoiding collisions.

Editorial extensions

If this is right

A robot could begin avoiding collisions in an environment from a single photograph, without a depth sensor or a separate mapping pass.
Because the velocity update is a closed-form rank-one correction, the motion policy can run in real time inside a control loop.
Occluded geometry, such as the back of a cabinet or the space under a table, is reconstructed and used by the policy, where single-image depth and single-image 3D baselines leave gaps.
Since the shaping term is smooth and differentiable, the same metric construction can be pulled back through a manipulator Jacobian to generate joint-space motion.
Removing frustum-shaped artifacts stops the policy from being funneled into false free-space local minima, which is the failure mode the paper documents for monocular-depth baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The representation and the metric policy are separable: any dense distance field, including one updated online from a SLAM map, could be plugged into the same metric-modulation step to obtain collision-avoiding motion, so the method is not tied to video-generated geometry.
The paper's stated limitation that video generators can hallucinate structure beyond the observed frustum implies a natural extension: attach confidence or uncertainty to the reconstructed geometry and let the metric field discount low-confidence regions, which would address the main residual failure mode.
A controlled comparison that feeds ground-truth multi-view geometry through the same metric policy would isolate how much of VGER's trajectory quality comes from the reconstruction versus the policy construction itself.
The pipeline's reliance on a camera-trajectory-conditioned video generator suggests a scaling prediction: as video generators improve multi-view consistency, VGER's reconstruction error in occluded regions should drop with no change to the motion-policy layer.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline: this is a plausible pipeline for turning a single RGB image into a reactive motion policy, and it is the first to do so by stringing together a video generator, a 3D foundation model, and an RMP-style metric modulation. The individual pieces are known, but the integration is genuinely new, and the motion construction in Sec. 3.3 is mathematically clean: the rank-one metric update and Sherman-Morrison inversion are correct, and the blow-up function behaves sensibly near the surface. The multi-scale noise-contrastive scheme for the unsigned distance field is a reasonable way to train f_theta from a point cloud and is properly connected to score-based denoising.

The soft spots are mostly empirical. The ground truth for all quantitative comparisons is a DUSt3R reconstruction from the ten real images. Since VGER also uses DUSt3R to fuse the generated frames, the Chamfer distances in Fig. 8 and the Frechet distances in Fig. 12 partly measure agreement between two DUSt3R outputs, not fidelity to measured geometry. No error bars, no collision checker, no robot experiment, no clearance metric. The paper's own Sec. 5 admits that Seva hallucinates beyond the seen frustum, and the pipeline has no confidence gate, so the 'collision-free' claim is a statement about the constructed representation, not about the real world. These are fixable: use LiDAR or a mesh from another source as ground truth, add a simple collision check on the trajectories, and report variance over multiple seeds.

Also minor: the DFD comparison is odd because both trajectories start and end at the same points, so the metric mostly measures path shape; a simple swept-volume clearance would have been more direct. And the ablation over frame count is reported for one scene only.

I don't think these flaws sink the paper. The central construction is coherent and the idea is worth testing on real hardware. The right verdict is conditional: the representation and motion generation are internally consistent, but the empirical support is thinner than the conclusion claims.

Who is this for? Robotics researchers working on reactive motion generation from weak perception, and anyone interested in using generative video models as scene priors. It deserves a serious referee, but the reviewers should push for independent ground truth and a real collision check before acceptance.

My recommendation: engage with it, but treat the numbers as demonstrative, not as evidence of safety.

Referee Report

3 major / 4 minor

Summary. VGER aims to produce a reactive collision-avoidance motion policy from a single RGB image. The pipeline (Sec. 3) uses Stable Virtual Camera (Seva) to generate a short moving-camera video from the input image, feeds sampled frames together with the original image into DUSt3R to obtain a dense point cloud, trains an implicit unsigned distance field f_theta by regressing to KD-tree distances at multi-scale Gaussian-perturbed query points with surface and Eikonal regularization, and then constructs a time-invariant dynamics x_dot = G(x)^{-1} g_base(x), where G(x) = I + f_blow(x) u(x)u(x)^T is a rank-one update with a blow-up factor that diverges near surfaces. The empirical sections report normalized Chamfer distances against a DUSt3R reconstruction from ten images and normalized discrete Frechet distances for generated trajectories, plus a frame-count ablation on the table scene. The central claims are that VGER avoids frustum-shaped artifacts, completes occluded geometry, and yields smooth collision-free trajectories in real time from one image.

Significance. The core idea is timely, and the motion-construction part is technically sound: Eq. (9) is an exact Sherman-Morrison inversion; the metric field is continuous and differentiable; and Table A1 provides enough hyperparameters to reproduce the implicit model and policy. If the representation were verified against independent geometry, converting a single image into a reactive metric-modulated policy would be a useful contribution. The paper is also candid about the hallucination risk in Sec. 5. However, the current evaluation does not establish the two headline claims of geometric fidelity and collision-free motion, because the ground truth is itself produced by the same 3D foundation model used inside VGER, and the trajectory evaluation contains no collision or clearance metric.

major comments (3)

[Sec. 4, Fig. 8] The ground-truth point cloud for the Chamfer comparison is constructed by passing all ten collected images through DUSt3R (Sec. 4), which is the same model used inside VGER to fuse the generated video frames. Any systematic DUSt3R error, including hallucinated or missing structure in occluded regions, is therefore shared by both sides of the comparison, so the reported Chamfer values measure agreement with a DUSt3R reconstruction rather than with true scene geometry. This is load-bearing for the claim in Sec. 1 that VGER 'faithfully captures scene geometry.' I would like to see an independent geometric reference, such as LiDAR or RGB-D scans, or COLMAP/SfM with verified scale, with per-scene errors; without that, the quantitative superiority over the baselines is not established.
[Sec. 4.2, Fig. 12c] The trajectory evaluation does not test the central 'collision-free' property. The only quantitative metric is normalized discrete Frechet distance between trajectories generated from VGER/DA-V2 and trajectories generated from the same DUSt3R-based ground truth (Sec. 4.2). There is no collision check, no clearance or penetration metric, no robot footprint model, and no physical experiment. A trajectory can receive a low DFD while still penetrating a surface, and the qualitative images in Fig. 12b are not a substitute for a measured safety metric. Please add quantitative collision/clearance evaluation on independent geometry, including failure counts, and state explicitly whether 'collision-free' is guaranteed with respect to the learned representation or the true environment.
[Sec. 5, Fig. 13] The paper acknowledges that Seva may hallucinate structure when extrapolating beyond the observed frustum (Sec. 5, Fig. 13), and the pipeline has no mechanism for detecting or filtering such hallucinated geometry before it becomes part of f_theta. Consequently, the collision-free guarantee of Sec. 3.3 holds only if the generated multiview geometry used to train f_theta is correct over the region traversed by the policy. If hallucinated structure lies on the path, the metric field will repel the trajectory from a non-existent obstacle; if real structure is missing from the generated frames, the field will not repel at all. This is not an internal inconsistency, but it is an empirical-support gap in the safety claim. I recommend bounding the claim by the validated reconstruction region and adding an uncertainty or confidence mechanism, or at minimum evaluating on scenes with independent ground truth.

minor comments (4)

[Sec. 4.2] The real-time claim 'per-step velocities in under 1 ms' is stated without measurement methodology; please report hardware, integration step, network inference time, and the number of repeated runs.
[Fig. 8, Table 1] Fig. 8 and Table 1 report single values without error bars or the number of seeds, and the ablation over video-frame count is performed on one scene only; the robustness conclusion is therefore weakly supported.
[Sec. 3.1] The notation for the point-cloud dataset, written as {pi, ci} with i from 1 to N, should explicitly state that pi and ci are the 3D coordinates and RGB color vector of each point, respectively.
[Sec. 2] The claim that VGER is 'the first to apply structures and representations produced by 3D foundation models for reactive motion generation' is difficult to verify as stated and should be softened or supported by a comparative discussion of closely related work.

Circularity Check

1 steps flagged · score 3.0 of 10

VGER's derivation is non-circular (Seva→DUSt3R→supervised fθ→hand-built metric policy), but the quantitative reconstruction evaluation is self-referential because the DUSt3R-built 'ground truth' shares the same model used inside VGER.

other [Sec. 4 (ground-truth construction, before Fig. 8) and Sec. 3.1 (DUSt3R fusion of generated frames)]
"We use the entire set of ten images, passed to DUSt3R, to construct a representation that we then consider to be the ground truth. ... The generated images, along with the conditioned input image, are then all inputted into the 3D foundation model, DUSt3R."

The reference used to score VGER's reconstruction is produced by the same DUSt3R model that VGER uses internally to fuse its generated video frames. The Chamfer distances in Fig. 8 therefore compare DUSt3R(generated views) against DUSt3R(real views), so systematic DUSt3R biases — including hallucinated or missing structure, which Sec. 5/Fig. 13 admits — are inherited by both sides. This makes the evaluation partly a measure of DUSt3R's self-consistency across input distributions rather than an independent test of geometric fidelity. It does not make the motion-policy construction itself circular, but it weakens the quantitative claim that VGER 'accurately' reconstructs the scene.

full rationale

The core derivation chain is a pipeline, not a circular argument. Sec. 3.1 defines the point cloud P as DUSt3R applied to Seva-generated frames; Sec. 3.2 trains fθ by regression to exact KD-tree distances to P, plus surface and Eikonal terms; Sec. 3.3 defines the metric G explicitly from fθ and the policy as G^{-1}g_base. Each object is constructed from the previous one; no equation is reused as its own conclusion, and no parameter fitted to a benchmark is later renamed as a prediction. The metric-modulated motion construction follows the external RMP/geometric-fabrics recipe (refs [7], [23]); self-citations such as [8], [16], [17], [24], and [34] are contextual or standard (e.g., Jacobian pullback) and are not load-bearing. The known limitations are empirical rather than circular: Sec. 5/Fig. 13 explicitly admits Seva can hallucinate structure beyond the observed frustum, and Sec. 4.2 reports normalized Fréchet distances with no collision check, clearance metric, or physical experiment. These affect the safety claim but do not reduce the derivation. The one genuinely circular flavor is in the reconstruction benchmark: the 'ground truth' is itself built with DUSt3R, the same model used inside VGER, so the numerical scores partly measure DUSt3R's agreement with itself. That is an evaluation validity concern, not a derivation-level circularity, which is why the score is moderate rather than high.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions about pretrained models (Seva's video consistency, DUSt3R's accuracy) and on a hand-tuned training procedure for the distance field. The paper's evaluation adds an additional domain assumption treating a DUSt3R reconstruction as ground truth. No new physical entities are introduced.

free parameters (4)

sigma_min, sigma_max = 0.0025, 0.1
Bounds of the log-uniform noise scale for negative query sampling in the UDF training (Sec. 3.2, Table A1). Hand-chosen; directly controls the distribution of query points and the behavior of the distance field away from the surface.
alpha_surf, alpha_eik = 0.5, 0.1
Weights of the surface consistency and Eikonal regularization terms in the loss (Eq. 1, Table A1). Chosen by hand without ablation; affect the accuracy and gradient cleanliness of the distance field.
k, beta, epsilon = k=20, beta=100, epsilon=1e-8
Parameters of the blow-up function f_blow in the metric (Eq. 8, Table A1). Hand-selected; determine how strongly and how far from surfaces the metric penalizes normal motion.
omega0, network size, learning rates = omega0=25, 3x256, LR schedule 3e-4 to 1e-5
SIREN architecture and training hyperparameters (Table A1). Standard choices, no ablation or sensitivity analysis; the central representation depends on them.

assumptions (4)

domain assumption Seva-generated frames are geometrically consistent with the real scene, including occluded regions.
Invoked in Sec. 3.1 (the generated image sequences maintain geometry and temporal consistency) and used to construct the point cloud. Directly contradicted for extrapolated regions in Sec. 5 (Fig. 13 hallucination), and no mechanism in the pipeline detects this failure.
domain assumption DUSt3R produces accurate dense point-maps from the generated (synthetic) views.
Used in Sec. 3.1 to fuse frames into a point cloud. The paper treats DUSt3R as a reliable 3D foundation model without validating on the synthetic video domain; domain shift from real images may degrade its accuracy.
domain assumption The 10-image DUSt3R reconstruction is an accurate ground truth for scene geometry.
Sec. 4 defines the evaluation ground truth as DUSt3R applied to all ten collected images. No independent geometric measurement (LiDAR, CAD, depth sensor) is used, so all Chamfer and DFD numbers measure agreement with a learned reconstruction, not with true geometry.
ad hoc to paper Training f_theta with multi-scale noisy query points near the surface yields a valid unsigned distance field over the workspace.
Sec. 3.2 introduces the multi-scale noise-contrastive training on top of KD-tree distance labels. The connection to diffusion score matching is analogical; no theoretical or empirical (held-out) validation of the resulting distance field is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Single Images to Motion Policies via Video-Generation Environment Representations." pith.science (2026). https://pith.science/paper/5AZ5AZGN

@misc{pith2026250519306,
  author       = {Pith},
  title        = {Pith review of: From Single Images to Motion Policies via Video-Generation Environment Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5AZ5AZGN}},
  note         = {Machine review of arXiv:2505.19306}
}

read the original abstract

Autonomous robots typically need to construct representations of their surroundings and adapt their motions to the geometry of their environment. Here, we tackle the problem of constructing a policy model for collision-free motion generation, consistent with the environment, from a single input RGB image. Extracting 3D structures from a single image often involves monocular depth estimation. Developments in depth estimation have given rise to large pre-trained models such as DepthAnything. However, using outputs of these models for downstream motion generation is challenging due to frustum-shaped errors that arise. Instead, we propose a framework known as Video-Generation Environment Representation (VGER), which leverages the advances of large-scale video generation models to generate a moving camera video conditioned on the input image. Frames of this video, which form a multiview dataset, are then input into a pre-trained 3D foundation model to produce a dense point cloud. We then introduce a multi-scale noise approach to train an implicit representation of the environment structure and build a motion generation model that complies with the geometry of the representation. We extensively evaluate VGER over a diverse set of indoor and outdoor environments. We demonstrate its ability to produce smooth motions that account for the captured geometry of a scene, all from a single RGB input image.

Figures

Figures reproduced from arXiv: 2505.19306 by the authors.

**Figure 3.** Examples of extracting a 3D structure of an outdoor bench (top) and indoor office envi [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

**Figure 2.** Pipeline of VGER. Here, we elaborate on the technical details of the VGER method. This includes: leveraging pre-trained video generators to extract a structure conditional on the input image (section 3.1); the construction of an implicit model via multi-scale noise contrastive samples (section 3.2); generating motion from an environment-dependent metric field (section 3.3), giving collisionfree motion. An overvi… view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: We leverage the 3D foundation model, DUSt3R [6], which can produce 3D structures from sets of 2D images, and filter based on confidence maps. The generated images, along with the conditioned input image, are then all inputted into the 3D foundation model, DUSt3R [6]. …

**Figure 5.** Figure 5: With a single example input image, shown in (a), of a stone model in an indoor environ [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 6.** Figure 6: Constructed metric field stretches and warps a base dynamical system gbase(x) with flows (left), to produce the flows G(x) −1 gbase(x) which avoid colliding into the black surface (right). Our goal is now to construct a Riemannian metric field to induce smooth collisi…

**Figure 7.** Figure 7: Qualitative evaluations of the complex 3D environment scenes (from top to bottom: gar [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

**Figure 8.** Figure 8: (Left) Before computing distances, structure from one image by VGER (green) vs. ground [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

**Figure 9.** Figure 9: Single-image extraction leaves occluded regions incomplete, VGER reliably reconstructs them. Left: Top: DepthAnythingV2; bottom: VGER.; Right: Trajectories approaching the cabinet from the occluded backside, avoiding collision to reach the green goal. The qualitat…

**Figure 10.** Figure 10: Left: Input image; Middle: Comparisons do not reconstruct unseen areas; Right: VGER reconstructs geometrically-consistent details in unseen regions, such as under the table. Here, we also highlight VGER’s ability to deal with obstructions in fig. 9, where VGER can r…

**Figure 11.** Figure 11: VGER enables smooth collision-free motion. Here, two trajectories (red and blue) warp over benches and around the leg of the table to reach the goal (green). Next, we assess how VGER improves downstream motion generation. Using the approach outlined in section 3.3, …

**Figure 12.** Figure 12: (a) Frustum-shaped noise artifacts often introduce local minima in the motion objec [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]

**Figure 13.** Figure 13: Left: The input image; Right: Generated frame contains hallucinated structure outlined in red that does not match the setting. The VGER pipeline exploits pre-trained video generators as foundation models for reconstruction and motion generation. In practice, these …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 26 canonical work pages

[1]

Seitz, and Richard Szeliski

Sameer Agarwal, Noah Snavely, Ian Simon, Steven M. Seitz, and Richard Szeliski. Building rome in a day. In 2009 IEEE 12th International Conference on Computer Vision, pages 72–79, 2009

work page 2009
[2]

Srinivasan, Matthew Tancik, Jonathan T

Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020

2020
[3]

3d gaussian splatting for real-time radiance field rendering

Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023

work page 2023
[4]

Splatam: Splat, track & map 3d gaussians for dense rgb-d slam

Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

2024
[5]

Ra ´ul Mur-Artal and Juan D. Tard´os. ORB-SLAM2: an open-source SLAM system for monoc- ular, stereo and RGB-D cameras. IEEE Transactions on Robotics, 33(5):1255–1262, 2017

work page 2017
[6]

Dust3r: Geometric 3d vision made easy

Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024

2024
[7]

Ratliff, Jan Issac, Daniel Kappler, Stan Birchfield, and Dieter Fox

Nathan D. Ratliff, Jan Issac, Daniel Kappler, Stan Birchfield, and Dieter Fox. Riemannian motion policies, 2018

work page 2018
[8]

Diffeomorphic transforms for generalised imitation learning

Weiming Zhi, Tin Lai, Lionel Ott, and Fabio Ramos. Diffeomorphic transforms for generalised imitation learning. In Learning for Dynamics and Control Conference, L4DC, 2022

work page 2022

Show all 38 references

[9]

Depth anything v2

Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Heng- shuang Zhao. Depth anything v2. In The Thirty-eighth Annual Conference on Neural Infor- mation Processing Systems, 2024

2024
[10]

Repurposing diffusion-based image generators for monocular depth estimation

Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

2024
[11]

Depth anything: Unleashing the power of large-scale unlabeled data

Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

2024
[12]

The interpretation of structure from motion

Shimon Ullman. The interpretation of structure from motion. Proceedings of the Royal Society of London. Series B. Biological Sciences, 1979

1979
[13]

Structure-from-motion revisited

Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. In Con- ference on Computer Vision and Pattern Recognition (CVPR), 2016

2016
[14]

Grounding image matching in 3d with mast3r, 2024

Vincent Leroy, Yohann Cabon, and Jerome Revaud. Grounding image matching in 3d with mast3r, 2024

2024
[15]

Vggt: Visual geometry grounded transformer

Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

2025
[16]

Unifying repre- sentation and calibration with 3d foundation models

Weiming Zhi, Haozhan Tang, Tianyi Zhang, and Matthew Johnson-Roberson. Unifying repre- sentation and calibration with 3d foundation models. IEEE Robotics and Automation Letters, 2024

2024
[17]

Simultaneous geometry and pose estimation of held objects via 3d foundation models

Weiming Zhi, Haozhan Tang, Tianyi Zhang, and Matthew Johnson-Roberson. Simultaneous geometry and pose estimation of held objects via 3d foundation models. IEEE Robotics and Automation Letters, 9(12), 2024. 10

2024
[18]

Stylegan-v: A continuous video gener- ator

Ilya Skorokhodov, Stefano Sabatini, and Guang Wang. Stylegan-v: A continuous video gener- ator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

2022
[19]

Video diffusion models

Jonathan Ho, Jascha Sohl-Dickstein, and Pieter Abbeel. Video diffusion models. arXiv preprint arXiv:2209.14792, 2022

2022 arXiv
[20]

Magicvideo: Aligning latent diffusion with trans- formers for text-guided video generation

Hu Zhou, Liang Lin, and Shih-Fu Chang. Magicvideo: Aligning latent diffusion with trans- formers for text-guided video generation. arXiv preprint arXiv:2212.12367, 2022

2022 arXiv
[21]

Sora: A review on background, technology, limitations, and opportunities of large vision models, 2024

Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun. Sora: A review on background, technology, limitations, and opportunities of large vision models, 2024

2024
[22]

Stable virtual camera: Generative view synthesis with diffusion models

Jinghao Zhou, Hang Gao, Vikram V oleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. arXiv preprint arXiv:2503.14489, 2025

2025 arXiv
[23]

Karl Van Wyk, Mandy Xie, Anqi Li, Muhammad Asif Rana, Buck Babich, Bryan Peele, Qian Wan, Iretiayo Akinola, Balakumar Sundaralingam, Dieter Fox, Byron Boots, and Nathan D. Ratliff. Geometric fabrics: Generalizing classical mechanics to capture the physics of behavior. IEEE Rob...

2022
[24]

Global and reactive motion generation with geometric fabric command sequences

Weiming Zhi, Iretiayo Akinola, Karl van Wyk, Nathan Ratliff, and Fabio Ramos. Global and reactive motion generation with geometric fabric command sequences. In IEEE International Conference on Robotics and Automation, ICRA, 2023

2023
[25]

Continuous occupancy map fusion with fast bayesian hilbert maps

Weiming Zhi, Lionel Ott, Ransalu Senanayake, and Fabio Ramos. Continuous occupancy map fusion with fast bayesian hilbert maps. In International Conference on Robotics and Automation (ICRA), 2019

2019
[26]

Spatiotemporal learning of directional uncertainty in urban environments with kernel recurrent mixture density networks

Weiming Zhi, Ransalu Senanayake, Lionel Ott, and Fabio Ramos. Spatiotemporal learning of directional uncertainty in urban environments with kernel recurrent mixture density networks. IEEE Robotics and Automation Letters, 2019

2019
[27]

Mesh manifold based riemannian motion planning for omnidirectional micro aerial vehicles

Michael Pantic, Lionel Ott, Cesar Cadena, Roland Siegwart, and Juan Nieto. Mesh manifold based riemannian motion planning for omnidirectional micro aerial vehicles. IEEE Robotics and Automation Letters, 2021

2021
[28]

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022

2022
[29]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

2021
[30]

Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Proceedings of the 34th International Conference on Neural Information Processing Systems, 2020

2020
[31]

A connection between score matching and denoising autoencoders

Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 2011

2011
[32]

Denoising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021

2021
[33]

Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, 2018

2018
[34]

Bonilla, and Fabio Ramos

Weiming Zhi, Tin Lai, Lionel Ott, Edwin V . Bonilla, and Fabio Ramos. Learning efficient and robust ordinary differential equations via invertible neural networks. In International Confer- ence on Machine Learning, ICML, 2022. 11

2022
[35]

Morrison

Jack Sherman and Winifred J. Morrison. Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Annals of Mathematical Statistics , 21:124–127, 1950

1950
[36]

K. S. Arun, T. S. Huang, and S. D. Blostein. Least-squares fitting of two 3-d point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1987

1987
[37]

H. G. Barrow, J. M. Tenenbaum, R. C. Bolles, and H. C. Wolf. Parametric correspondence and chamfer matching: two new techniques for image matching. In Proceedings of the 5th International Joint Conference on Artificial Intelligence, 1977

1977
[38]

Computing discrete fr ´echet distance

Thomas Eiter and Heikki Mannila. Computing discrete fr ´echet distance. Technical Report CD-TR 94/64, 1994. 12 Appendix A1 Implementation Details We run our experiments on a standard desktop with an Intel i9 CPU and an NVIDIA RTX 4090 GPU with 24GB VRAM. We use all the standar...

1994

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Seitz, and Richard Szeliski

Sameer Agarwal, Noah Snavely, Ian Simon, Steven M. Seitz, and Richard Szeliski. Building rome in a day. In 2009 IEEE 12th International Conference on Computer Vision, pages 72–79, 2009

work page 2009

[2] [2]

Srinivasan, Matthew Tancik, Jonathan T

Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020

2020

[3] [3]

3d gaussian splatting for real-time radiance field rendering

Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023

work page 2023

[4] [4]

Splatam: Splat, track & map 3d gaussians for dense rgb-d slam

Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

2024

[5] [5]

Ra ´ul Mur-Artal and Juan D. Tard´os. ORB-SLAM2: an open-source SLAM system for monoc- ular, stereo and RGB-D cameras. IEEE Transactions on Robotics, 33(5):1255–1262, 2017

work page 2017

[6] [6]

Dust3r: Geometric 3d vision made easy

Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024

2024

[7] [7]

Ratliff, Jan Issac, Daniel Kappler, Stan Birchfield, and Dieter Fox

Nathan D. Ratliff, Jan Issac, Daniel Kappler, Stan Birchfield, and Dieter Fox. Riemannian motion policies, 2018

work page 2018

[8] [8]

Diffeomorphic transforms for generalised imitation learning

Weiming Zhi, Tin Lai, Lionel Ott, and Fabio Ramos. Diffeomorphic transforms for generalised imitation learning. In Learning for Dynamics and Control Conference, L4DC, 2022

work page 2022