A deep network predicts multiplane images from narrow-baseline stereo pairs to synthesize novel views that extrapolate beyond the input baseline.
arXiv preprint arXiv:1704.07804 (2017)
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
We propose SfM-Net, a geometry-aware neural network for motion estimation in videos that decomposes frame-to-frame pixel motion in terms of scene and object depth, camera motion and 3D object rotations and translations. Given a sequence of frames, SfM-Net predicts depth, segmentation, camera and rigid object motions, converts those into a dense frame-to-frame motion field (optical flow), differentiably warps frames in time to match pixels and back-propagates. The model can be trained with various degrees of supervision: 1) self-supervised by the re-projection photometric error (completely unsupervised), 2) supervised by ego-motion (camera motion), or 3) supervised by depth (e.g., as provided by RGBD sensors). SfM-Net extracts meaningful depth estimates and successfully estimates frame-to-frame camera rotations and translations. It often successfully segments the moving objects in the scene, even though such supervision is never provided.
fields
cs.CV 2verdicts
UNVERDICTED 2representative citing papers
Current pose tracking algorithms are evaluated with metrics that do not match the precision needs of movement science for variables such as 3D position, velocity, acceleration, and forces.
citing papers explorer
-
Stereo Magnification: Learning View Synthesis using Multiplane Images
A deep network predicts multiplane images from narrow-baseline stereo pairs to synthesize novel views that extrapolate beyond the input baseline.
-
Movement science needs different pose tracking algorithms
Current pose tracking algorithms are evaluated with metrics that do not match the precision needs of movement science for variables such as 3D position, velocity, acceleration, and forces.