Creating action shot video from multi-view capture data

1. A method comprising:

determining a three-dimensional representation of a scene, captured in an action shot base video, the three-dimensional representation of the scene including a three-dimensional representation of a ground surface, the three-dimensional representation of the scene identifying a camera pose for motion of a camera along a path through space in the action shot base video;

determining a representation of an object by estimating a three-dimensional model from a multi-view capture of the object, the multi-view capture including a plurality of images of the object, each of the images being captured from a different respective viewpoint, the multi-view capture being navigable in one or more dimensions; and

generating an action shot video of the scene by combining the three-dimensional representation of the scene captured in the action shot base video and the three-dimensional model from the multi-view capture of the object, the action shot video including a rendered object determined based on the representation of the object and the camera pose and animated via an animation effect, the rendered object being positioned along the path through space.

2. The method recited in claim 1 , wherein the action shot base video is captured by a camera, and wherein determining the three-dimensional representation comprises applying a 3D reconstruction of the scene.

3. The method recited in claim 1 , wherein the action shot base video is a virtual scene, and wherein determining the three-dimensional representation comprises retrieving 3D model information associated with the virtual scene.

4. The method recited in claim 1 , wherein the rendered object is a vehicle that includes one or more wheels, and wherein the animation effect comprises turning one or more of the wheels.

5. The method recited in claim 1 , wherein applying the animation effect comprises animating one or more components of the rendered object.

6. The method recited in claim 5 , wherein determining the representation of the object comprises generating a respective three-dimensional representation of each of the one or more components.

7. The method recited in claim 1 , wherein the multi-view capture of the object is generated on a mobile computing device comprising a camera, and wherein each of the plurality of images of the object are captured by the camera.

8. The method recited in claim 7 , wherein the mobile computing device includes an inertial measurement unit configured to capture inertial measurement data, and wherein determining the representation of the object involves analyzing the inertial measurement data.

9. The method recited in claim 1 , wherein determining the representation of the object comprises determining a respective segmentation mask for the object in each of the images.

10. The method recited in claim 1 , wherein determining the three-dimensional representation of the scene comprises estimating a location for a light source associated with the scene.

11. The method recited in claim 1 , wherein generating an action shot video of the scene comprises rendering a reflection of the scene onto the determined object.

12. The method recited in claim 1 , wherein determining the representation of the object comprises estimating a respective pose of the object for each of the images.

13. The method recited in claim 1 , wherein estimating the three-dimensional model of the object comprises applying a neural network to one or more of the images.

14. The method recited in claim 1 , the method further comprising:

generating a transition sequence between the action shot base video and the action shot video.

15. A system comprising:

a communications interface operable to determine a three-dimensional representation of a scene, captured in an action shot base video, the three-dimensional representation of the scene including a three-dimensional representation of a ground surface, the three-dimensional representation of the scene identifying a camera pose for motion of a camera along a path through space in the action shot base video;

a memory module operable to store a representation of an object by estimating a three-dimensional model from a multi-view capture of the object, the multi-view capture including a plurality of images of the object, each of the images being captured from a different respective viewpoint, the multi-view capture being navigable in one or more dimensions; and

a processor operable to generate an action shot video of the scene by combining the three-dimensional representation of the scene captured in the action shot base video and the three-dimensional model from the multi-view capture of the object, the action shot video including a rendered object determined based on the representation of the obiect and the camera pose and animated via an animation effect, the rendered object being positioned along the path through space.

16. The system recited in claim 15 , wherein the action shot base video is captured by a camera, and wherein determining the three-dimensional representation comprises applying a 3D reconstruction of the scene.

17. The system recited in claim 15 , wherein the action shot base video is a virtual scene, and wherein determining the three-dimensional representation comprises retrieving 3D model information associated with the virtual scene.

18. One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

determining a three-dimensional representation of a scene, captured in an action shot base video, the three-dimensional representation of the scene including a three-dimensional representation of a ground surface, the three-dimensional representation of the scene identifying a camera pose for motion of a camera along a path through space in the action shot base video;

determining a representation of an object by estimating a three-dimensional model from a multi-view capture of the object, the multi-view capture including a plurality of images of the object, each of the images being captured from a different respective viewpoint, the multi-view capture being navigable in one or more dimensions; and

generating an action shot video of the scene by combining the three-dimensional representation of the scene captured in the action shot base video and the three-dimensional model from the multi-view capture of the object, the action shot video including a rendered object determined based on the representation of the object and the camera pose and animated via an animation effect, the rendered object being positioned along the path through space.