Video capture and rendering system control using multiple virtual cameras
There is provided a system and method for integrating a virtual rendering system and a video capture system using flexible camera control to provide an augmented reality. There is provided a method for integrating a virtual rendering system and a video capture system for outputting a composite render to a display, the method comprising obtaining, from the virtual rendering system, a virtual camera configuration of a virtual camera in a virtual environment, programming the video capture system using the virtual camera configuration to correspondingly control a robotic camera in a real environment, capturing a video capture feed using the robotic camera, obtaining a virtually rendered feed using the virtual camera, rendering the composite render by processing the feeds, and outputting the composite render to the display.
1. A method comprising:
tracking a moving virtual object in a virtual environment by a first virtual camera traveling on a first virtual camera path;
tracking the moving virtual object in the virtual environment by a second virtual camera traveling on a second virtual camera path;
obtaining from the first virtual camera a first virtually rendered feed of the virtual environment while tracking the moving virtual object by the first virtual camera;
obtaining from the second virtual camera a second virtually rendered feed of the virtual environment while tracking the moving virtual object by the second virtual camera;
obtaining from a virtual rendering system a first virtual camera position data and a first virtual camera movement data of the first virtual camera while the first virtual camera is travelling on the virtual camera path and tracking the moving virtual object in the virtual environment;
obtaining from the virtual rendering system a second virtual camera position data and a second virtual camera movement data of the second virtual camera while the second virtual camera is travelling on the second virtual camera path and tracking the moving virtual object in the virtual environment;
providing a composite render by processing the first virtually rendered feed, the second virtually rendered feed, the first virtual camera position data, the second virtual camera position data, the first virtual camera movement data and the second virtual camera movement data, wherein the composite render combines the first virtually rendered feed and the second virtually rendered feed such that a juxtaposition of a first virtual element of the virtual environment and a second virtual element of the virtual environment is visually unobtrusive;
outputting the composite render that combines the first virtually rendered feed and the second virtually rendered feed to the display.
2. The method of claim 1 further comprising:
capturing, using a first real camera, a first video capture feed of a real object in a real environment;
wherein providing the composite render includes processing the first video capture feed, and wherein the outputting the composite render to the display includes simultaneously displaying the first virtually rendered feed, the second virtually rendered feed and the first video capture feed.
3. The method of claim 2 , wherein the first virtual camera provides a first motion path of the first virtual camera in the virtual environment, and wherein a first robotic camera uses the first motion path in the real environment.
4. The method of claim 3 , wherein the first motion path includes a three-dimensional position with respect to time.
5. The method of claim 3 , wherein the first motion path includes a camera orientation or field of view with respect to time.
6. The method of claim 1 , wherein the composite render combines the first virtually rendered feed and the second virtually rendered feed to form a three-dimensional surface.
7. The method of claim 1 , wherein the moving virtual object comprises a virtual player of a virtual game.
8. The method of claim 1 , wherein a rendering engine controller coordinates control and data sharing between a first sub-rendering engine and a second sub-rendering engine providing the first virtually rendered feed and the second virtually rendered feed, respectively.
9. The method of claim 1 , wherein the first virtual camera provides a velocity of the first virtual camera, and wherein a first robotic camera uses the velocity in the real environment.
10. The method of claim 1 , wherein the first virtual camera provides an acceleration of the first virtual camera, and wherein a first robotic camera uses the acceleration in the real environment.
11. A rendering controller comprising: one or more subsystems to:
track a moving virtual object in a virtual environment by a first virtual camera traveling on a first virtual camera path;
track the moving virtual object in the virtual environment by a second virtual camera traveling on a second virtual camera path;
obtain from the first virtual camera a first virtually rendered feed of the virtual environment while tracking the moving virtual object by the first virtual camera;
obtain from the second virtual camera a second virtually rendered feed of the virtual environment while tracking the moving virtual object by the second virtual camera;
obtain from a virtual rendering system a first virtual camera position data and a first virtual camera movement data of the first virtual camera while the first virtual camera is travelling on the virtual camera path and tracking the moving virtual object in the virtual environment;
obtain from the virtual rendering system a second virtual camera position data and a second virtual camera movement data of the second virtual camera while the second virtual camera is travelling on the second virtual camera path and tracking the moving virtual object in the virtual environment;
provide a composite render by processing the first virtually rendered feed, the second virtually rendered feed, the first virtual camera position data, the second virtual camera position data, the first virtual camera movement data and the second virtual camera movement data, wherein the composite render combines the first virtually rendered feed and the second virtually rendered feed such that a juxtaposition of a first virtual element of the virtual environment and a second virtual element of the virtual environment is visually unobtrusive; and
output the composite render that combines the first virtually rendered feed and the second virtually rendered feed to the display.
12. The rendering controller of claim 11 , wherein the one or more subsystems capture, using a first real camera, a first video capture feed of a real object in a real environment;
wherein providing the composite render includes processing the first video capture feed, and wherein the outputting the composite render to the display includes simultaneously displaying the first virtually rendered feed, the second virtually rendered feed and the first video capture feed.
13. The rendering controller of claim 12 , wherein the first virtual camera provides a first motion path of the first virtual camera in the virtual environment, and wherein a first robotic camera uses the first motion path in the real environment.
14. The rendering controller of claim 12 , wherein the first motion path includes a three-dimensional position with respect to time.
15. The rendering controller of claim 12 , wherein the first motion path includes a camera orientation or field of view with respect to time.
16. The rendering controller of claim 11 , wherein the composite render combines the first virtually rendered feed and the second virtually rendered feed to form a three-dimensional surface.
17. The rendering controller of claim 11 , wherein the moving virtual object comprises a virtual player of a virtual game.
18. The rendering controller of claim 11 , wherein a rendering engine controller coordinates control and data sharing between a first sub-rendering engine and a second sub-rendering engine providing the first virtually rendered feed and the second virtually rendered feed, respectively.
19. The rendering controller of claim 11 , wherein the first virtual camera provides a velocity of the first virtual camera, and wherein a first robotic camera uses the velocity in the real environment.
20. The rendering controller of claim 11 , wherein the first virtual camera provides an acceleration of the first virtual camera, and wherein a first robotic camera uses the acceleration in the real environment.