System and method of controlling a virtual camera

1. A method of controlling a virtual viewpoint, the method comprising:

receiving information for specifying a viewpoint path of a virtual viewpoint corresponding to a virtual viewpoint video to be generated based on video data of a scene captured by a plurality of imaging apparatus, the viewpoint path defining movement of the virtual viewpoint over a period of time when the plurality of imaging apparatus capture the video data of the scene;

detecting one or more events that have occurred in the scene in the period of time;

determining a plurality of control points on the viewpoint path based on a plurality of specified time points within the period of time, each of the specified time points corresponding to at least one event of a predetermined type detected in the scene, wherein the plurality of control points at least partially define the viewpoint path;

receiving an input for modifying at least one of the plurality of determined control points; and

controlling the virtual viewpoint according to a viewpoint path defined based on the plurality of control points, at least one of which is modified according to the received input.

2. The method according to claim 1 , wherein the virtual viewpoint is controlled by enabling modification of properties of the viewpoint path in response to user input in relation to at least one of the plurality of control points, wherein modification of the viewpoint path is disabled for user inputs in relation to the viewpoint path outside of a vicinity of the plurality of control points.

3. The method according to claim 1 , wherein the viewpoint path is recreated by interpolating a path between the plurality of control points using position in space of the virtual viewpoint at the determined plurality of control points.

4. The method according to claim 1 , wherein modification of the viewpoint path is enabled in a vicinity of each of the determined plurality of control points and disabled for other portions of the viewpoint path.

5. The method according to claim 1 , wherein modification of the viewpoint path in the vicinity of one of the plurality of control points is enabled, and in response to the modification, the viewpoint path is modified based on further control points of the plurality of control points.

6. The method according to claim 1 , further comprising recreating the viewpoint path using the plurality of control points such that a geometry of the recreated viewpoint path substantially matches a geometry of the received viewpoint path.

7. The method according to claim 1 , wherein each of the specified time points is associated with an importance score of a corresponding predetermined event, and wherein the plurality of control points is determined based on the associated importance scores.

8. The method according to claim 1 , wherein the plurality of control points is further identified based on a quality of a viewpoint path recreated by the determined plurality of control points.

9. The method according to claim 8 , wherein the quality of the recreated viewpoint path is determined by comparing the received viewpoint path to the viewpoint path recreated using the determined plurality of control points.

10. The method according to claim 1 , further comprising reproducing at a graphical user interface a timeline of the plurality of control points with each control point being accompanied by a type of a corresponding event, the input being received in relation to the timeline.

11. The method according to claim 1 , wherein the information comprises data of speed and acceleration of the virtual viewpoint on the viewpoint path, the speed and acceleration of the view point being measured and collected at regular intervals along the viewpoint path.

12. The method according to claim 11 , wherein the viewpoint path is recreated based on the speed and acceleration of the virtual viewpoint.

13. The method according to claim 1 , wherein the viewpoint path is recreated by interpolating data of speed and acceleration of the virtual viewpoint between the determined plurality of control points.

14. The method according to claim 1 , wherein the plurality of control points are determined based on a hierarchy of events of predetermined type detected in the video data.

15. The method according to claim 1 , wherein

the receiving of the input includes receiving a user input to a graphical user interface reproducing a timeline of the control points, the user input being in relation to at least one of the plurality of control points.

16. The method according to claim 15 , wherein the timeline includes at least one annotation identifying an event associated with each of the plurality control points.

17. A non-transitory computer readable storage medium storing program instructions for controlling a virtual viewpoint, the program comprising:

code for receiving information for specifying a viewpoint path of a virtual viewpoint corresponding to a virtual viewpoint video to be generated based on video data of a scene captured by a plurality of imaging apparatuses, the view point path defining movement of the virtual viewpoint over a period of time when the plurality of imaging apparatuses capture the video data of the scene;

code for detecting one or more events that have occurred in the scene in the period of time;

code for determining a plurality of control points on the viewpoint path based on a plurality of specified time points within the period of time, each of the specified time points corresponding to at least one event of a predetermined type detected in the scene, wherein the plurality of control points at least partially define the viewpoint path;

code for receiving an input for modifying at least one of the plurality of determined control points; and

code for controlling the virtual viewpoint according to a viewpoint path defined based on the plurality of control points, at least one of which is modified according to the received input.

18. An apparatus, comprising:

one or more processors; and

one or more memories storing instructions for directing the one or more processors to perform a method of controlling a virtual viewpoint, the method comprising:

receiving information for specifying a viewpoint path of a virtual viewpoint corresponding to a virtual viewpoint video to be generated based on video data of a scene captured by a plurality of imaging apparatuses, the viewpoint path defining movement of the virtual viewpoint over a period of time when the plurality of imaging apparatuses capture the video data of the scene;

detecting one or more events that have occurred in the scene in the period of time;

determining a plurality of control points on the viewpoint path based on a plurality of specified time points within the period of time, each of the specified time points corresponding to at least one event of a predetermined type detected in the scene, wherein the plurality of control points at least partially define the viewpoint path;

receiving an input for modifying at least one of the plurality of determined control points; and

controlling the virtual viewpoint according to a viewpoint path defined based on the plurality of control points, at least one of which is modified

according to the received input.

19. A system comprising:

one or more processors; and

one or more memories storing instructions for directing the one or more processors to perform a method of controlling a virtual viewpoint, the method comprising:

receiving information for specifying a viewpoint path of a virtual viewpoint corresponding to a virtual viewpoint video to be generated based on video data of a scene captured by a plurality of imaging apparatuses, the viewpoint path defining movement of the virtual viewpoint over a period of time when the plurality of imaging apparatuses capture the video data of the scene;

detecting one or more events that have occurred in the scene in the period of time;

determining a plurality of control points on the viewpoint path based on a plurality of specified time points within the period of time, each of the specified time points corresponding to at least one event of a predetermined type detected in the scene, wherein the plurality of control points at least partially define the viewpoint path;

receiving an input for modifying at least one of the plurality of determined control points; and

controlling the virtual viewpoint according to a viewpoint path defined based on the plurality of control points, at least one of which is modified

according to the received input.