IP Library Granted Patent US 10,771,760
Granted Patent B2
US 10,771,760 · App. 16/113,967 · Granted Sep 8, 2020

Information processing device, control method of information processing device, and storage medium

Inventor: Kazuna Maruyama (Fuchu, JP)
Assignee: Canon Kabushiki Kaisha
H04N13/117A63F13/5255G06K9/00724G06T15/20H04N13/194G06K2009/00738H04N13/243H04N2013/0088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,771,760
App. No.
16/113,967
Granted
Sep 8, 2020
Kind
B2
Abstract

An information processing device decides a viewpoint position and generates a virtual viewpoint image based on the decided viewpoint position by using a plurality of images shot by a plurality of imaging apparatuses. The information processing device includes a determining unit configured to determine a scene related to the virtual viewpoint image to be generated, and a deciding unit configured to decide the viewpoint position related to the virtual viewpoint image in the scene determined by the determining unit, based on the scene determined by the determining unit.

Claims (48)

1. An information processing apparatus comprising:

one or more hardware processors; and

one or more memories which store instructions executable by the one or more hardware processors to cause the information processing apparatus to perform at least:

determining a scene regarding which a virtual viewpoint image is to be generated, the scene being included in an event captured by a plurality of imaging apparatuses that obtain captured images for generating the virtual viewpoint image; and

determining, based on the determined scene, a view direction and a position of a virtual viewpoint corresponding to the virtual viewpoint image to be generated.

2. The information processing apparatus according to claim 1 , wherein the instructions further cause the information processing apparatus to perform

determining a composition scenario corresponding to the determined scene,

wherein the view direction and the position of the virtual viewpoint is determined based on the determined composition scenario.

3. The information processing apparatus according to claim 2 ,

wherein the composition scenario is selected based on a user operation from among a plurality of composition scenarios corresponding to the determined scene.

4. The information processing apparatus according to claim 2 ,

wherein a successive transition of the view direction and the position of the virtual viewpoint is determined based on the determined composition scenario.

5. The information processing apparatus according to claim 1 ,

wherein the event includes at least one of a goal scene, a shot scene, and a foul scene.

6. The information processing apparatus according to claim 1 ,

wherein the scene is determined based on a user operation.

7. The information processing apparatus according to claim 1 ,

wherein at least one of a capturing time associated with the virtual viewpoint image to be generated, a location shown in the virtual viewpoint image to be generated, and an incident shown in the virtual viewpoint image to be generated, is determined based on the determined scene.

8. An information processing apparatus comprising:

one or more hardware processors; and

one or more memories which store instructions executable by the one or more hardware processors to cause the information processing apparatus to perform at least:

specifying a condition selected based on a user operation, from among a plurality of conditions for determining a view direction and a position of a virtual viewpoint corresponding to a virtual viewpoint image that is to be generated based on captured images obtained by a plurality of imaging apparatuses; and

determining, based on the specified condition, the view direction and the position of the virtual viewpoint corresponding to the virtual viewpoint image to be generated.

9. The information processing apparatus according to claim 1 , wherein the instructions further cause the information processing apparatus to perform generating the virtual viewpoint image in accordance with the determined view direction and the determined position of the virtual viewpoint.

10. The information processing apparatus according to claim 9 , wherein the instructions further cause the information processing apparatus to perform outputting

the generated virtual viewpoint image.

11. The information processing apparatus according to claim 1 , wherein the instructions further cause the information processing apparatus to perform determining a condition selected based on a user operation from among a plurality of conditions for determining the view direction and the position of the virtual viewpoint,

wherein the view direction and the position of the virtual viewpoint is determined based on the determined condition.

12. The information processing apparatus according to claim 1 , wherein the instructions further cause the information processing apparatus to perform obtaining position information of a particular subject in the captured images,

wherein the view direction and the position of the virtual viewpoint is determined based on the obtained position information of the particular subject.

13. The information processing apparatus according to claim 3 ,

wherein the plurality of composition scenarios includes a composition scenario for taking a line-of-sight of a person as the view direction.

14. The information processing apparatus according to claim 3 ,

wherein the plurality of composition scenarios includes a composition scenario for rotating the virtual viewpoint centered on a predetermined position.

15. The information processing apparatus according to claim 14 ,

wherein the predetermined position is the position of a predetermined person.

16. A method comprising:

determining a scene regarding which a virtual viewpoint image is to be generated, the scene being included in an event captured by a plurality of imaging apparatuses that obtain captured images for generating the virtual viewpoint image; and

determining, based on the determined scene, a view direction and a position of a virtual viewpoint corresponding to the virtual viewpoint image to be generated.

17. A method comprising:

specifying a condition selected based on a user operation, from among a plurality of conditions for determining a view direction and a position of a virtual viewpoint corresponding to a virtual viewpoint image that is to be generated based on captured images obtained by a plurality of imaging apparatus; and

determining, based on the specified condition, the view direction and the position of the virtual viewpoint corresponding to the virtual viewpoint image to be generated.

18. A non-transitory computer-readable storage medium storing a program comprising instructions which, when the program is executed by a computer, cause the computer to carry out a method, the method comprising:

determining a scene regarding which a virtual viewpoint image is to be generated, the scene being included in an event captured by a plurality of imaging apparatuses that obtain captured images for generating the virtual viewpoint; and

determining, based on the determined scene, a view direction and a position of a virtual viewpoint corresponding to the virtual viewpoint image to be generated.

19. A non-transitory computer-readable storage medium storing a program comprising instructions which, when the program is executed by a computer, cause the computer to carry out a method, the method comprising:

specifying a condition selected based on a user operation, from among a plurality of conditions for determining a view direction and a position of a virtual viewpoint corresponding to a virtual viewpoint image that is to be generated based on captured images obtained by a plurality of imaging apparatuses; and

determining, based on the specified condition, the view direction and the position of the virtual viewpoint corresponding to the virtual viewpoint image to be generated.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 6, 2018
From: MARUYAMA, KAZUNA
To: CANON KABUSHIKI KAISHA
Reel/Frame 047416/0097 →
Priority Claims (1)
JP 2017-166106 · Aug 30, 2017 · national
Continuity (1)
Related Publication 20190068945A1 · Feb 28, 2019