IP Library › Granted Patent US 11,622,099
Granted Patent B2
US 11,622,099 · App. 17/152,632 · Granted Apr 4, 2023

Information-processing apparatus, method of processing information, and program

Inventor: Yuji Kato (Kanagawa, JP)
Assignee: Canon Kabushiki Kaisha
H04N13/282G06T7/74H04N5/2224H04N13/111
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,622,099
App. No.
17/152,632
Granted
Apr 4, 2023
Kind
B2
Abstract

An information-processing apparatus generates camera path information that represents changes with time in the position and posture of a virtual viewpoint and provides another apparatus with the generated camera path information.

Claims (48)

1. An information-processing apparatus comprising:

one or more memories that store a set of instructions;

one or more processors that are in communication with the one or more memories, the one or more processors executing the instructions to cause the information-processing apparatus to:

determine key frame management information of a moving image,

wherein the key frame management information includes a key frame number, a moving image identifier, a key frame time, a moving image time, and parameters including a position of virtual viewpoint and a view direction from the virtual viewpoint;

generate, based on the determined key frame management information, camera path information that represents changes in a position of a virtual viewpoint and a view direction from the virtual viewpoint representing a viewpoint of a virtual viewpoint image generated based on images captured by imaging apparatuses, wherein the camera path information indicates parameters of a position of a virtual viewpoint, a view direction from the virtual viewpoint, and a moving image time that correspond to each of multiple frames; and

provide another apparatus with the camera path information that is generated.

2. The information-processing apparatus according to claim 1 , wherein the camera path information contains the position of the virtual viewpoint and the view direction from the virtual viewpoint at every predetermined time.

3. The information-processing apparatus according to claim 1 , wherein generation of the camera path information is started in response to reception of a user operation related to start of generation of the camera path information.

4. The information-processing apparatus according to claim 1 , further causing the information-processing apparatus to:

operate the position of the virtual viewpoint and the view direction from the virtual viewpoint,

wherein the camera path information is generated in response to an operation being performed during a period from reception of a user operation related to start of generation of the camera path information to reception of a user operation related to end of generation of the camera path information.

5. The information-processing apparatus according to claim 1 , wherein the camera path information is provided to the other apparatus in response to reception of a user operation related to provision of the camera path information.

6. An information-processing apparatus comprising:

one or more memories that store a set of instructions;

one or more processors that are in communication with the one or more memories, the one or more processors executing the instructions to cause the information-processing apparatus to:

acquire, from another apparatus, camera path information that represents changes in a position of a virtual viewpoint and a view direction from the virtual viewpoint representing a viewpoint of a virtual viewpoint image generated based on images captured by imaging apparatuses;

specify a key frame of the acquired camera path information;

generate the virtual viewpoint image, based on the acquired camera path information and the specified key frame; and

display the generated virtual viewpoint image on a display screen.

7. The information-processing apparatus according to claim 6 , wherein the camera path information includes the position of the virtual viewpoint and the view direction from the virtual viewpoint at every predetermined time.

8. The information-processing apparatus according to claim 6 , further causing the information-processing apparatus to:

acquire material data that is used for generating the virtual viewpoint image,

wherein the virtual viewpoint image is generated based on the camera path information that is acquired by using the acquired material data.

9. The information-processing apparatus according to claim 8 , wherein the material data includes three-dimensional shape data of an object and texture data of the object.

10. The information-processing apparatus according to claim 6 , wherein the camera path information is acquired from the other apparatus in response to reception of a user operation related to acquisition of the camera path information.

11. A method of processing information comprising:

determining key frame management information of a moving image,

wherein the key frame management information includes a key frame number, a moving image identifier, a key frame time, a moving image time, and parameters including a position of virtual viewpoint and a view direction from the virtual viewpoint;

generating camera path information, based on the determined key frame management information, that represents changes in a position of a virtual viewpoint and a view direction from the virtual viewpoint representing a viewpoint of a virtual viewpoint image generated based on images captured by imaging apparatuses, wherein the camera path information indicates parameters of a position of a virtual viewpoint, a view direction from the virtual viewpoint, and a moving image time that correspond to each of multiple frames; and

providing another apparatus with the generated camera path information.

12. A method of processing information comprising:

acquiring, from another apparatus, camera path information that represents changes in a position of a virtual viewpoint and a view direction from the virtual viewpoint representing a viewpoint of a virtual viewpoint image generated based on images captured by imaging apparatuses;

determining key frame management information of the acquired camera path information,

wherein the key frame management information includes a key frame number, a moving image identifier, a key frame time, a moving image time, and parameters including a position of virtual viewpoint and a view direction from the virtual viewpoint;

generating the virtual viewpoint image, based on the acquired camera path information and the determined key frame management information; and

displaying the generated virtual viewpoint image on a display screen.

13. A non-transitory computer-readable storage medium storing a program for causing a computer to perform a control method that controls an information-processing apparatus, the control method comprising:

determining key frame management information of a moving image,

wherein the key frame management information includes a key frame number, a moving image identifier, a key frame time, a moving image time, and parameters including a position of virtual viewpoint and a view direction from the virtual viewpoint;

generating camera path information based on the determined key frame management information, that represents changes in a position of a virtual viewpoint and a view direction from the virtual viewpoint representing a viewpoint of a virtual viewpoint image generated based on images captured by imaging apparatuses, wherein the camera path information indicates parameters of a position of a virtual viewpoint, a view direction from the virtual viewpoint, and a moving image time that correspond to each of multiple frames; and

providing another apparatus with the generated camera path information.

14. A non-transitory computer-readable storage medium storing a program for causing a computer to perform a control method that controls an information-processing apparatus, the control method comprising:

acquiring, from another apparatus, camera path information that represents changes in a position of a virtual viewpoint and a view direction from the virtual viewpoint representing a viewpoint of a virtual viewpoint image generated based on images captured by imaging apparatuses;

determining key frame management information of the acquired camera path information,

wherein the key frame management information includes a key frame number, a moving image identifier, a key frame time, a moving image time, and parameters including a position of virtual viewpoint and a view direction from the virtual viewpoint;

generating the virtual viewpoint image, based on the acquired camera path information and the determined key frame management information; and

displaying the generated virtual viewpoint image on a display screen.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2021
From: KATO, YUJI
To: CANON KABUSHIKI KAISHA
Reel/Frame 055188/0460 →
Priority Claims (1)
JP JP2018-136850 · Jul 20, 2018 · national
Continuity (2)
Continuation PCTJP2019026785 · Jul 5, 2019
Related Publication 20210144358A1 · May 13, 2021