IP Library › Granted Patent US 11,521,346
Granted Patent B2
US 11,521,346 · App. 17/382,473 · Granted Dec 6, 2022

Image processing apparatus, image processing method, and storage medium

Inventor: Keigo Yoneda (Kanagawa, JP)
Assignee: CANON KABUSHIKI KAISHA
G06T15/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,521,346
App. No.
17/382,473
Granted
Dec 6, 2022
Kind
B2
Abstract

An image processing apparatus obtains virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint and generates a virtual viewpoint image based on the virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image, the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition.

Claims (53)

1. An image processing apparatus comprising:

one or more memories storing instructions; and

one or more processors executing the instructions to:

obtain virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint; and

generate a virtual viewpoint image based on the obtained virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image, the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition wherein

the virtual viewpoint image is a moving image including a plurality of frames, and

the condition is based on the virtual viewpoint information in a single frame and representing that a resolution of an object in the single frame is equal to or greater than a predetermined threshold.

2. The image processing apparatus according to claim 1 , wherein

the condition comprises a plurality of conditions, and

the one or more processors further execute the instructions to determine that a region in which the virtual viewpoint information satisfies all of the plurality of conditions is the specific region.

3. The image processing apparatus according to claim 1 , wherein

the one or more processors further execute the instructions to distinguish an object to be subjected to the hiding processing according to a position in a three-dimensional space.

4. The image processing apparatus according to claim 1 , wherein

the hiding processing the specific region is performed by performing processing of obscuring the specific region.

5. The image processing apparatus according to claim 1 , wherein

the hiding processing the specific region is performed by performing masking processing for the specific region.

6. The image processing apparatus according to claim 1 , wherein

the one or more processors further execute the instructions to receive, from an external apparatus, the virtual viewpoint information determined according to an amount of operation of the virtual viewpoint by a user, and

the received virtual viewpoint information is obtained.

7. The image processing apparatus according to claim 1 , wherein

the one or more processors further execute the instructions to:

receive an amount of operation of the virtual viewpoint by a user from an external apparatus, and

determine virtual viewpoint information according to the amount of operation, and

the determined virtual viewpoint information is obtained.

8. An image processing apparatus comprising:

one or more memories storing instructions; and

one or more processors executing the instructions to:

obtain virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint; and

generate a virtual viewpoint image based on the obtained virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image, the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition, wherein

the virtual viewpoint image is a moving image including a plurality of frames, and

the condition is based on the virtual viewpoint information in the plurality of frames.

9. The image processing apparatus according to claim 8 , wherein

the condition is that a time during which an object is included in the plurality of frames is equal to or greater than a predetermined threshold.

10. The image processing apparatus according to claim 8 , wherein

the condition is that an amount of movement of the virtual viewpoint in the plurality of frames is less than a predetermined threshold.

11. An image processing apparatus comprising:

one or more memories storing instructions; and

one or more processors executing the instructions to:

obtain virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint; and

generate a virtual viewpoint image based on the obtained virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image, the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition, wherein

the hiding processing the specific region is performed by masking processing gradually for the specific region in a case where the virtual viewpoint information satisfies the condition.

12. The image processing apparatus according to claim 11 , wherein

in a case where a difference between the virtual viewpoint information and a threshold included in the condition is equal to a first value, strength of the masking processing is reduced as compared with a case where the difference between the virtual viewpoint information and the threshold included in the condition is equal to a second value greater than the first value.

13. An image processing method comprising:

obtaining virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint; and

generating a virtual viewpoint image based on the obtained virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image, the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition, wherein

the virtual viewpoint image is a moving image including a plurality of frames, and

the condition is based on the virtual viewpoint information in a single frame and representing that a resolution of an object in the single frame is equal to or greater than a predetermined threshold.

14. A non-transitory computer readable storage medium storing a program which causes a computer to perform:

obtaining virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint; and

generating a virtual viewpoint image based on the obtained virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition, wherein

the virtual viewpoint image is a moving image including a plurality of frames, and

the condition is based on the virtual viewpoint information in a single frame and representing that a resolution of an object in the single frame is equal to or greater than a predetermined threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2021
From: YONEDA, KEIGO
To: CANON KABUSHIKI KAISHA
Reel/Frame 057015/0401 →
Priority Claims (1)
JP JP2020-136320 · Aug 12, 2020 · national
Continuity (1)
Related Publication 20220051469A1 · Feb 17, 2022
Cited By (1)
US 12,610,035