IP Library Granted Patent US 11,302,021
Granted Patent B2
US 11,302,021 · App. 16/329,014 · Granted Apr 12, 2022

Information processing apparatus and information processing method

Inventor: Hiroshi Oryoji (Kanagawa, JP)
Assignee: SONY CORPORATION
G06T7/593G06T7/73G06T7/75G06T17/20H04N5/232G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,302,021
App. No.
16/329,014
Granted
Apr 12, 2022
Kind
B2
Abstract

Provided is an information processing apparatus, including: an estimation processing section that, on the basis of three-dimensional data and a first image feature value that are set and a second image feature value that is acquired from a captured image captured by a first imaging apparatus, estimates a position and attitude of the first imaging apparatus.

Claims (46)

1. An information processing apparatus, comprising:

a processing unit configured to:

acquire a second image feature value corresponding to a first image captured by a first imaging apparatus which is a portable type imaging apparatus;

set three-dimensional data of an object, wherein the three-dimensional data is generated based on a second image captured by a second imaging apparatus at a fixed position;

set a first image feature value, wherein the first image feature value is generated based on a quantitative value corresponding to a feature of the second image;

match the first image feature value with the second image feature value;

extract a third image feature value based on the match of the first image feature value with the second image feature value, wherein the third image feature value is within a range between the first image feature value and the second image feature value; and

estimate a position and an attitude of the first imaging apparatus based on the extracted third image feature value and the set three-dimensional data.

2. The information processing apparatus according to claim 1 , wherein the second imaging apparatus has a specific position and a specific attitude.

3. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to acquire the first image feature value corresponding to the second image captured at a time same as a time of the capture of the first image.

4. The information processing apparatus according to claim 1 , wherein the three-dimensional data is generated based on the second image captured by the second imaging apparatus with a specific position and a specific attitude.

5. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to update the three-dimensional data based on the first image captured by the first imaging apparatus with the estimated position and the estimated attitude.

6. The information processing apparatus according to claim 5 , wherein the processing unit is further configured to

execute a three-dimensional construction based on the first image captured by the first imaging apparatus.

7. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to update the three-dimensional data based on the second image captured by the second imaging apparatus with a specific position and a specific attitude.

8. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to update the first image feature value based on the first image captured by the first imaging apparatus with the estimated position and the estimated attitude.

9. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to update the first image feature value based on the second image captured by the second imaging apparatus with a specific position and a specific attitude.

10. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to control transmission of information associated with the estimated position and the estimated attitude of the first imaging apparatus to an external apparatus.

11. The information processing apparatus according to claim 10 , wherein

the processing unit is further configured to:

generate a depth map based on at least one of:

the first image captured by the first imaging apparatus with the estimated position and the estimated attitude, or

the second image captured by the second imaging apparatus with a specific position and a specific attitude; and

control transmission of the depth map to the external apparatus.

12. The information processing apparatus according to claim 10 , wherein the external apparatus is the first imaging apparatus with the estimated position and the estimated attitude.

13. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to:

set a range for a time difference between a first time stamp corresponding to the capture of the first image and a second time stamp corresponding to the capture of the second image; and

acquire the first image feature value corresponding to the second image in which the second time stamp is within the set range of the time difference.

14. The information processing apparatus according to claim 1 , wherein the processing unit is further configured to:

exclude a number of three-dimensional data of the set three-dimensional data from the set three-dimensional data based on the estimated position and the attitude of the first imaging apparatus;

compare the excluded number of three-dimensional data and the second image feature value with a threshold number; and

determine, based on a result of the comparison of the excluded number of three-dimensional data and the second image feature value with the threshold number, whether the position and the attitude of the first imaging apparatus has been estimated.

15. An information processing method executed by an information processing apparatus, the information processing method comprising:

acquiring a second image feature value corresponding to a first image captured by a first imaging apparatus which is a portable type imaging apparatus;

setting three-dimensional data of an object, wherein the three-dimensional data is generated based on a second image captured by a second imaging apparatus at a fixed position;

setting a first image feature value, wherein the first image feature value is generated based on a quantitative value corresponding to a feature of the second image;

matching the first image feature value with the second image feature value;

extracting a third image feature value based on the match of the first image feature value with the second image feature value, wherein the third image feature value is within a range between the first image feature value and the second image feature value; and

estimating a position and an attitude of the first imaging apparatus based on the extracted third image feature value and the set three-dimensional data.

16. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a computer, cause the computer to execute operations, the operations comprising:

acquiring a second image feature value corresponding to a first image captured by a first imaging apparatus which is a portable type imaging apparatus;

setting three-dimensional data of an object, wherein the three-dimensional data is generated based on a second image captured by a second imaging apparatus at a fixed position;

setting a first image feature value, wherein the first image feature value is generated based on a quantitative value corresponding to a feature of the second image;

matching the first image feature value with the second image feature value;

extracting a third image feature value based on the match of the first image feature value with the second image feature value, wherein the third image feature value is within a range between the first image feature value and the second image feature value; and

estimating a position and an attitude of the first imaging apparatus based on the extracted third image feature value and the set three-dimensional data.

Assignments (2)
CHANGE OF NAME Recorded Dec 17, 2025
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 073986/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 27, 2019
From: ORYOJI, HIROSHI
To: SONY CORPORATION
Reel/Frame 048457/0571 →