IP Library › Granted Patent US 10,462,406
Granted Patent B2
US 10,462,406 · App. 15/326,058 · Granted Oct 29, 2019

Information processing apparatus and information processing method

Inventors: Tatsuya Igarashi (Tokyo, JP); Norifumi Kikkawa (Tokyo, JP)
Assignee: Sony Corporation
H04N5/44504G06F3/048G06T7/30G06T7/579G06T7/74G06T19/00H04N5/23293G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,462,406
App. No.
15/326,058
Granted
Oct 29, 2019
Kind
B2
Abstract

The present disclosure provides an apparatus and a method to convert a registration information setting point into a coordinate system with position of an information processing apparatus at an origin point in association with position of a real object in a real three-dimensional space, and display registration information in accordance with the position and direction of the information processing apparatus. Display control data recording information associated with an anchor and the registration information setting point in virtual three-dimensional spatial coordinates is input, the anchor is detected from a shot image, the position of the information processing apparatus in the virtual three-dimensional space is determined, three-dimensional position in a terminal coordinate system and display position on a display device in a two-dimensional coordinate system are calculated from the three-dimensional position and positional information on the registration information setting point recorded in the display control data, and the point position is displayed.

Claims (73)

1. An information processing apparatus, comprising:

a camera;

a screen configured to display an image shot by the camera; and

a data processing unit configured to:

acquire virtual object display control data that records first positional information associated with an anchor and second positional information associated with a real-world registration information setting point in a virtual three-dimensional spatial coordinate system;

detect the anchor from the image shot by the camera;

determine a position of the camera in the virtual three-dimensional spatial coordinate system based on the first positional information associated with the anchor and a direction of the detected anchor in the virtual three-dimensional spatial coordinate system;

calculate a first position of the real-world registration information setting point in a coordinate system of the screen, based on the second positional information of the real-world registration information setting point; and

display, on the screen, a virtual object that indicates the real-world registration information setting point, wherein

the virtual object is displayed based on the calculated first position, and

the real-world registration information setting point is included in the image shot by the camera.

2. The information processing apparatus according to claim 1 , wherein the data processing unit is further configured to:

transmit a beacon received from a beacon transmitter to a search server;

receive, from the search server, access information to acquire the virtual object display control data, wherein

the virtual object display control data records the anchor as a reference image of a real object shot by the camera, and

the real object is in a vicinity of the beacon transmitter;

receive, from the search server, relative position information of a representation of the real object in the reference image with respect to the real-world registration information setting point; and

receive, from an information provision server, the virtual object display control data, wherein

the virtual object display control data is received based on the received access information, and

the virtual object display control data includes information associated with the real object in the vicinity of the beacon transmitter.

3. The information processing apparatus according to claim 1 , wherein

the virtual object display control data includes registration information that corresponds to the real-world registration information setting point, and

the data processing unit is further configured to superimpose a point position identification mark and the registration information on the shot image displayed on the screen.

4. The information processing apparatus according to claim 1 , wherein

the virtual object display control data includes hyper text markup language (HTML) data that records the first positional information associated with the anchor and the second positional information associated with the real-world registration information setting point in the virtual three-dimensional spatial coordinate system, and

the data processing unit is further configured to calculate, based on the HTML data, the position of the real-world registration information setting point in the coordinate system of the screen.

5. The information processing apparatus according to claim 1 , wherein

the virtual object display control data records a procedure of a process to calculate the first position of the real-world registration information setting point with respect to a position of the information processing apparatus, and

the data processing unit is further configured to calculate the first position of the real-world registration information setting point.

6. The information processing apparatus according to claim 2 , wherein

the data processing unit is further configured to calculate a three-dimensional position of the information processing apparatus from the reference image of the anchor, and

the three-dimensional position of the information processing apparatus is calculated based on simultaneous localization and mapping (SLAM).

7. The information processing apparatus according to claim 2 , further comprising a motion sensor configured to detect motion of the information processing apparatus,

wherein the data processing unit is further configured to calculate a three-dimensional position of the information processing apparatus based on information from the motion sensor and the anchor recorded as the reference image.

8. The information processing apparatus according to claim 1 , wherein

the data processing unit is further configured to calculate a second position of the real-world registration information setting point with respect to the information processing apparatus, based on a motion of the shot image displayed on the screen, and

the screen is further configured to display a point associated with the calculated second position.

9. The information processing apparatus according to claim 1 , wherein

the data processing unit is further configured to update the display on the screen, and

the display is updated based on detection of an event recorded in the virtual object display control data.

10. The information processing apparatus according to claim 3 , wherein

the screen is further configured to display a three-dimensional image, and

the data processing unit is further configured to:

calculate the first position of the real-world registration information setting point for each of a left-eye image and a right-eye image; and

superimpose the point position identification mark as the virtual object at the calculated first position of the real-world registration information setting point.

11. An information provision server, comprising:

a memory configured to store virtual object display control data which includes information associated with a real object in a vicinity of a beacon transmitter; and

circuitry configured to transmit the virtual object display control data, based on a request for acquisition of control information, wherein

the request is received from an information processing apparatus, and

the virtual object display control data records a reference image of the real object shot in the vicinity of the beacon transmitter, first positional information associated with the real object, and second positional information associated with a registration information setting point in a vicinity of the real object in a virtual three-dimensional spatial coordinate system.

12. The information provision server according to claim 11 , wherein

the virtual object display control data includes hyper text markup language (HTML) data that records the first positional information and the second positional information, and

the virtual object display control data records a procedure of a process to calculate a display position of the registration information setting point on a screen of the information processing apparatus.

13. A search server, comprising:

circuitry configured to:

receive, from an information processing apparatus, a transfer signal of a beacon received by the information processing apparatus; and

transmit, to the information processing apparatus, access information based on the transfer signal of the beacon,

wherein the access information is for acquisition of virtual object display control data that records a reference image of a real object in a vicinity of a transmitter of the beacon, first positional information associated with the real object, and second positional information associated with a registration information setting point in a vicinity of the real object in a virtual three-dimensional spatial coordinate system.

14. An information processing method, comprising:

in an information processing apparatus:

displaying an image shot by a camera;

acquiring virtual object display control data that records first positional information associated with an anchor and second positional information associated with a real-world registration information setting point in a virtual three-dimensional spatial coordinate system;

detecting the anchor from the image shot by the camera;

determining a position of the camera in the virtual three-dimensional spatial coordinate system based on the first positional information associated with the anchor and a direction of the detected anchor in the virtual three-dimensional spatial coordinate system;

calculating a position of the real-world registration information setting point in a coordinate system of a screen of the information processing apparatus, based on the second positional information of the real-world registration information setting point; and

displaying a virtual object that indicates the real-world registration information setting point based on the calculated position, wherein the real-world registration information setting point is included in the image shot by the camera.

15. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor in an information processing apparatus, cause the information processing apparatus to execute operations, the operations comprising:

displaying an image shot by a camera;

acquiring virtual object display control data that records first positional information associated with an anchor and second positional information associated with a real-world registration information setting point in a virtual three-dimensional spatial coordinate system;

detecting the anchor from the image shot by the camera;

determining a position of the camera in the virtual three-dimensional spatial coordinate system based on the first positional information associated with the anchor and a direction of the detected anchor in the virtual three-dimensional spatial coordinate system;

calculating a position of the real-world registration information setting point in a coordinate system of a screen of the information processing apparatus, based on the second positional information of the real-world registration information setting point; and

displaying a virtual object that indicates the real-world registration information setting point based on the calculated position, wherein the real-world registration information setting point is included in the image shot by the camera.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 13, 2017
From: IGARASHI, TATUYA; KIKKAWA, NORIFUMI
To: SONY CORPORATION
Reel/Frame 041369/0045 →
Priority Claims (1)
JP 2014-158132 · Aug 1, 2014 · national
Continuity (1)
Related Publication 20170201708A1 · Jul 13, 2017