IP Library Granted Patent US 9,823,821
Granted Patent B2
US 9,823,821 · App. 14/382,505 · Granted Nov 21, 2017

Information processing apparatus, display control method, and program for superimposing virtual objects on input image and selecting an interested object

Inventors: Kenichirou Ooi (Kanagawa, JP); Akihiko Kaino (Kanagawa, JP)
Assignee: SONY CORPORATION
G06F3/04842G06F3/0482G06F3/04815G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,823,821
App. No.
14/382,505
Granted
Nov 21, 2017
Kind
B2
Abstract

There is provided an information processing apparatus, including an image acquisition unit which acquires an input image generated by imaging a real space using an image pickup apparatus, a display control unit which superimposes virtual objects, which are associated with one or more real objects shown in the input image, on the input image, and a selection unit which selects, after at least one real object from among the one or more real objects is designated by a user, the designated real object as an interested object across a plurality of frames. The display control unit changes displayed information, which is displayed by the virtual object associated with the interested object, according to a relative position or attitude between the interested object and the image pickup apparatus.

Claims (58)

1. An information processing apparatus, comprising:

circuitry configured to

acquire an input image generated by imaging a real space using an image pickup apparatus;

superimpose virtual objects, which are associated with one or more real objects shown in the input image, on the input image;

select, after at least one real object from among the one or more real objects is designated by a user, the designated real object as an interested object across a plurality of frames; and

change displayed information, which is displayed by the virtual object associated with the interested object, according to a relative position or attitude between the interested object and the image pickup apparatus, wherein

in case the interested object is not designated by the user, the circuitry selects a real object identified by a given standard, from among the one or more real objects, as a default object,

the circuitry is configured to control display of a virtual object associated with the default object in a manner that the default object is capable of being identified by the user,

in case the interested object is not designated by the user, the circuitry changes the default object to a new default object in case the input image changes to a new input image as an angle of the image pickup apparatus changes such that the new default object is selected from among another one or more real objects shown in the new input image after the angle of the image pickup apparatus has been changed, and

in case the interested object is designated by the user, the circuitry keeps the interested object without changing the interested object to a new interested object even in case the input image changes to the new input image as the angle of the image pickup apparatus changes.

2. The information processing apparatus according to claim 1 ,

wherein the circuitry selects the default object, which is selected when a predetermined user input is detected, as the interested object.

3. The information processing apparatus according to claim 1 ,

wherein the circuitry changes the displayed information, according to a parameter which depends on at least one of a distance of the image pickup apparatus from the interested object, and an angle of the interested object with respect to an image pickup surface.

4. The information processing apparatus according to claim 3 ,

wherein, when the distance of the image pickup apparatus from the interested object falls within a first range, the circuitry adopts first information as the displayed information, and when the distance falls within a second range nearer than the first range, the circuitry adopts second information as the displayed information, which is more detailed than the first information as the displayed information.

5. The information processing apparatus according to claim 4 ,

wherein the first range and the second range partially overlap.

6. The information processing apparatus according to claim 4 ,

wherein the circuitry controls display of the virtual object associated with the interested object in a manner that a change of the displayed information is predicted by the user.

7. The information processing apparatus according to claim 3 ,

wherein, when the angle of the interested object with respect to the image pickup surface falls within a first range, the circuitry adopts first information as the displayed information, and when the angle falls within a second range different from the first range, the circuitry adopts second information different from the first information as the displayed information.

8. The information processing apparatus according to claim 3 ,

wherein, when a scale of the interested object projected in the image pickup surface falls within a first range, the circuitry adopts first information as the displayed information, and when the scale falls within a second range larger than the first range, the circuitry adopts second information as the displayed information, which is more detailed than the first information as the displayed information.

9. The information processing apparatus according to claim 1 ,

wherein the circuitry recognizes the one or more real objects using the input image.

10. The information processing apparatus according to claim 9 ,

wherein the circuitry recognizes only the interested object within the input image while the interested object is selected.

11. The information processing apparatus according to claim 1 ,

wherein the circuitry recognizes the one or more real objects by making an inquiry to a database using a position of the image pickup apparatus.

12. The information processing apparatus according to claim 1 ,

wherein the given standard is a standard related to a two-dimensional position of the real object within the input image.

13. The information processing apparatus according to claim 12 ,

wherein the given standard is a standard related to an attitude of the real object with the image pickup apparatus regarded as a standard.

14. The information processing apparatus according to claim 1 ,

wherein the given standard is a standard related to a three-dimensional position of the real object within the real space with the image pickup apparatus regarded as a standard.

15. The information processing apparatus according to claim 1 ,

wherein the information processing apparatus is a portable terminal further including the image pickup apparatus.

16. The information processing apparatus according to claim 1 ,

wherein at least one of steps performed by the circuitry is realized using an apparatus in a cloud computing environment instead of the information processing apparatus.

17. A display control method, comprising:

acquiring an input image generated by imaging a real space using an image pickup apparatus;

superimposing virtual objects, which are associated with one or more real objects shown in the input image, on the input image;

selecting, after at least one real object from among the one or more real objects is designated by a user, the designated real object as an interested object across a plurality of frames;

changing, using circuitry, displayed information, which is displayed by the virtual object associated with the interested object, according to a relative position or attitude between the interested object and the image pickup apparatus;

in case the interested object is not designated by the user, selecting a real object identified by a given standard, from among the one or more real objects, as a default object;

controlling display of a virtual object associated with the default object in a manner that the default object is capable of being identified by the user;

in case the interested object is not designated by the user, changing the default object to a new default object in case the input image changes to a new input image as an angle of the image pickup apparatus changes such that the new default object is selected from among another one or more real objects shown in the new input image after the angle of the image pickup apparatus has been changed; and

in case the interested object is designated by the user, keeping the interested object without changing the interested object to a new interested object even in case the input image changes to the new input image as the angle of the image pickup apparatus changes.

18. A non-transitory computer readable medium including executable instructions, which when executed by a computer cause the computer to:

acquire an input image generated by imaging a real space using an image pickup apparatus;

superimpose virtual objects, which are associated with one or more real objects shown in the input image, on the input image;

select, after at least one real object from among the one or more real objects is designated by a user, the designated real object as an interested object across a plurality of frames; and

change displayed information, which is displayed by the virtual object associated with the interested object, according to a relative position or attitude between the interested object and the image pickup apparatus, wherein

in case the interested object is not designated by the user, the computer is configured by the executable instructions to select a real object identified by a given standard, from among the one or more real objects, as a default object,

the computer is configured by the executable instructions to control display of a virtual object associated with the default object in a manner that the default object is capable of being identified by the user,

in case the interested object is not designated by the user, the circuitry is configured by the executable instructions to change the default object to a new default object in case the input image changes to a new input image as an angle of the image pickup apparatus changes such that the new default object is selected from among another one or more real objects shown in the new input image after the angle of the image pickup apparatus has been changed, and

in case the interested object is designated by the user, the circuitry is configured by the executable instructions to keep the interested object without changing the interested object to a new interested object even in case the input image changes to the new input image as the angle of the image pickup apparatus changes.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 2, 2014
From: OOI, KENICHIROU; KAINO, AKIHIKO
To: SONY CORPORATION
Reel/Frame 033652/0932 →
Priority Claims (1)
JP 2012-090049 · Apr 11, 2012 · national
Continuity (1)
Related Publication 20150052479A1 · Feb 19, 2015