IP Library Granted Patent US 11,263,456
Granted Patent B2
US 11,263,456 · App. 16/757,974 · Granted Mar 1, 2022

Virtual object repositioning versus motion of user and perceived or expected delay

Inventor: Atsushi Ishihara (Kanagawa, JP)
Assignee: SONY CORPORATION
G06K9/00671G06T7/20G06T7/90G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,263,456
App. No.
16/757,974
Granted
Mar 1, 2022
Kind
B2
Abstract

Provided is an information processing device that includes a position acquisition section that acquires a position of a virtual object in a real space, having been determined on the basis of a recognition result of a real space corresponding to an image captured by an imaging device, and a display control section that controls display of at least a boundary of the virtual object, on the basis of the position of the virtual object, and a feature of a real object that resides in the real space and is away from the position of the virtual object.

Claims (76)

1. An information processing device, comprising:

a central processing unit (CPU) configured to:

determine a virtual object in a real space based on a recognition result of the real space corresponding to an image captured by an imaging device;

acquire a first position of the virtual object in the real space based on the determination of the virtual object;

calculate a first degree of delay in display of the virtual object based on a feature of a real object in the real space; and

control display of at least a boundary of the virtual object at a second position away from the first position with respect to the real object, wherein

the display of at least the boundary of the virtual object is controlled based on

the acquired first position of the virtual object, and

the calculated first degree of delay in the display of the virtual object is greater than a first value, and

a distance between the acquired first position and the real object is smaller than a second value.

2. The information processing device of claim 1 , wherein

in a case where the virtual object and the real object are in a first positional relation, the CPU is further configured to control display of at least the boundary of the virtual object in a manner different from a case where the virtual object and the real object are in a second positional relation which is different from the first positional relation.

3. The information processing device of claim 2 , wherein

in a case where positional relation between the virtual object and the real object is changed from the first positional relation to the second positional relation, in response to a motion of a field of view of a user, the CPU is further configured to modify the display of at least the boundary of the virtual object, and

the user is associated with the information processing device.

4. The information processing device of claim 1 , wherein

the CPU is further configured to modify a shape of the virtual object based on the first degree of delay in the display of the virtual object is greater than the first value.

5. The information processing device of claim 1 , wherein

the CPU is further configured to one of give a motion to the virtual object or increase the motion of the virtual object based on the first degree of delay in the display of the virtual object is greater than the first value.

6. The information processing device of claim 1 , wherein

the CPU is further configured to modify at least one of color or luminance of the virtual object based on the first degree of delay in the display of the virtual object, and

the at least one of the color or the luminance of the virtual object is modified to be similar to at least one of color or luminance of the real object.

7. The information processing device of claim 1 , wherein

the CPU is further configured to reduce recognizability of the boundary of the virtual object based on the first degree of delay in the display of the virtual object.

8. The information processing device of claim 1 , wherein

the feature of the real object includes at least one of shape of the real object or motion of the real object.

9. The information processing device of claim 1 , wherein

the CPU is further configured to calculate the first degree of delay in the display of the virtual object based on a feature of the virtual object.

10. The information processing device of claim 9 , wherein

the feature of the real object includes a position of a straight line that composes the real object,

the feature of the virtual object includes the first position of the virtual object,

the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and

the position of the virtual object and the position of the straight line that composes the real object is closer corresponding to the first degree of delay than the second degree of delay.

11. The information processing device of claim 9 , wherein

the feature of the real object includes a direction of a straight line that composes the real object,

the feature of the virtual object includes a direction of a straight line that composes the virtual object,

the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and

an angle between the direction of the straight line that composes the virtual object and the direction of the straight line that composes the real object is smaller corresponding to the first degree of delay than the second degree of delay.

12. The information processing device of claim 9 ,

wherein the feature of the virtual object includes at least one of a shape of the virtual object or motion of the virtual object.

13. The information processing device of claim 9 , wherein

the feature of the real object includes a color of the real object,

the feature of the virtual object includes a color of the virtual object,

the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and

the color of the real object and the color of the virtual object is less similar corresponding to the first degree of delay than the second degree of delay.

14. The information processing device of claim 9 , wherein

the feature of the real object includes luminance of the real object,

the feature of the virtual object includes luminance of the virtual object,

the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and

a difference between the luminance of the real object and the luminance of the virtual object is larger corresponding to the first degree of delay than the second degree of delay.

15. The information processing device of claim 1 , wherein

the CPU is further configured to:

calculate motion of a virtual camera from the recognition result of the real space; and

control display of at least the boundary of the virtual object based on the calculated motion of the virtual camera.

16. The information processing device of claim 15 , wherein

the CPU is further configured to modify the display of at least the boundary of the virtual object based on the motion of the virtual camera exceeds a threshold motion value.

17. An information processing method, comprising:

determining a position of a virtual camera based on a recognition result of a real space corresponding to an image captured by an imaging device;

determining a virtual object in the real space based on the determined position of the virtual camera;

acquiring a first position of the virtual object in the real space based on the determination of the virtual object;

calculating a degree of delay in display of the virtual object based on a feature of a real object in the real space; and

controlling display of at least a boundary of the virtual object at a second position away from the first position with respect to the real object, wherein

the display of at least the boundary of the virtual object is controlled based on

the acquired first position of the virtual object, and

the calculated degree of delay in the display of the virtual object is greater than a first value, and

a distance between the acquired first position and the real object is smaller than a second value.

18. A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by a computer, cause the computer to execute operations, the operations comprising:

determining a position of a virtual camera based on a recognition result of a real space corresponding to an image captured by an imaging device;

determining a virtual object in the real space based on the determined position of the virtual camera;

acquiring a first position of the virtual object in the real space based on the determination of virtual object;

calculating a degree of delay in display of the virtual object based on a feature of a real object in the real space; and

controlling display of at least a boundary of the virtual object at a second position away from the first position with respect to the real object, wherein

the display of at least the boundary of the virtual object is controlled based on

the acquired first position of the virtual object, and

the calculated degree of delay in the display of the virtual object being greater than a first value, and

a distance between the acquired first position and the real object is smaller than a second value.

Assignments (2)
CHANGE OF NAME Recorded Dec 17, 2025
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 073986/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2021
From: ISHIHARA, ATSUSHI
To: SONY CORPORATION
Reel/Frame 055589/0566 →