IP Library › Granted Patent US 11,721,074
Granted Patent B2
US 11,721,074 · App. 17/474,713 · Granted Aug 8, 2023

Display control method and device

Inventors: Yingling Luo (Beijing, CN); Bin Cui (Beijing, CN)
Assignee: LENOVO (BEIJING) LIMITED
G06T19/006G06F3/04815H04N13/279
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,721,074
App. No.
17/474,713
Granted
Aug 8, 2023
Kind
B2
Abstract

A display control method includes performing plane detection on a physical environment within a front field of view (FOV) of an augmented reality (AR) device, determining a first object in the physical environment, determining a display position of a second object based at least on the first object and a current FOV of the AR device, and displaying the second object at the display position. The display position includes a display height and a display distance.

Claims (69)

1. A display control method comprising:

performing plane detection, with a detection range, detecting a three-dimensional space environment on a physical environment within a front field of view (FOV) of an augmented reality (AR) device;

determining a first object in the physical environment, the first object being a physical object for holding other objects;

determining a display position of a second object based at least on the first object, the detection range, and a current FOV of the AR device, the first object being at a rest position, the second object being a virtual object, the display position of the second object including a display height and a display distance; and

displaying the second object at the display position, wherein determining the display position of the second object includes,

in response to the FOV of the AR device being moved and the second object not satisfying a relationship with a range of the FOV of the AR device, maintaining the display position of the second object unchanged;

in response to the FOV of the AR device being moved in a moving direction and the second object satisfying the relationship with the range of the FOV of the AR device, controlling the display position of the second object to move in the moving direction; and

in response to a distance between the second object and a boundary of the detection range of the plane detection being less than a predetermined distance and the FOV continuing to move in the moving direction, stopping controlling the display position of the second object.

2. The display control method of claim 1 , wherein determining the display position of the second object based on the first object and the current FOV of the AR device includes:

determining a first display position of the second object based on a range of the current FOV of the AR device and a size of the second object; and

adjusting the first display position to a second display position based on the first display position and the first object to cause the second object to satisfy a relationship with the first object when being at the second display position, the first display position and the second display position being same or different.

3. The display control method of claim 2 , wherein:

the relationship is that there is no occlusion relationship between the first object and the second object; and

adjusting the first display position to the second display position based on the first display position and the first object includes:

determining whether there is an occlusion relationship between the second object and the first object when the second object is at the first display position, the occlusion relationship including:

that one of the first object and the second object occludes another one of the first object and the second object, or

that an occlusion area between the first object and the second object is larger than a threshold; and

in response to determining that there is the occlusion relationship between the second object and the first object, adjusting the first display position to the second display position to avoid occlusion between the second object at the second position and the first object.

4. The display control method of claim 1 , wherein determining the display position of the second object based at least on the first object and the current FOV of the AR device includes:

determining the display position of the second object based on the first object and a position and a posture of a user of the AR device, to cause a distance between the display position and a boundary of the current FOV of the AR device to be greater than or equal to a first distance, and cause a distance between the display position and the AR device to be less than or equal to a second distance.

5. The display control method of claim 1 ,

wherein the second object includes a plurality of third objects;

the method further comprising:

obtaining an operation, the operation including placing a movement target, and the movement target being one of the plurality of third objects; and

controlling to place the movement target at a position based on the operation, the position being located above and in contact with the first object.

6. The display control method of claim 5 ,

wherein:

a fixed relative positional relationship exists among the plurality of third objects; and

the operation is a first operation and the position is a first position;

the method further comprising, before controlling to place the movement target at the first position:

obtaining a second operation, the second operation including grabbing the movement target; and

controlling the movement target to release the relative positional relationship with other one or more of the third objects.

7. The display control method of claim 6 , further comprising:

in response obtaining the second operation, locking the display position of the second object.

8. A display control device comprising:

a memory storing a computer program; and

a processor configured to execute the computer program to:

perform plane detection, with a detection range, detecting a three-dimensional space environment on a physical environment within a front field of view (FOV) of an augmented reality (AR) device;

determine a first object in the physical environment, the first object being a physical object for holding other objects;

determine a display position of a second object based at least on the first object the detection range, and a current FOV of the AR device, the first object being at a rest position, the second object being a virtual object, the display position including a display height and a display distance; and

display the second object at the display position, wherein to determine the display position of the second object includes,

in response to the FOV of the AR device being moved and the second object not satisfying a relationship with a range of the FOV of the AR device, maintain the display position of the second object unchanged;

in response to the FOV of the AR device being moved in a moving direction and the second object satisfying the relationship with the range of the FOV of the AR device, control the display position of the second object to move in the moving direction; and

in response to a distance between the second object and a boundary of the detection range of the plane detection being less than a predetermined distance and the FOV continuing to move in the moving direction, stop controlling the display position of the second object.

9. The display control device of claim 8 , wherein the processor is further configured to execute the computer program to:

determine a first display position of the second object based on a range of the current FOV of the AR device and a size of the second object; and

adjust the first display position to a second display position based on the first display position and the first object to cause the second object to satisfy a relationship with the first object when being at the second display position, the first display position and the second display position being same or different.

10. The display control device of claim 9 , wherein:

the relationship is that there is no occlusion relationship between the first object and the second object; and

the processor is further configured to execute the computer program to:

determine whether there is an occlusion relationship between the second object and the first object when the second object is at the first display position, and the occlusion relationship including:

that one of the first object and the second object occludes another one of the first object and the second object, or

that an occlusion area between the first object and the second object is larger than a threshold; and

in response to determining that there is the occlusion relationship between the second object and the first object, adjust the first display position to the second display position to avoid occlusion between the second object at the second position and the first object.

11. The display control device of claim 8 , wherein the processor is further configured to execute the computer program to:

determine the display position of the second object based on the first object and a position and a posture of a user of the AR device, to cause a distance between the display position and a boundary of the current FOV of the AR device to be greater than or equal to a first distance, and cause a distance between the display position and the AR device to be less than or equal to a second distance.

12. The display control device of claim 8 , wherein:

the second object includes a plurality of third objects;

the processor is further configured to execute the computer program to:

obtain an operation, the operation including placing a movement target, and the movement target being one of the plurality of third objects; and

control to place the movement target at a position based on the operation, the position being located above and in contact with the first object.

13. The display control device of claim 12 , wherein:

a fixed relative positional relationship exists among the plurality of third objects;

the operation is a first operation and the position is a first position; and

the processor is further configured to execute the computer program to, before controlling to place the movement target at the first position:

obtain a second operation, the second operation including grabbing the movement target; and

control the movement target to release the relative positional relationship with other one or more of the third objects.

14. The display control device of claim 13 , wherein the processor is further configured to execute the computer program to:

in response obtaining the second operation, lock the display position of the second object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2021
From: LUO, YINGLING; CUI, BIN
To: LENOVO (BEIJING) LIMITED
Reel/Frame 058494/0696 →
Priority Claims (1)
CN 202011565110.4 · Dec 25, 2020 · national
Continuity (1)
Related Publication 20220207835A1 · Jun 30, 2022
Cited By (1)
US 12,254,244