IP Library Granted Patent US 10,565,761
Granted Patent B2
US 10,565,761 · App. 15/834,125 · Granted Feb 18, 2020

Augmented reality z-stack prioritization

Inventors: Lisa Seacat DeLuca (Baltimore, MD); Jeremy A. Greenberger (San Jose, CA); Zachary M. Greenberger (San Jose, CA)
Assignee: Wayfair LLC
G06T11/60G02B27/0172G06F3/013G02B2027/014G02B2027/0138G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,565,761
App. No.
15/834,125
Filed
Dec 7, 2017
Granted
Feb 18, 2020
Kind
B2
Art Unit
2613
USPC
345/633
Abstract

A system, method and computer program products for prioritizing the stacking order of virtualized objects within a z-index of an augmented reality system by presenting the most valuable, relevant or important information more prominently. Prioritization of the virtualized object may occur dynamically, as a function of the user's primary focus. As the user's primary focus changes from one physical object to another, the stacking order may change in response to the shift in focus. Embodiments may use one or more techniques to identify the primary focus such as the direction of the user's gaze, the focal point of one or more visual recording systems (i.e. cameras) or the user may manually highlight one or more objects via the HUD or interface of the augmented display system. The disclosed embodiments allow for the more predominate or relevant virtualized objects within the augmented display to receive priority, unobscured viewing by the user.

Claims (50)

1. A method for prioritizing a stacking order of a z-index of virtual overlays in an augmented display system comprising the steps of:

determining, by a processor, a primary focus of a user operating an augmented display system;

detecting, by the processor, one or more objects viewed through the augmented display system;

designating, by the processor, each of the one or more objects as a primary object or a secondary object as a function of the primary focus of the user;

retrieving, by the processor, the virtual overlays from one or more data sources corresponding to each primary object and secondary object;

assigning, by the processor, the stacking order for each of the virtual overlays in the z-index of a heads-up display (HUD) of the augmented display system generated by the processor, wherein a stacking order of the virtual overlays of the primary object is higher than a stacking order of the virtual overlays of each secondary object; and

displaying, by the processor, each of the virtual overlays in the z-index of the HUD in accordance with the stacking order assigned, and juxtaposing each of the virtual overlays near each corresponding primary object or secondary object.

2. The method of claim 1 , wherein the stacking order of two or more virtual overlays of secondary objects are further assigned as a function of secondary factors selected from the group consisting of relevance to the user, social relationship, purchase history, environment, cognitive state and value.

3. The method of claim 1 , wherein the stacking order of two or more virtual overlays are assigned as a function of a value of a promotion, wherein a promotion having a highest promotional value is stacked highest in the stacking order and the stacking order dynamically adjusts based on the value of the promotion as the value of the promotion changes in real time.

4. The method of claim 1 , wherein the stacking order of two or more virtual overlays are assigned as a function of brand preference to the user, wherein more preferable brands are stacked higher in the stacking order than less preferable brands.

5. The method of claim 1 , wherein the stacking order of two or more virtual overlays are assigned as a function of a surrounding environment, wherein a primary object or secondary object that is more relevant to the surrounding environment is stacked higher in the stacking order.

6. The method of claim 1 , wherein the stacking order of two or more virtual overlays are assigned as a function of a cognitive state of the user and virtual overlays corresponding to a primary object or secondary object more relevant to the user's cognitive state is stacked higher in the stacking order.

7. The method of claim 1 , further comprises the steps of:

recording, by a camera of a visual recording system, video or images of the primary object or the secondary object to a display device; and

the step of displaying further comprises displaying the recorded video or images of the primary object or the secondary object as virtual representations on the display device as part of the HUD.

8. The method of claim 1 , further comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable program code in a computer system, where the computer-readable program code in combination with the computer system is configured to implement the steps of determining, detecting, designating, retrieving, assigning, and displaying.

9. A computer system, comprising:

a processor;

a memory device coupled to the processor;

a visual recording system comprising a camera, coupled to the processor;

a display device, coupled to the processor; and

a computer readable storage device coupled to the processor, wherein the storage device contains program code executable by the processor via the memory device to implement a method for prioritizing a stacking order of a z-index of virtual overlays comprising the steps of:

determining, by the processor, a primary focus of a user operating an augmented display system;

detecting, by the processor, one or more objects viewed through the augmented display system;

designating, by the processor, each of the one or more objects as a primary object or a secondary object as a function of the primary focus of the user;

retrieving, by the processor, the virtual overlays from one or more data sources corresponding to each primary object and secondary object;

assigning, by the processor, the stacking order for each of the virtual overlays in the z-index of a heads-up display (HUD) of the augmented display system generated by the processor, wherein a stacking order of the virtual overlays of the primary object is higher than a stacking order of the virtual overlays of each secondary object; and

displaying, by the processor, each of the virtual overlays in the z-index of the HUD in accordance with the stacking order assigned, and juxtaposing each of the virtual overlays near each corresponding primary object or secondary object.

10. The computer system of claim 9 , wherein the stacking order of two or more virtual overlays of secondary objects are further assigned as a function of secondary factors selected from the group consisting of relevance to the user, social relationship, purchase history, environment, cognitive state and value.

11. The computer system of claim 9 , wherein the stacking order of two or more virtual overlays are assigned as a function of a value of a promotion, wherein a promotion having a highest promotional value is stacked highest in the stacking order and the stacking order dynamically adjusts based on the value of the promotion as the value of the promotion changes in real time.

12. The computer system of claim 9 , wherein the stacking order of two or more virtual overlays are assigned as a function of brand preference to the user, wherein more preferable brands are stacked higher in the stacking order than less preferable brands.

13. The computer system of claim 9 , wherein the stacking order of two or more virtual overlays are assigned as a function of a surrounding environment, wherein a primary object or secondary object that is more relevant to the surrounding environment is stacked higher in the stacking order.

14. The computer system of claim 9 , further comprises:

recording, by the camera of the visual recording system, video or images of the primary object or the secondary object to the display device; and

displaying further comprises displaying the recorded video or images of the primary object or secondary object as virtual representations on the display device as part of the HUD.

15. The computer system of claim 9 , wherein the primary objects and secondary objects are re-designated and the stacking order of the virtual overlays are dynamically reassigned as a function of a change in the primary focus of a user.

16. A computer program product comprising:

one or more computer readable hardware storage devices having computer readable program code stored therein, said program code containing instructions executable by the one or more central processing units (CPU) of a computer system implementing a method for comparing products using augmented reality comprising the steps of:

determining, by the CPU, a primary focus of a user operating an augmented display system;

detecting, by the CPU, one or more objects viewed through the augmented display system;

designating, by the CPU, each of the one or more objects as a primary object or a secondary object as a function of the primary focus of the user;

retrieving, by the CPU, the virtual overlays from one or more data sources corresponding to each primary object and secondary object;

assigning, by the CPU, a stacking order for each of the virtual overlays in a z-index of a heads-up display (HUD) of the augmented display system generated by the CPU, wherein a stacking order of the virtual overlays of the primary object is higher than a stacking order of the virtual overlays of each secondary object; and

displaying, by the CPU, each of the virtual overlays in the z-index of the HUD in accordance with the stacking order assigned, and juxtaposing each of the virtual overlays near each corresponding primary object or secondary object.

17. The computer program product of claim 16 , further comprising:

recording by a camera of a visual recording system video or images of the primary object or the secondary object to a display device; and

the step of displaying further comprises displaying the recorded video or images of the primary object or secondary object as virtual representations on the display device as part of the HUD.

18. The computer program product of claim 16 , wherein the stacking order of two or more virtual overlays are assigned as a function of a value of a promotion, wherein a promotion having a highest promotional value is stacked highest in the stacking order and the stacking order dynamically adjusts based on the value of the promotion as the value of the promotion changes in real time.

19. The computer program product of claim 16 , wherein the stacking order of two or more virtual overlays are assigned as a function of brand preference to the user, wherein more preferable brands are stacked higher in the stacking order than less preferable brands.

20. The computer program product of claim 16 , wherein the stacking order of two or more virtual overlays are assigned as a function of a surrounding environment, wherein a primary object or secondary object that is more relevant to the surrounding environment is stacked higher in the stacking order.

Assignments (7)
SECURITY AGREEMENT Recorded May 20, 2026
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 075591/0399 →
SECURITY INTEREST Recorded Nov 10, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 073514/0326 →
SECURITY AGREEMENT Recorded Mar 13, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 070513/0542 →
SECURITY AGREEMENT Recorded Oct 10, 2024
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 069143/0399 →
SECURITY AGREEMENT Recorded Mar 24, 2021
From: WAYFAIR LLC
To: CITIBANK, N.A.
Reel/Frame 055708/0832 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 29, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: WAYFAIR LLC
Reel/Frame 050867/0899 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2017
From: DELUCA, LISA SEACAT; GREENBERGER, JEREMY A.; GREENBERGER, ZACHARY M.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 044325/0356 →
Continuity (1)
Related Publication 20190180483A1 · Jun 13, 2019
Cited By (16)
US 12,249,034 US 12,277,654 US 12,340,010 US 12,524,977 US 12,541,280 US 12,561,924 US 12,602,865 US 12,608,115 US 12,608,877 US 12,608,890 US 12,625,608 US 12,632,170 US 12,633,044 US 12,675,958 US 12,687,922 US 12,688,660