Granted Patent B2

US 12,444,217 · App. 17/540,516 · Granted Oct 14, 2025

Inventors: Manbinder Pal Singh (Coral Springs, FL); Christopher Fleck (Boca Raton, FL)

G06V30/147G02B27/017G06N3/08G06V30/153G02B2027/0138G02B2027/014

Abstract

Claims (84)

1. A computing device comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the computing device to:

provide, to a user and via an extended reality (XR) device, an XR environment;

detect one or more first locations of one or more display devices, wherein at least one first display device of the one or more display devices is in a physical environment around the XR device;

retrieve, from a server, one or more notifications for display in the XR environment;

determine, based on the one or more first locations of the one or more display devices and based on whether content is displayed by the one or more display devices or displayed in the physical environment, one or more second locations for the one or more notifications that are not displaying the content;

provide, in the XR environment and at the one or more second locations, the one or more notifications;

based on determining an unused portion of one of the one or more display devices, determine a second location of the one or more second locations to be the unused portion; and

provide, in the XR environment and at the second location, a notification of the one or more notifications.

2. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:

capture, using an imaging device of the XR device, one or more images of the physical environment around the XR device;

determine, based on the one or more images, at least one notification; and

send, to the server, the at least one notification, wherein the one or more notifications comprise the at least one notification.

3. The computing device of claim 2 , wherein the instructions, when executed by the one or more processors, further cause the computing device to determine the at least one notification by causing the computing device to:

determine text in the one or more images using an Optical Character Recognition (OCR) algorithm; and

determine the at least one notification based on the text.

4. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:

receive, in the XR environment, a user interaction with a first notification of the one or more notifications;

determine an action associated with the first notification; and

provide, via a second computing device, a user interface associated with the action.

5. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to determine the one or more second locations by causing the computing device to:

determine a bounding box corresponding to a first display device of the one or more display devices; and

determine the one or more second locations based on the bounding box.

6. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to determine the one or more second locations by causing the computing device to:

capture, using an imaging device of the XR device, one or more images of the physical environment around the XR device;

provide, to a trained machine learning model, input comprising the one or more images, wherein the trained machine learning model has been trained to indicate notification locations based on training data comprising a plurality of different images, wherein each of the plurality of different images comprise:

at least one display device; and

one or more written notes nearby the at least one display device; and

receive, from the trained machine learning model, output comprising an indication of the one or more second locations.

7. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:

receive, from the user, a selection of text;

generate, based on the text, at least one notification; and

send, to the server, the at least one notification, wherein the one or more notifications comprise the at least one notification.

8. The computing device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:

receive, in the XR environment, a user interaction with a physical note in the physical environment around the XR device;

determine a first notification corresponding to the physical note;

determine an action associated with the first notification; and

provide a user interface associated with the action.

9. The computing device of claim 1 , wherein the content displayed in the physical environment is displayed on one or more of: a notebook, a reference manual, a sticky note, or a user input device associated with the one or more display devices.

10. A method comprising:

providing, by a computing device to a user and via an extended reality (XR) device, an XR environment;

detecting one or more first locations of one or more display devices, wherein at least one first display device of the one or more display devices is in a physical environment around the XR device;

retrieving, from a server, one or more notifications for display in the XR environment;

determining, based on the one or more first locations of the one or more display devices and based on whether content is displayed by the one or more display devices or displayed in the physical environment, one or more second locations for the one or more notifications that are not displaying the content;

providing, in the XR environment and at the one or more second locations, the one or more notifications;

based on determining an unused portion of one of the one or more display devices, determining a second location of the one or more second locations to be the unused portion; and

providing, in the XR environment and at the second location, a notification of the one or more notifications.

11. The method of claim 10 , further comprising:

capturing, using an imaging device of the XR device, one or more images of the physical environment around the XR device;

determining, based on the one or more images, at least one notification; and

sending, to the server, the at least one notification, wherein the one or more notifications comprise the at least one notification.

12. The method of claim 11 , wherein determining the at least one notification comprises:

determining text in the one or more images using an Optical Character Recognition (OCR) algorithm; and

determining the at least one notification based on the text.

13. The method of claim 10 , further comprising:

retrieving, in the XR environment, a user interaction with a first notification of the one or more notifications;

determining an action associated with the first notification; and

providing, via a second computing device, a user interface associated with the action.

14. The method of claim 10 , wherein determining the one or more second locations comprises:

determining a bounding box corresponding to a first display device of the one or more display devices; and

determining the one or more second locations based on the bounding box.

15. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a computing device, cause the computing device to:

provide, to a user and via an extended reality (XR) device, an XR environment;

detect one or more first locations of one or more display devices, wherein at least one first display device of the one or more display devices is in a physical environment around the XR device;

retrieve, from a server, one or more notifications for display in the XR environment;

determine, based on the one or more first locations of the one or more display devices and based on whether content is displayed by the one or more display devices or displayed in the physical environment, one or more second locations for the one or more notifications that are not displaying the content;

provide, in the XR environment and at the one or more second locations, the one or more notifications;

based on determining an unused portion of one of the one or more display devices, determine a second location of the one or more second locations to be the unused portion; and

provide, in the XR environment and at the second location, a notification of the one or more notifications.

16. The non-transitory computer-readable media of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:

capture, using an imaging device of the XR device, one or more images of the physical environment around the XR device;

determine, based on the one or more images, at least one notification; and

send, to the server, the at least one notification, wherein the one or more notifications comprise the at least one notification.

17. The non-transitory computer-readable media of claim 16 , wherein the instructions, when executed by the one or more processors, further cause the computing device to determine the at least one notification by causing the computing device to:

determine text in the one or more images using an Optical Character Recognition (OCR) algorithm; and

determine the at least one notification based on the text.

18. The non-transitory computer-readable media of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the computing device to:

receive, in the XR environment, a user interaction with a first notification of the one or more notifications;

determine an action associated with the first notification; and

provide, via a second computing device, a user interface associated with the action.

19. The non-transitory computer-readable media of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the computing device to determine the one or more second locations by causing the computing device to:

determine a bounding box corresponding to a first display device of the one or more display devices; and

determine the one or more second locations based on the bounding box.

Assignments (9)

Continuity (1)

References Cited (15)