IP Library Granted Patent US 12688793
Granted Patent B2
US 12688793 · App. 18/311,635 · Granted Jul 21, 2026

System and method for integrating a simulated reality training environment and an augmented reality environment

Inventors: Justin Schassler (Glendale, AZ); Robert Pickell (Atlanta, GA); Robert E. De Mers (Nowthen, MN); Luke Robertson (Wickford, GB); Raj Poojara (Katy, TX); Christian Salazar (Houston, TX); Ramesh Babu Koniki (Bangalore, IN)
Assignee: HONEYWELL INTERNATIONAL INC.
G09B19/003G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688793
App. No.
18/311,635
Granted
Jul 21, 2026
Kind
B2
Abstract

A method is disclosed for integrating a simulated reality training environment and an augmented reality environment. The method comprises receiving simulated reality training feedback associated with performance of a simulated task in the simulated reality training environment; determining a source of error in the performance of the simulated task in the simulated reality training environment based on the simulated reality training feedback; determining a preventative action for preventing occurrence of the source of error during a real-world task to be performed in the augmented reality environment; generating virtual content based on the preventative action to assist a user in avoiding the source of error during the real-world task to be performed in the augmented reality environment; and causing the virtual content to be displayed to the user in the augmented reality environment during performance of the real-world task.

Claims (76)

1 . A method for integrating a simulated reality training environment and an augmented reality environment, the method comprising:

receiving, by at least one processor, simulated reality training feedback associated with performance of a simulated task in the simulated reality training environment;

determining, by at least one processor, a source of error in the performance of the simulated task in the simulated reality training environment based on the simulated reality training feedback;

determining, by at least one processor, a preventative action for preventing occurrence of the source of error during a real-world task to be performed in the augmented reality environment;

generating, by at least one processor, virtual content based on the preventative action to assist a user in avoiding the source of error during the real-world task to be performed in the augmented reality environment; and

causing, by at least one processor, the virtual content to be displayed to the user in the augmented reality environment during performance of the real-world task.

2 . The method of claim 1 , wherein the simulated reality training feedback comprises at least one of:

a time spent performing individual portions of the simulated task;

a time spent performing the simulated task in its entirety; and

one or more errors made while performing the simulated task.

3 . The method of claim 1 , wherein the virtual content comprises at least one of:

a message;

a timer;

instructions for performing a step;

an audio feed to a remote person; and

an image to be overlaid on a real-world component.

4 . The method of claim 1 , further comprising:

gathering, by at least one processor, augmented reality feedback from the augmented reality environment; and

updating, by at least one processor, the simulated reality training environment based on the augmented reality feedback.

5 . The method of claim 4 , wherein the augmented reality feedback comprises at least one of:

a time spent performing individual portions of the real-world task;

a time spent performing the real-world task in its entirety; and

one or more errors made while performing the real-world task.

6 . The method of claim 1 , wherein the augmented reality environment is generated by one or more wearable or handheld devices.

7 . The method of claim 1 , wherein the simulated reality training environment is a virtual reality environment.

8 . A computer system for integrating a simulated reality training environment and an augmented reality environment, the computer system comprising:

a memory having processor-readable instructions stored therein; and

one or more processors configured to access the memory and execute the processor-readable instructions, which when executed by the one or more processors configures the one or more processors to perform a plurality of functions, including functions for:

receiving, by at least one processor, simulated reality training feedback associated with performance of a simulated task in the simulated reality training environment;

determining, by at least one processor, a source of error in the performance of the simulated task in the simulated reality training environment based on the simulated reality training feedback;

determining, by at least one processor, a preventative action for preventing occurrence of the source of error during a real-world task to be performed in the augmented reality environment;

generating, by at least one processor, virtual content based on the preventative action to assist a user in avoiding the source of error during the real-world task to be performed in the augmented reality environment; and

causing, by at least one processor, the virtual content to be displayed to the user in the augmented reality environment during performance of the real-world task.

9 . The system of claim 8 , wherein the simulated reality training feedback comprises at least one of:

a time spent performing individual portions of the simulated task;

a time spent performing the simulated task in its entirety; and

one or more errors made while performing the simulated task.

10 . The system of claim 8 , wherein the virtual content comprises at least one of:

a message;

a timer;

instructions for performing a step;

an audio feed to a remote person; and

an image to be overlaid on a real-world component.

11 . The system of claim 8 , wherein the plurality of functions further includes functions for:

gathering, by at least one processor, augmented reality feedback from the augmented reality environment; and

updating, by at least one processor, the simulated reality training environment based on the augmented reality feedback.

12 . The system of claim 11 , wherein the augmented reality feedback comprises at least one of:

a time spent performing individual portions of the real-world task;

a time spent performing the real-world task in its entirety; and

one or more errors made while performing the real-world task.

13 . The system of claim 8 , wherein the augmented reality environment is generated by one or more wearable or handheld devices.

14 . The system of claim 8 , wherein the simulated reality training environment is a virtual reality environment.

15 . A non-transitory computer-readable medium containing instructions for integrating a simulated reality training environment and an augmented reality environment, the instructions comprising:

receiving, by at least one processor, simulated reality training feedback associated with performance of a simulated task in the simulated reality training environment;

determining, by at least one processor, a source of error in the performance of the simulated task in the simulated reality training environment based on the simulated reality training feedback;

determining, by at least one processor, a preventative action for preventing occurrence of the source of error during a real-world task to be performed in the augmented reality environment;

generating, by at least one processor, virtual content based on the preventative action to assist a user in avoiding the source of error during the real-world task to be performed in the augmented reality environment; and

causing, by at least one processor, the virtual content to be displayed to the user in the augmented reality environment during performance of the real-world task.

16 . The non-transitory computer-readable medium of claim 15 , wherein the simulated reality training feedback comprises at least one of:

a time spent performing individual portions of the simulated task;

a time spent performing the simulated task in its entirety; and

one or more errors made while performing the simulated task.

17 . The non-transitory computer-readable medium of claim 15 , wherein the virtual content comprises at least one of:

a message;

a timer;

instructions for performing a step;

an audio feed to a remote person; and

an image to be overlaid on a real-world component.

18 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further comprise:

gathering, by at least one processor, augmented reality feedback from the augmented reality environment; and

updating, by at least one processor, the simulated reality training environment based on the augmented reality feedback.

19 . The non-transitory computer-readable medium of claim 18 , wherein the augmented reality feedback comprises at least one of:

a time spent performing individual portions of the real-world task;

a time spent performing the real-world task in its entirety; and

one or more errors made while performing the real-world task.

20 . The non-transitory computer-readable medium of claim 15 , wherein the augmented reality environment is generated by one or more wearable or handheld devices.