Granted Patent B2

US 12,640,049 · App. 18/206,461 · Granted May 26, 2026

Inventor: Kyler Lemery (Grain Valley, MO)

G09B9/052G06T19/006

Abstract

Claims (56)

1 . A method of training a user in forklift operation, the method comprising:

booting up a virtual reality forklift simulator having a virtual reality environment and a physics simulation system configured to simulate a virtual reality forklift comprising a body, a set of tynes, and four wheels, the four wheels each having a rigid body and a circle collider that provides friction with virtual ground in the virtual reality environment, the virtual reality forklift further comprising four wheel colliders configured to provide a three-point suspension system for the virtual reality forklift that defines a safety triangle for a center of mass of the virtual reality forklift, wherein the virtual reality forklift simulator bases movements of the virtual reality forklift on physics simulations that dynamically update the center of mass of the virtual reality forklift based on at least a weight and a position of cargo carried on the tynes, a height and tilt of the tynes, a speed of the virtual reality forklift, and a slope of virtual ground in the virtual reality environment;

instructing the user to complete one or more objectives using a virtual reality forklift;

completing the one or more objectives using one or more controllers;

determining a respective force on each of the four wheels based on the dynamically updated center of mass of the virtual reality forklift and the three-point suspension system implemented by the four wheel colliders;

determining, based on the respective forces, a state of the safety triangle, wherein the state of the safety triangle comprises one of a safe state, an approaching unsafe state, or an unsafe state, wherein the unsafe state occurs when the respective force on one of the four wheels exceeds a support threshold;

providing guidance and/or warnings to the user based on a performance of the user in the virtual reality forklift, the guidance and/or warnings provided by an assist overlay system in the virtual reality forklift simulator, the assist overlay system comprising a safety triangle indicator that graphically indicates a level of danger to stability of the virtual reality forklift based on the safety triangle;

in response to determining the approaching unsafe state, causing the assist overlay system to display, via the safety triangle indicator, a tilted virtual reality forklift warning; and

in response to determining the unsafe state or detecting at least one of a crash, a mishandling of cargo, a dropped cargo, or an excessive speed that exceeds a safety speed threshold, halting the virtual reality forklift simulator, displaying a warning describing an unsafe condition to the user, and requiring the user to restart at least a current objective of the one or more objectives before resuming operation of the virtual reality forklift.

2 . The method of claim 1 , wherein the one or more controllers comprise:

a right hand controller configured to be grasped by a right hand of the user, the right hand controller having a grip button and a trigger;

a left hand controller configured to be grasped by a left hand of the user, the left hand controller having a grip button and a trigger button; and

a headset, wherein the headset allows the user to see the virtual reality environment,

wherein pressing the grip buttons allows the user to grip an object in the virtual reality environment,

wherein pressing the trigger button of the right hand controller accelerates the virtual reality forklift, and

wherein pressing the trigger button of the left hand controller engages a brake on the virtual reality forklift.

3 . The method of claim 2 , wherein the one or more objectives comprise:

securing safety equipment of the virtual reality forklift;

driving the virtual reality forklift using various virtual reality forklift controls;

loading cargo on a set of tynes of the virtual reality forklift using the various virtual reality forklift controls;

transporting cargo using the virtual reality forklift using the various virtual reality forklift controls; and

unloading cargo using the various virtual reality forklift controls.

4 . The method of claim 1 , wherein the guidance is one or more of an objective arrow, highlighting a slot or slots of a pallet, prompts to use various virtual reality forklift controls, audio prompts, a guidance tablet with a text description of the one or more objectives.

5 . The method of claim 1 , further comprising providing an evaluation to the user.

6 . The method of claim 1 , wherein the virtual reality forklift and the virtual reality environment simulate real world physics.

7 . The method of claim 1 , further comprising generating an evaluation report for the user, wherein the evaluation report comprises a time to complete each of the one or more objectives, a number of successfully complete objectives, unsafe conditions encountered, instances of improper use of the virtual reality forklift, and a comparison of past performance to current performance.

8 . The method of claim 1 , wherein the body has a first rigid body encompassing a lower portion of the body, a second rigid body extending to a top of an overhead guard of body, and a third rigid body encompassing a top portion of a mast of the body.

9 . The method of claim 8 , wherein each of the tynes has a corresponding rigid body.

10 . The method of claim 9 , further comprising: determining a tyne center of mass based on the corresponding rigid body of each of the tynes, wherein the center of mass of the virtual reality forklift is based, at least in part, on the tyne center of mass and the first rigid body, the second rigid body, and the third rigid body.

11 . The method of claim 1 , further comprising, for at least one of the one or more objectives, randomizing at least one of a weight, a size, and an initial position of cargo to be carried on the set of tynes such that, between different training sessions, the dynamically updated center of mass of the virtual reality forklift and a resulting state of the safety triangle differ for the same objective.

12 . A non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations for training a user in forklift operation, the operations comprising:

receiving a first input from the user to boot-up a virtual reality forklift simulator having a virtual reality environment and a physics simulation system configured to simulate a virtual reality forklift comprising a body, a set of tynes, and four wheels, the four wheels each having a rigid body and a circle collider that provides friction with virtual ground in the virtual reality environment, the virtual reality forklift further comprising four wheel colliders configured to provide a three-point suspension system for the virtual reality forklift that defines a safety triangle for a center of mass of the virtual reality forklift, wherein the virtual reality forklift simulator bases movements of the virtual reality forklift on physics simulations that dynamically update the center of mass of the virtual reality forklift based on at least a weight and a position of cargo carried on the tynes, a height and tilt of the tynes, a speed of the virtual reality forklift, and a slope of virtual ground in the virtual reality environment;

receiving one or more inputs from one or more controllers operated by the user to move a virtual reality forklift to complete one or more objectives;

determining a respective force on each of the four wheels based on the dynamically updated center of mass of the virtual reality forklift and the three-point suspension system implemented by the four wheel colliders;

determining, based on the respective forces, a state of the safety triangle, wherein the state of the safety triangle comprises one of a safe state, an approaching unsafe state, or an unsafe state, wherein the unsafe state occurs when the respective force on one of the four wheels exceeds a support threshold;

providing guidance and/or warning to the user based on a performance of the user with the virtual reality forklift, the guidance and/or warnings provided by an assist overlay system in the virtual reality forklift simulator, the assist overlay system comprising a safety triangle indicator that graphically indicates a level of danger to stability of the virtual reality forklift based on the safety triangle;

in response to determining the approaching unsafe state, causing the assist overlay system to display, via the safety triangle indicator, a tilted virtual reality forklift warning; and

in response to determining the unsafe state or detecting at least one of a crash, a mishandling of cargo, a dropped cargo, or an excessive speed that exceeds a safety speed threshold, halting the virtual reality forklift simulator, displaying a warning describing an unsafe condition to the user, and requiring the user to restart at least a current objective of the one or more objectives before resuming operation of the virtual reality forklift.

13 . The non-transitory computer readable medium of claim 12 , wherein the non-transitory computer readable medium stores data from a user session.

14 . The non-transitory computer readable medium of claim 13 , wherein the data comprises successful completion of objectives, failure of objectives, unsafe operations of the virtual reality forklift, and time between boot-up and shut-down.

15 . The non-transitory computer readable medium of claim 12 , wherein the one or more controllers comprise:

a right hand controller configured to be grasped by a right hand of the user having a grip button and a trigger button;

a left hand controller configured to be grasped by a left hand of the user having a grip button and a trigger button; and

a headset configured to allow the user to see a virtual reality environment,

wherein pressing the trigger button of the right hand controller sends an acceleration input to the non-transitory computer readable medium to move the virtual reality forklift, and

wherein pressing the trigger button of the left hand controller sends a brake input to the non-transitory computer readable medium to slow the virtual reality forklift or stop the virtual reality forklift.

16 . The non-transitory computer readable medium of claim 15 , wherein the one or more objectives comprise:

securing safety equipment of the virtual reality forklift by providing inputs to the one or more controllers;

driving the virtual reality forklift by providing inputs to the one or more controllers;

loading cargo on a set of tynes of the virtual reality forklift by providing inputs to the one or more controllers;

transporting cargo using the virtual reality forklift by providing inputs to the one or more controllers; and

unloading cargo by providing inputs to the one or more controllers.

17 . The non-transitory computer readable medium of claim 16 , wherein the user provides inputs to the right hand controller, the left hand controller, and the headset.

18 . The non-transitory computer readable medium of claim 17 , wherein the inputs to the right hand controller, the left hand controller, and the headset provide the inputs to the non-transitory computer readable medium causing the at least one processor to move the virtual reality forklift and objects in the virtual reality environment.

19 . The non-transitory computer readable medium of claim 12 , wherein the non-transitory computer readable medium and processor simulate a real world forklift in the virtual reality forklift simulator.

20 . The non-transitory computer readable medium of claim 12 , wherein the at least one processor is configured to provide an evaluation to the user after completion of a training session.

Continuity (1)

References Cited (17)