IP Library Granted Patent US 11,914,761
Granted Patent B1
US 11,914,761 · App. 18/143,659 · Granted Feb 27, 2024

Systems and methods for virtual artificial intelligence development and testing

Inventors: Richard Raymond Schweet (Kirkland, WA); Bendenetto Christopher Ruggiero (Cincinnati, OH); Kyle Robert Hartshorn (Cincinnati, OH); Gregory Ryan Sweeney (Cincinnati, OH); Kyle Dean Cypher (Houston, TX); Melissa Yenni Scharf (Cincinnati, OH); Emily Ann Meyer (Denver, CO); Alec Brenders Lisy (Cincinnati, OH); Jeremy David Jarrett (Cincinnati, OH); Matthew David Fye (Cincinnati, OH)
Assignee: SAEC/Kinetic Vision, Inc.
G06F3/011G06N5/04G06N20/00G06Q30/0201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,914,761
App. No.
18/143,659
Granted
Feb 27, 2024
Kind
B1
Abstract

Systems and methods are provided to create training data, validate, deploy and test artificial intelligence (AI) systems in a virtual development environment, incorporating virtual spaces, objects, machinery, devices, subsystems, and actual human action and behavior.

Claims (54)

1. A computer implemented method, comprising:

provisioning, by a virtual environment computer system, a virtual environment, wherein the virtual environment is a digital twin virtual environment of a real-world physical environment; wherein the virtual environment comprises:

a plurality of virtual objects, wherein the plurality of virtual objects are each based on real-world physical objects, and

a virtual sensor, wherein the virtual sensor is modeled to generate a virtual output corresponding to an output of a real-world sensor of the real-world physical environment;

collecting, with an artificial intelligence computing system, information based on detection of at least one of the plurality of virtual objects and the virtual output of the virtual sensor for training of the artificial intelligence computing system; and

testing an accuracy of the artificial intelligence computing system, wherein testing the accuracy is based on the detection of the at least one of the plurality of virtual objects within the digital twin virtual environment.

2. The computer implemented method of claim 1 , wherein the real-world physical environment is any of a retail environment, an industrial environment, a medical environment, a marine environment, a manufacturing environment, a military environment, and an outdoor environment.

3. The computer implemented method of claim 1 , wherein one or more of the plurality of virtual objects are based on variations of retail products.

4. The computer implemented method of claim 1 , and wherein one or more of the plurality of virtual objects are modeled from real-world surgical instruments.

5. The computer implemented method of claim 1 , wherein at least one of the plurality of virtual objects is modeled from a human.

6. The computer implemented method of claim 1 , wherein at least one of the plurality of virtual objects is modeled from a robotic system.

7. The computer implemented method of claim 1 , wherein provisioning the virtual environment further comprises provisioning the virtual environment to a human actor through a virtual reality interface.

8. The computer implemented method of claim 7 , further comprising:

tracking, by the virtual environment computer system, physical motion of the human actor.

9. The computer implemented method of claim 5 , wherein the information collected from the digital twin virtual environment is based on an action of the human.

10. The computer implemented method of claim 1 , wherein testing the artificial intelligence computing system comprises varying an operational condition of the digital twin virtual environment.

11. The computer implemented method of claim 10 , wherein the operational condition is any of a lighting level of the digital twin virtual environment, a sensor position, a sensor type, a sensor parameter, a lens parameter, and a camera parameter.

12. The computer implemented method of claim 10 , varying the operational condition comprises varying as least one of the plurality of virtual objects provisioned in the virtual environment.

13. The computer implemented method of claim 10 , wherein testing the artificial intelligence computing system comprises providing data variability in the virtual environment provisioned by the virtual environment computer system.

14. The computer implemented method of claim 1 , wherein one or more of the plurality of virtual objects are based on a real-world physical object that is in existence at the time of the modeling.

15. The computer implemented method of claim 1 , wherein one or more of the plurality of virtual objects are based on a proposed real-world physical object.

16. The computer implemented method of claim 1 , wherein the virtual sensor comprises a camera.

17. The computer implemented method of claim 1 , wherein the virtual environment comprises a plurality of virtual sensors.

18. A computer implemented method, comprising:

provisioning, by a virtual environment computer system, a virtual environment, wherein the virtual environment is a digital twin virtual environment of a real-world physical environment comprising a plurality of real-world physical objects and a real-world sensor, wherein the virtual environment comprises a plurality of virtual objects and a virtual sensor, wherein each of the plurality of virtual objects are based on a respective one of the plurality of real-world physical objects, wherein the virtual sensor is any of a LiDAR system, a motion sensor, an infrared system, and a camera system;

collecting, with an artificial intelligence computing system, information from the digital twin virtual environment, wherein the information is based on detection of at least one of the plurality of virtual objects and an output of the virtual sensor; and

varying an operational condition of the digital twin virtual environment to test the accuracy of the artificial intelligence computing system.

19. The computer implemented method of claim 18 , wherein provisioning the virtual environment further comprises provisioning the virtual environment to a human actor through a virtual reality interface.

20. The computer implemented method of claim 19 , further comprising:

tracking, by the virtual environment computer system, physical motion of the human actor.

21. The computer implemented method of claim 18 , wherein the virtual environment comprises a plurality of different types of virtual sensors.

22. The computer implemented method of claim 18 , further comprising

automatically calibrating the artificial intelligence computing system based on whether the artificial intelligence computing system correctly detected the at least one of the plurality of virtual objects.

23. The computer implemented method of claim 18 , further comprising:

improving the accuracy of the artificial intelligence computing system by providing data variability in the virtual environment provisioned by the virtual environment computer system.

24. The computer implemented method of claim 18 , wherein at least one of the plurality of real-world physical objects is a human and wherein at least one of the plurality of virtual objects is an avatar of the human.

25. A computer implemented method, comprising:

provisioning, by a virtual environment computer system, a virtual environment to a human actor through a virtual reality interface, wherein the virtual environment is a digital twin virtual environment of a real-world physical environment, wherein the virtual environment comprises a virtual sensor modeled to generate a virtual output corresponding to an output of a real-world sensor, wherein a virtual object is presented to the human actor in the digital twin virtual environment through the virtual reality interface, wherein the virtual object is based on a real-world physical object associated with the real-world physical environment, wherein a physical training object is physically positioned proximate to the human actor;

tracking, by the virtual environment computer system, physical motion of the human actor;

tracking, by the virtual environment computer system, an interaction of the human actor with the physical training object;

collecting information from the digital twin virtual environment with an artificial intelligence computing system; and

testing the artificial intelligence computing system, wherein the testing is based on the tracking of the physical motion of the human actor, the tracking of the interaction of the human actor with the physical training object, and the detection of the virtual object.

26. The computer implemented method of claim 25 , wherein the physical training object and the human actor are in a studio.

27. The computer implemented method of claim 25 , wherein the virtual output of the virtual sensor is modeled to correspond to an output of a real-world sensor, where the real-world sensor is any of a LiDAR system, a motion sensor, an infrared system, and a camera system.

28. The computer implemented method of claim 27 , wherein the virtual environment comprises a plurality of virtual sensors, wherein each of the plurality of virtual sensors are modeled to correspond to an output of a respective real-world camera.

29. A computer implemented method, comprising:

provisioning, by a virtual environment computer system, a virtual environment, wherein the virtual environment is a digital twin virtual environment of a real-world physical environment; wherein the virtual environment comprises:

a plurality of sub-systems that are each modeled to generate a virtual output corresponding to an output of a respective real-world sub-system, wherein the plurality of sub-systems comprise any of a LiDAR system, a motion sensor, an infrared system, and a camera system;

a plurality of virtual objects, wherein the plurality of virtual objects are each modeled from real-world physical objects associated with the real-world physical environment, and

collecting, with an artificial intelligence computing system, information from the digital twin virtual environment to train the artificial intelligence computing system, wherein the information is based on detection of at least one of the plurality of virtual objects and the virtual output of the plurality of sub-systems, and wherein the artificial intelligence computing system is for use at the real-world physical environment that comprises the real-world sensor and the real-world sub-system; and

prior to use of the artificial intelligence computing system at the real-world physical environment, testing the artificial intelligence computing system, wherein the testing is based on the detection of the at least one of the plurality of virtual objects within the digital twin virtual environment.

30. The computer implemented method of claim 29 , wherein the plurality of sub-systems comprise the LiDAR system, the motion sensor, the infrared system, and the video system.

31. The computer implemented method of claim 29 , wherein at least one of the plurality of virtual objects is modeled from a human.

32. The computer implemented method of claim 29 , wherein provisioning the virtual environment further comprises provisioning the virtual environment to a human actor through a virtual reality interface.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE/RECEIVING PARTY DATA TO SAEC/KINETIC VISION, INC. PREVIOUSLY RECORDED ON REEL 066144 FRAME 0442. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST. Recorded Jan 23, 2024
From: JARRETT, JEREMY DAVID
To: SAEC/KINETIC VISION, INC.
Reel/Frame 066363/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2024
From: SCHWEET, RICHARD RAYMOND; RUGGIERO, BENDENETTO CHRISTOPHER; HARTSHORN, KYLE ROBERT; SWEENEY, GREGORY RYAN; CYPHER, KYLE DEAN; SCHARF, MELISSA YENNI; MEYER, EMILY ANN; LISY, ALEC BRENDERS
To: SAEC/KINETIC VISION, INC.
Reel/Frame 066160/0674 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2024
From: FYE, MATTHEW DAVID
To: SAEC/KINETIC VISION, INC.
Reel/Frame 066144/0439 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2024
From: JARRETT, JEREMY DAVID
To: JARRETT, JEREMY DAVID
Reel/Frame 066144/0442 →
Continuity (3)
Continuation 17752018 · May 24, 2022
Continuation 16919983 · Jul 2, 2020
Provisional Application 62870326 · Jul 3, 2019