Systems and methods for dynamic identification of a surgical tray and the items contained thereon
The invention provides artificial intelligence-enabled image recognition methods and systems for continuously training a computer system to accurately identify a surgical item in a tray using at least 100 randomly created 2-dimensional images of a 3-dimensional synthetic item having unique identifiers assigned to the images or item. The invention also provides an artificial intelligence-enabled image recognition method and system for use to determine whether surgical instruments are present or missing on a surgical tray, and, if applicable, identifying those missing. In one aspect, a server receives an image and analyzes the image with a deep convolutional neural network to classify the type of tray and then compares a list of items that should be on the tray to that which the computer recognizes on the tray to generate an output displayed to a user identifying the items present and/or missing.
1. A system for dynamically identifying a surgical tray and items contained thereon, the system comprising:
a software application, the software application operating on a mobile computer device or a computer device in communication with at least one image data collection device configured to produce an image of the surgical tray, the software application is configured to receive the image of the surgical tray from the image data collection device and then communicate the image through a wired and/or wireless communication network to a server located at a site where the surgical tray is located or at a location remote from the site; and
a processor in communication through the wired and/or wireless communication network with the software application, as well as the server, the processer is configured to call up from a library database of the system, upon communication of the image to the server:
a plurality of tray identification models comprised of tray tensors, the tray identification models uploaded to the library database of the system;
whereby the processor is configured to:
analyze the image of the surgical tray and classify the type of tray in the image based on the tray identification models applied to the image,
call up from the library database:
a list of items linked to the classification of the type of tray, and a plurality of instrument identification models comprised of instrument tensors generated using a 3-dimensional synthetic item that exists entirely in a virtual environment, the instrument identification models linked to the items and uploaded to the library database of the system;
analyze the image and identify the type of items in the image based on the instrument identification models,
compare the classified items to the list of items linked to the classified tray to determine any missing items, and
notify the software application of the classified items and any missing items.
2. The system of claim 1 wherein the image data collection device is a camera.
3. The system of claim 1 wherein the image data collection device is mounted on a wearable device.
4. The system of claim 1 , wherein the tray identification models comprised of tray tensors and the instrument identification models comprised of instrument tensors are generated using a computer vision-driven artificial intelligence network trained using 2-dimensional views of the 3-dimensional synthetic item, as rendered by a view generation module.
5. The system of claim 4 , wherein the artificial intelligence network is a convolutional neural network.
6. The system of claim 4 , wherein the computer vision-driven artificial intelligence network is continuously trained using 2-dimensional views of the 3-dimensional synthetic item, as rendered by a view generation module.
7. A method for identifying a surgical tray and items contained thereon, the method comprising:
receiving an image of the surgical tray and items contained thereon from an image data collector connected to a server or a remote server using a software application operating on a mobile computer device or a computer device that may be synced with the mobile computer device, and wherein the mobile computer device or the computer device communicate through a wired and/or wireless communication network with the server at a site the surgical tray is located at or with a remote server in a location that is remote to the site and in communication with the server;
upon receiving the information, calling up from a database using a processor:
a plurality of tray identification models comprised of tray tensors, wherein the tray identification models including tray names and items intended to be contained in the trays have been previously uploaded to the database;
analyzing the image and classifying the type of tray in the image based on the tray identification models;
upon classifying the tray, calling up from the database a plurality of instrument identification models linked to the classification of the tray and comprised of instrument tensors generated using a 3-dimensional synthetic item that exists entirely in a virtual environment, the instrument identification models including: (a) surface texture, (b) item material composition, and (c) a size tolerance; and a list of items linked to the tray classification;
analyzing the image and classifying the type of items in the image based on the instrument identification models;
comparing the classified items to the list of items linked to the classified tray to determine any missing items, and
notifying the software application of the classified items and any missing items.
8. The method of claim 7 wherein the image data collection device is a camera.
9. The method of claim 7 wherein the image data collection device is mounted on a wearable device.
10. The method of claim 7 , wherein the tray identification models comprised of tray tensors and the instrument identification models comprised of instrument tensors are generated using a computer vision-driven artificial intelligence network trained using 2-dimensional views of the 3-dimensional synthetic item, as rendered by a view generation module.
11. The method of claim 10 , wherein the artificial intelligence network is a convolutional neural network.
12. The method of claim 10 , wherein the computer vision-driven artificial intelligence network is continuously trained using 2-dimensional views of the 3-dimensional synthetic items rendered by a view generation module.