Surgical scene assessment based on computer vision
Implementations generally relate to surgical scene assessment based on computer vision. In some implementations, a method includes receiving a first image frame of a plurality of image frames associated with a surgical scene. The method further includes detecting one or more objects in the first image frame. The method further includes determining one or more positions corresponding to the one or more objects. The method further includes tracking each position of the one or more objects in other image frames of the plurality of image frames.
1. A system comprising:
one or more processors; and
logic encoded in one or more non-transitory computer-readable storage media for execution by the one or more processors and when executed operable to cause the one or more processors to perform operations comprising:
receiving a first image frame of a plurality of image frames associated with a surgical scene;
detecting one or more objects in the first image frame;
determining that at least one object of the one or more objects is a scissor tool;
determining one or more positions corresponding to the one or more objects;
determining a state of the scissor tool;
determining a degree in which the scissor tool is open or closed;
tracking each position of the one or more objects in other image frames of the plurality of image frames; and
predicting future positions of the one or more objects based on current positions tracked.
2. The system of claim 1 , wherein at least one object of the one or more objects is a surgical tool.
3. The system of claim 1 , wherein at least one object of the one or more objects is a gauze.
4. The system of claim 1 , wherein at least one object of the one or more objects is smoke, and wherein the logic when executed is further operable to cause the one or more processors to perform operations comprising:
estimating a level of detected smoke; and
controlling a smoke evacuator based on the level of detected smoke.
5. The system of claim 1 , wherein the detecting of the one or more objects in the first image frame is performed utilizing a convolutional neural network.
6. The system of claim 1 , wherein the operations are performed in real-time.
7. A non-transitory computer-readable storage medium with program instructions stored thereon, the program instructions when executed by one or more processors are operable to cause the one or more processors to perform operations comprising:
receiving a first image frame of a plurality of image frames associated with a surgical scene;
detecting one or more objects in the first image frame;
determining that at least one object of the one or more objects is a scissor tool;
determining one or more positions corresponding to the one or more objects;
determining a state of the scissor tool;
determining a degree in which the scissor tool is open or closed;
tracking each position of the one or more objects in other image frames of the plurality of image frames; and
predicting future positions of the one or more objects based on current positions tracked.
8. The computer-readable storage medium of claim 7 , wherein at least one object of the one or more objects is a surgical tool.
9. The computer-readable storage medium of claim 7 , wherein at least one object of the one or more objects is a gauze.
10. The computer-readable storage medium of claim 7 , wherein at least one object of the one or more objects is a bleeding region.
11. The computer-readable storage medium of claim 7 , wherein at least one object of the one or more objects is smoke.
12. The computer-readable storage medium of claim 7 , wherein the detecting of the one or more objects in the first image frame is performed utilizing a convolutional neural network.
13. The computer-readable storage medium of claim 7 , wherein the operations are performed in real-time.
14. A computer-implemented method comprising:
receiving a first image frame of a plurality of image frames associated with a surgical scene;
detecting one or more objects in the first image frame;
determining that at least one object of the one or more objects is a scissor tool;
determining one or more positions corresponding to the one or more objects;
determining a state of the scissor tool;
determining a degree in which the scissor tool is open or closed;
tracking each position of the one or more objects in other image frames of the plurality of image frames; and
predicting future positions of the one or more objects based on current positions tracked.
15. The method of claim 14 , wherein at least one object of the one or more objects is a surgical tool.
16. The method of claim 14 , wherein at least one object of the one or more objects is a gauze.
17. The method of claim 14 , wherein at least one object of the one or more objects is a bleeding region.
18. The method of claim 14 , wherein at least one object of the one or more objects is smoke.
19. The method of claim 14 , wherein the detecting of the one or more objects in the first image frame is performed utilizing a convolutional neural network.