Surgical instrument detection system and computer program
A surgical instrument detection system is provided that can determine the kinds of surgical instruments without special processing, such as application of an optically readable symbol, to the surgical instruments. A surgical instrument detection system 100 includes: an image input section 31 to input an image taken by a camera 1 ; an object extraction section 32 to clip an object image of a small steel article from the input image; a determination section 33 to input the object image to a learned classification model 331 and determine a kind of the small steel article based on features included in the object image; and an output image generation section 34 to generate an image representing the result of determination by the determination section and output such image to a monitor 2.
1. A surgical instrument detection system, including:
an image input section to input an image taken by a camera;
an object extraction section to clip an object image of a surgical instrument from the input image;
a determination section to input the object image to a learned classification model and determine a kind of the surgical instrument together with probability of the decision based on features included in the object image; and
an output image generation section to generate an image representing the result and probability of determination by the determination section and output such image to a monitor, wherein
the determination section outputs data indicating the kind of the surgical instrument output from the learned classification model as the result of determination for the object image to the output image generation section when the probability of the decision to the individual object image is greater than a predetermined value, and outputs “unknown” as the result of determination for the object image to the output image generation section when the probability of the decision is the predetermined value or less; and
the surgical instrument detection system accepts correction of the result of the determination from a user, thereby updates the learned classification model if there is an object output as unknown or falsely recognized,
wherein the learned classification model is updated in its actual operation.
2. The surgical instrument detection system of claim 1 , wherein the output image generation section outputs an image representing the kind of the surgical instrument overlaid on the object image in the taken image.
3. The surgical instrument detection system of claim 1 , wherein the object extraction section inputs the taken image to a learned object extraction model and clips the object image based on features included in the taken image.
4. The surgical instrument detection system of claim 1 , wherein the object extraction section converts pixel data of the taken image to data in an HSV color space and clips the object image based on edge information on hue and saturation.
5. The surgical instrument detection system of claim 1 , further comprising a comparison section to compare the kinds and number of surgical instruments after surgery with the kinds and number of surgical instruments before surgery, wherein
the output image generation section generates an image representing the result of comparison by the comparison section and outputs such image to the monitor.
6. A non-transitory computer readable storage medium comprising a program causing a processor of a computer to execute surgical instrument detection, wherein
the surgical instrument detection comprises instructions for inputting an image taken by a camera;
clipping an object image of a surgical instrument from the input image;
determining, by inputting the object image to a learned classification model, a kind of the surgical instrument together with probability of the decision based on features included in the object image; and
generating an image representing the result and probability of determination and outputting such image to a monitor, wherein
data indicating the kind of the surgical instrument output from the learned classification model as the result of determination for the object image is utilized for the generating the image representing the result and probability of determination when the probability of the decision to the individual object image is greater than a predetermined value, and the learned classification model outputs “unknown” as the result of determination for the object image for the generating the image representing the result and probability of determination when the probability of the decision is the predetermined value or less, and
the surgical instrument detection accepts correction of the result of the determination from a user, thereby updates the learned classification model if there is an object output as unknown or falsely recognized,
wherein the learned classification model is updated in its actual operation.
7. A surgical instrument detection method executed by a processor of a computer, the method comprising:
inputting an image taken by a camera;
clipping an object image of a surgical instrument from the input image;
determining, by inputting the object image to a learned classification model, a kind of the surgical instrument together with probability of the decision based on features included in the object image; and
generating an image representing the result and probability of determination and outputting such image to a monitor, wherein
data indicating the kind of the surgical instrument output from the learned classification model as the result of determination for the object image is utilized for the generating the image representing the result and probability of determination when the probability of the decision to the individual object image is greater than a predetermined value, and the learned classification model outputs “unknown” as the result of determination for the object image for the generating the image representing the result and probability of determination when the probability of the decision is the predetermined value or less, and
the surgical instrument detection method accepts correction of the result of the determination from a user, thereby updates the learned classification model if there is an object output as unknown or falsely recognized,
wherein the learned classification model is updated in its actual operation.
8. A surgical instrument detection system, including:
an image input section to input an image taken by a camera;
an object extraction section to clip an object image of a surgical instrument from the input image;
a determination section to input the object image to a learned classification model and determine a kind of the surgical instrument together with probability of the decision based on features included in the object image; and
an output image generation section to generate an image representing the result and probability of determination by the determination section and output such image to a monitor, wherein
the surgical instrument detection system accepts correction of the result of the determination from a user, thereby updates the learned classification model,
the object extraction section clips the object image based on comparison of a color space vector for each pixel in a background image and the taken image.