Shape measurement system for endoscope and shape measurement method for endoscope
A display processing unit causes a display device to display an endoscopic image of a living tissue in a somatic cavity captured by an endoscope. An operation reception unit receives a user operation that is performed so as to set an area of interest in the endoscopic image. An area-of-interest setting unit sets the area of interest in the endoscopic image based on the user operation. A three-dimensional information acquisition unit acquires three-dimensional shape information of the living tissue captured by the endoscope. A virtual surface derivation unit derives three-dimensional shape information of a virtual surface in the area of interest from three-dimensional shape information of an area different from the area of interest. A size information identification unit identifies information concerning a size of the virtual surface from the three-dimensional shape information of the virtual surface.
1 . A shape measurement system for use with an endoscope, the shape measurement system comprising:
a processor comprising hardware, wherein the processor is configured to:
display an endoscopic image of a living tissue in a somatic cavity captured by the endoscope on a display device;
receive a first user operation for setting an area of interest in the endoscopic image;
set the area of interest in the endoscopic image based on the first user operation;
acquire three-dimensional shape information of the living tissue captured by the endoscope;
set a reference area surrounding the area of interest;
receive a second user operation for setting an exclusion area in the reference area;
derive three-dimensional shape information of a virtual surface in the area of interest from three-dimensional shape information of the reference area excluding three-dimensional information of the exclusion area; and
identify information concerning a size of the virtual surface from the three-dimensional shape information of the virtual surface.
2 . The shape measurement system according to claim 1 , wherein the processor is configured to:
receive a third user operation for setting the reference area in the endoscopic image; and
set the reference area in the endoscopic image based on the third user operation.
3 . The shape measurement system according to claim 1 , wherein the processor is configured to:
generate a three-dimensional image of the living tissue based on the three-dimensional shape information of the living tissue;
combine the three-dimensional image of the living tissue and the image showing the position of the area of interest; and
display the combined three-dimensional image of the living tissue and the image showing the position of the area of interest on the display device.
4 . The shape measurement system according to claim 1 , wherein the processor is configured to:
generate a three-dimensional image of the living tissue based on the three-dimensional shape information of the living tissue;
combine the three-dimensional image of the living tissue and the image showing the position of the reference area; and
display the combined three-dimensional image of the living tissue and the image showing the position of the reference area on the display device.
5 . The shape measurement system according to claim 3 , wherein the processor is configured to:
combine the virtual surface with the three-dimensional image of the living tissue; and
display the combined virtual surface with the three-dimensional image of the living tissue on the display device.
6 . The shape measurement system according to claim 1 , wherein the processor is configured to:
derive three-dimensional shape information of the virtual surface in the area of interest from three-dimensional shape information of the reference area by performing a fitting process for a three-dimensional shape.
7 . The shape measurement system according to claim 1 , further comprising:
a trained model that is generated by machine learning using endoscopic images for training and information concerning a lesion area contained in the endoscopic images as training data and that outputs a position of the lesion area when an endoscopic image is input, wherein
the processor is configured to:
display information indicating the position of the lesion area output by the trained model.
8 . The shape measurement system according to claim 1 , further comprising:
a trained model that is generated by machine learning using endoscopic images for training and information concerning a lesion area contained in the endoscopic images as training data and that outputs a position of the lesion area when an endoscopic image is input, wherein
the processor is configured to:
generate auxiliary information concerning an image-capturing range of the endoscope during image capturing based on the position of the lesion area output by the trained model; and
display the auxiliary information on the display device.
9 . The shape measurement system according to claim 1 , wherein the processor is configured to:
set the reference area based on the area of interest.
10 . A shape measurement method for use with an endoscope, the method comprising:
displaying an endoscopic image of a living tissue in a somatic cavity captured by the endoscope on a display device;
receiving a first user operation for setting an area of interest in the endoscopic image;
setting the area of interest in the endoscopic image based on the user operation;
acquiring three-dimensional shape information of the living tissue captured by the endoscope;
setting a reference area surrounding the area of interest;
receiving a second user operation for setting an exclusion area in the reference area;
deriving three-dimensional shape information of a virtual surface in the area of interest from three-dimensional shape information of the reference area excluding three-dimensional information of the exclusion area; and
identifying information concerning a size of the virtual surface from the three-dimensional shape information of the virtual surface.