Processing system, endoscope system, and processing method
A processing system includes a processor including hardware. The processor acquires an endoscope image in which a duodenal papilla including an oral protrusion is imaged, estimates a depth map of a region including the duodenal papilla from the endoscope image, estimates a summit line of the oral protrusion from the estimated depth map, and performs display processing so as to superimpose a guide display based on the estimated summit line on the endoscope image.
1 . A processing system comprising:
a processor including hardware,
the processor being configured to:
acquire an endoscope image in which a duodenal papilla including an oral protrusion is imaged;
estimate a depth map of a region including the duodenal papilla from the endoscope image;
estimate a summit line of the oral protrusion from the estimated depth map, wherein the summit line is a linear region corresponding to a ridge line of the oral protrusion; and
superimpose a guide display based on the estimated summit line on the endoscope image.
2 . The processing system as defined in claim 1 , wherein the guide display is an incision guide display in endoscopic sphincterotomy (EST).
3 . The processing system as defined in claim 1 , further comprising a memory that stores a trained model that has been trained to estimate the depth map from an input image,
wherein the processor inputs the endoscope image to the trained model to estimate the depth map.
4 . The processing system as defined in claim 1 , wherein the processor estimates the summit line based on a first endoscope image in a state where a treatment tool is not inserted and a second endoscope image in a state where the treatment tool is inserted.
5 . The processing system as defined in claim 4 , wherein the processor estimates the summit line based on a difference between a first depth map estimated from the first endoscope image and a second depth map estimated from the second endoscope image.
6 . The processing system as defined in claim 4 , further comprising a memory that stores a trained model that has been trained to estimate the depth map from an input image,
wherein the processor:
inputs the first endoscope image to the trained model to estimate a first depth map,
inputs the second endoscope image to the trained model to estimate a second depth map, and
estimates the summit line based on the first depth map and the second depth map.
7 . The processing system as defined in claim 6 , wherein the processor estimates the summit line based on a difference between the first depth map and the second depth map in a state where a position of the duodenal papilla seen in the first endoscope image and a position of the duodenal papilla seen in the second endoscope image are aligned with each other.
8 . The processing system as defined in claim 1 , wherein;
the processor:
detects a papillary orifice based on the endoscope image, and
estimates the summit line starting from the papillary orifice.
9 . The processing system as defined in claim 1 , wherein:
the processor:
detects a portion of a treatment tool seen in the endoscope image based on the endoscope image, and
estimates the summit line starting from a tip of the detected portion.
10 . The processing system as defined in claim 1 , wherein the processor superimpose, on the endoscope image, the guide display indicating between a twelve o'clock direction and an eleven o'clock direction when the estimated summit line extends in the twelve o'clock direction centered on one end of the estimated summit line on a papillary orifice side.
11 . The processing system as defined in claim 1 , wherein:
the processor:
estimates position information of a hooding fold,
estimates information about an incision length based on the estimated position information of the hooding fold, and
superimpose the guide display having a length based on the estimated incision length on the endoscope image.
12 . The processing system as defined in claim 1 , wherein;
the processor:
estimates whether or not an intraluminal pressure of a duodenum is within a range of an appropriate pressure based on the endoscope image, and
in a case where the intraluminal pressure of the duodenum is outside the range of the appropriate pressure, performs insufflation processing or deaeration processing so that the intraluminal pressure of the duodenum is within the range of the appropriate pressure.
13 . An endoscope system comprising:
the processing system as defined in claim 1 ; and
an endoscope.
14 . A processing method comprising:
acquiring an endoscope image in which a duodenal papilla including an oral protrusion is imaged;
estimating a depth map of a region including the duodenal papilla from the endoscope image;
estimating a summit line of the oral protrusion from the estimated depth map, wherein the summit line is a linear region corresponding to a ridge line of the oral protrusion; and
superimposing a guide display based on the estimated summit line on the endoscope image.
15 . The processing method as defined in claim 14 , wherein the guide display is an incision guide display in EST.
16 . The processing method as defined in claim 14 , further comprising estimating the summit line based on a first endoscope image in a state where a treatment tool is not inserted and a second endoscope image in a state where the treatment tool is inserted.
17 . The processing method as defined in claim 16 , further comprising estimating the summit line based on a difference between a first depth map estimated from the first endoscope image and a second depth map estimated from the second endoscope image.
18 . The processing method as defined in claim 17 , further comprising estimating the summit line based on a difference between the first depth map and the second depth map in a state where a position of the duodenal papilla seen in the first endoscope image and a position of the duodenal papilla seen in the second endoscope image are aligned with each other.
19 . The processing method as defined in claim 14 , further comprising:
detecting a papillary orifice based on the endoscope image, and
estimating the summit line starting from the papillary orifice.
20 . The processing method as defined in claim 14 , further comprising:
estimating position information of a hooding fold;
estimating information about an incision length based on the estimated position information of the hooding fold; and
superimposing the guide display having a length based on the estimated incision length on the endoscope image.