Photography support device, image-capturing device, and control method of image-capturing device

1 . A photography support device configured to support photography of N types (where N is an integer greater than 1) of photography target parts defined in advance, the photography support device comprising at least one memory and at least one processor which function as:

a determining unit configured to determine, with respect to a plurality of images obtained by photography of inside of an oral cavity of a patient, which of the N types of photography target parts a photography target part in each image corresponds to, and to output a certainty that indicates a probability of determination results, along with the determination results of the photography target part in each image;

a missing-image sensing unit configured to, based on the determination results regarding each of the plurality of images, sense whether or not shooting of images of the N types of photography target parts is complete; and

a support information output unit configured to output information for supporting photography of the N types of photography target parts by a photographer, using at least one of the determination results by the determining unit, and sensing results by the missing-image sensing unit,

wherein the support information output unit displays all or part of the plurality of images as a plurality of candidate images, and outputs an image-selection graphical user interface (GUI) by which the photographer is caused to select an image from the plurality of candidate images, and

wherein the support information output unit displays, in the image-selection GUI, the plurality of candidate images in a manner of being grouped by the photography target part, and displays, in the image-selection GUI, certainty information of each candidate image along with the plurality of candidate images.

2 . The photography support device according to claim 1 , wherein

in a case in which a photography target part that is missing is sensed by the missing-image sensing unit, the support information output unit performs notification to the photographer.

3 . The photography support device according to claim 2 , wherein

the support information output unit performs the notification at a timing of ending of photography of the patient.

4 . The photography support device according to claim 3 , wherein

the timing of ending of photography of the patient is one timing of a timing of sensing an instruction for ending of photography, a timing of sensing an instruction for externally transmitting the images, and a timing of sensing switching of the patient to be photographed.

5 . The photography support device according to claim 1 , wherein

the support information output unit outputs a status of photography indicating whether photography-completed or unphotographed, while photography of the patient is being performed, for each of the N types of photography target parts.

6 . The photography support device according to claim 5 , wherein

the support information output unit displays N shapes indicating the status of photography for each of the N types of photography target parts, and differentiates a form of the shapes for a photography-completed photography target part and an unphotographed photography target part.

7 . The photography support device according to claim 1 , wherein

the support information output unit displays, in the image-selection GUI, a candidate image group of a same photography target part, in descending order of certainty.

8 . The photography support device according to claim 1 , wherein,

in a case in which a plurality of candidate images for a same photography target part exist, the support information output unit recommends to the photographer, in the image-selection GUI, a candidate image of which the certainty is highest.

9 . The photography support device according to claim 1 , wherein

the support information output unit outputs, to a display with which an image-capturing device used for photography of inside of the oral cavity of the patient is equipped, information for supporting photography of the N types of photography target parts by the photographer.

10 . The photography support device according to claim 1 , wherein

the determining unit determines, based on image features extracted from each of the plurality of images, which of the N types of photography target parts the photography target part in each image corresponds to.

11 . The photography support device according to claim 1 , wherein

the determining unit determines, based on feature objects photographed in each of the plurality of images, which of the N types of photography target parts the photography target part in each image corresponds to.

12 . The photography support device according to claim 11 , wherein

the feature objects include a cheek retractor, a tongue, or palate ridges.

13 . The photography support device according to claim 1 , wherein

the determining unit determines which of the N types of photography target parts each of the plurality of images corresponds to, using a trained model that is trained to output the photography target part corresponding to an input image.

14 . An image-capturing device, comprising:

the photography support device according to claim 1 ; and

a display that displays information output from the photography support device.

15 . A control method of an image-capturing device, the control method comprising:

storing an image obtained by photography of inside of an oral cavity of a patient in a storage;

determining which of N types (where N is an integer greater than 1) of photography target parts defined in advance a photography target part in the image corresponds to;

outputting a certainty that indicates a probability of determination results, along with the determination results of the photography target part in each image;

sensing, based on determination results regarding each of a plurality of images of the patient stored in the storage, whether or not shooting of images of the N types of photography target parts is complete;

outputting information for supporting photography of the N types of photography target parts by a photographer, using at least one of the determination results, and sensing results;

displaying all or part of the plurality of images as a plurality of candidate images, and outputs an image-selection graphical user interface (GUI) by which the photographer is caused to select an image from the plurality of candidate images; and

displaying, in the image-selection GUI, the plurality of candidate images in a manner of being grouped by the photography target part, and displaying, in the image-selection GUI, certainty information of each candidate image along with the plurality of candidate images.

16 . A non-transitory computer-readable medium that stores a program for causing a computer to execute a control method of an image-capturing device, the control method comprising:

storing an image obtained by photography of inside of an oral cavity of a patient in a storage;

determining which of N types (where N is an integer greater than 1) of photography target parts defined in advance a photography target part in the image corresponds to;

outputting a certainty that indicates a probability of determination results, along with the determination results of the photography target part in each image;

sensing, based on determination results regarding each of a plurality of images of the patient stored in the storage, whether or not shooting of images of the N types of photography target parts is complete;

outputting information for supporting photography of the N types of photography target parts by a photographer, using at least one of the determination results, and sensing results;

displaying all or part of the plurality of images as a plurality of candidate images, and outputs an image-selection graphical user interface (GUI) by which the photographer is caused to select an image from the plurality of candidate images; and

displaying, in the image-selection GUI, the plurality of candidate images in a manner of being grouped by the photography target part, and displaying, in the image-selection GUI, certainty information of each candidate image along with the plurality of candidate images.