Image processing apparatus and control method for an image processing apparatus that extract a region of interest based on a calculated confidence of unit regions and a modified reference value
An image processing apparatus includes an acquisition unit configured to acquire image data, a calculation unit configured to calculate, for each unit region of the image data, a confidence that the unit region is an extraction subject, the confidence as the extraction subject being calculated for each unit region of the image data by inputting the image data into a trained model of a neural network that has been trained using images of an existing extraction subject or a region of interest as training data, a modification unit configured to modify a reference value of the confidence, which is used to extract a region of interest, an extraction unit configured to extract the region of interest on the basis of the calculated confidence of each unit region and the modified reference value, and a display unit configured to display the extracted region of interest.
1. An image processing apparatus comprising:
one or more processors and a memory storing instruction that, when executed by the one or more processors, cause the image processing apparatus to function as:
an acquisition unit configured to acquire image data;
a calculation unit configured to calculate, for each unit region of the image data, a confidence that the unit region is an extraction subject, the confidence as the extraction subject being calculated for each unit region of the image data by inputting the image data into a trained model of a neural network that has been trained using images of an existing extraction subject or a region of interest as training data;
a modification unit configured to modify a reference value of the confidence, which is used to extract a region of interest;
an extraction unit configured to extract the region of interest on the basis of the calculated confidence of each unit region and the modified reference value;
a display unit configured to display the extracted region of interest; and
an assessment unit configured to calculate a surface area of the region of interest on the basis of inclination and variation in a depth direction of the subject, which are detected from the image data.
2. The image processing apparatus according to claim 1 , wherein the modification unit modifies the reference value of the confidence on the basis of a user operation.
3. The image processing apparatus according to claim 1 , wherein the modification unit modifies the reference value of the confidence on the basis of a user operation on a GUI (Graphical User Interface).
4. The image processing apparatus according to claim 1 , wherein the modification unit modifies the reference value of the confidence on the basis of the confidence relating to a unit region selected by the user.
5. The image processing apparatus according to claim 1 , wherein the extraction unit extracts, as the region of interest, a set of unit regions in which values of the calculated confidence of the unit regions equal or exceed the modified reference value of the confidence.
6. The image processing apparatus according to claim 1 , wherein the calculation unit calculates, for each unit region of the image data, a confidence as the extraction subject and a confidence as another extraction subject, the modification unit modifies at least one of a reference value of the confidence as the extraction subject and a reference value of the confidence as the other extraction subject, and the extraction unit extracts the region of interest on the basis of the confidence as the extraction subject, the confidence as the other extraction subject, and the modified reference value.
7. The image processing apparatus according to claim 1 , wherein the extraction unit re-extracts a partial region that partially overlaps the region of interest on the basis of the confidence relating to a unit region selected by the user.
8. The image processing apparatus according to claim 7 , wherein the partial region is a region including the unit region selected by the user.
9. The image processing apparatus according to claim 7 , wherein the extraction unit re-extracts a region outside the partial region on the basis of the reference value prior to the re-extraction.
10. The image processing apparatus according to claim 1 , wherein, when the region of interest is divided into a plurality of regions, the extraction unit re-extracts a partial region that overlaps at least any of the plurality of regions on the basis of the confidence relating to a unit region selected by the user.
11. The image processing apparatus according to claim 1 , further comprising a storage unit configured to store information relating to a region of interest that is re-extracted on the basis of the modified reference value by the extraction unit when the reference value of the confidence has been modified by the modification unit,
wherein the calculation unit modifies the reference value of the confidence on the basis of the stored information relating to the region of interest.
12. The image processing apparatus according to claim 11 , wherein, when the reference value of the confidence is modified, the storage unit stores the modified reference value, and the extraction unit extracts the region of interest on the basis of the stored reference value of the confidence.
13. The image processing apparatus according to claim 1 , wherein the image data include an affected-area region, and, when the reference value of the confidence is increased, the extraction unit extracts, as the region of interest, a more central region of the affected-area region than before the reference value of the confidence is increased.
14. A control method of controlling an image processing apparatus, the method comprising:
an acquisition step of acquiring image data;
a calculation step of calculating, for each unit region of the image data, a confidence that the unit region is an extraction subject, the confidence as the extraction subject being calculated for each unit region of the image data by inputting the image data into a trained model of a neural network that has been trained using images of an existing extraction subject or a region of interest as training data;
a modification step of modifying a reference value of the confidence, which is used to extract the region of interest;
an extraction step of extracting the region of interest on the basis of the calculated confidence of each unit region and the modified reference value;
a display step of displaying the extracted region of interest; and
an assessment step of calculating a surface area of the region of interest on the basis of inclination and variation in a depth direction of the subject, which are detected from the image data.
15. A non-transitory computer readable medium that stores a program, wherein the program causes a computer to execute:
an acquisition step of acquiring image data;
a calculation step of calculating, for each unit region of the image data, a confidence that the unit region is an extraction subject, the confidence as the extraction subject being calculated for each unit region of the image data by inputting the image data into a trained model of a neural network that has been trained using images of an existing extraction subject or a region of interest as training data;
a modification step of modifying a reference value of the confidence, which is used to extract the region of interest;
an extraction step of extracting the region of interest on the basis of the calculated confidence of each unit region and the modified reference value;
a display step of displaying the extracted region of interest; and
an assessment step of calculating a surface area of the region of interest on the basis of inclination and variation in a depth direction of the subject, which are detected from the image data.