Method and apparatus for generating high depth of field image, and apparatus for training high depth of field image generation model using stereo image
A high depth of field image generating apparatus according to the present disclosure includes a region segmentation unit which segments a region for a stereo image to generate region data, a depth estimating unit which estimates depths for the stereo image to generate depth data, and a high depth of field image generating unit which generates a high depth of field image from the stereo image, the region data, and the depth data.
1. A high depth of field image generating apparatus for generating a high depth of field image from a stereo image capturing a tissue or a cell and including a left image and a right image, the apparatus comprising a processor and instructions executable by the processor to implement:
a region segmentation unit which segments a region for the stereo image to generate region data;
a depth estimating unit which estimates depths of the cell or the tissue for the stereo image to generate depth data; and
a high depth of field image generating unit which generates the high depth of field image from the stereo image, the region data, and the depth data,
wherein the high depth of field image generating unit combines the left image and the right image in order that cells or tissues segmented from the left image and cells or tissues segmented from the right image are included in the high depth of field image.
2. The high depth of field image generating apparatus according to claim 1 , wherein the high depth of field image generating unit generates the high depth of field image using a trained deep learning model.
3. The high depth of field image generating apparatus according to claim 2 , wherein the region segmentation unit or the depth estimating unit generates the region data or the depth data using the trained deep learning model, respectively.
4. The high depth of field image generating apparatus according to claim 1 , wherein the region segmentation unit segments the region for each image which configures the stereo image to generate the region data, and the depth estimating unit generates the depth data for each segmented region.
5. The high depth of field image generating apparatus according to claim 1 , wherein the stereo image is obtained by capturing the tissue or the cell.
6. The high depth of field image generating apparatus according to claim 2 , wherein the trained deep learning model is implemented to simulate blind deconvolution using a point-spread function.
7. A high depth of field image generating method for generating a high depth of field image from a stereo image capturing a tissue or a cell and including a left image and a right image, the method comprising:
a region segmentation step of segmenting a region for the stereo image to generate region data;
a depth estimating step of estimating depths of the cell or the tissue for the stereo image to generate depth data; and
a high depth of field image generating step of generating a high depth of field image from the stereo image, the region data, and the depth data,
wherein the high depth of field image generating step combines the left image and the right image in order that cells or tissues segmented from the left image and cells or tissues segmented from the right image are included in the high depth of field image.
8. The high depth of field image generating method according to claim 7 , wherein in the high depth of field image generating step, a trained deep learning model is used.
9. The high depth of field image generating method according to claim 8 , wherein the region segmentation step and the depth estimating step use the trained deep learning model.
10. The high depth of field image generating method according to claim 7 , wherein the region segmentation step segments the region for each image which configures the stereo image to generate the region data and the depth estimating step generates the depth data for each segmented region.
11. The high depth of field image generating method according to claim 7 , wherein the stereo image is obtained by capturing the tissue or the cell.
12. The high depth of field image generating method according to claim 8 , wherein the trained deep learning model is implemented to simulate blind deconvolution using a point-spread function.
13. An apparatus for training a high depth of field image generation model, the apparatus comprising a processor and instructions executable by the processor to implement:
a learning model implemented to output a high depth of field image from an input stereo image capturing a tissue or a cell and including a left image and a right image, wherein the learning model comprises a region segmentation unit which segments a region for the stereo image to generate region data; a depth estimating unit which estimates depths of the cell or the tissue for the stereo image to generate depth data; a high depth of field image generating unit which generates the high depth of field image from the stereo image, the region data, and the depth data,
wherein the high depth of field image generating unit combines the left image and the right image in order that cells or tissues segmented from the left image and cells or tissues segmented from the right image are included in the high depth of field image; and
a training unit which trains the learning model with learning data including the stereo image and the high depth of field image corresponding thereto.
14. The apparatus according to claim 13 , wherein the training unit calculates a cost function from the high depth of field image output from the learning model and a high depth of field reference image and trains the learning model using the cost function.
15. The apparatus according to claim 13 , wherein the high depth of field image generating unit generates the high depth of field image using a deep learning model.
16. The apparatus according to claim 15 , wherein the region segmentation unit or the depth estimating unit generates the region data or the depth data using the deep learning model.
17. The apparatus according to claim 13 , wherein the region segmentation unit segments the region for each image which configures the stereo image to generate the region data and the depth estimating unit generates the depth data for each segmented region.
18. The apparatus according to claim 13 , wherein the stereo image is obtained by capturing the tissue or the cell.
19. The apparatus according to claim 15 , wherein the deep learning model is implemented to simulate blind deconvolution using a point-spread function.
20. The apparatus according to claim 13 , further comprising:
an image preprocessing unit which preprocesses the stereo image to input a preprocessed image to the learning model.