Image mask generation using a deep neural network

1. An electronic device, comprising:

a memory configured to store an input image that comprises an object of interest; and

circuitry configured to:

generate a pilot mask for the input image based on application of a trained deep neural network (DNN) on the input image;

extract, from the input image, a region of interest that includes the object of interest, wherein the region of interest is extracted based on the generated pilot mask for the input image;

resize the extracted region of interest such that a size of the resized region of interest equals a threshold scaling size; and

generate a final mask for the input image the application of the trained DNN on the resized region of interest.

2. The electronic device according to claim 1 , wherein

the circuitry is further configured to down-sample an image that comprises the object of interest, and

the input image is the down-sampled image.

3. The electronic device according to claim 1 , wherein the object of interest is a human object.

4. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect the object of interest in the input image based on the generated pilot mask; and

extract the region of interest from the input image based on the detection of the object of interest.

5. The electronic device according to claim 1 , wherein

the input image comprises a plurality of objects of interest, and

the circuitry is further configured to detect each object of interest of the plurality of objects of interest based on the generated pilot mask.

6. The electronic device according to claim 1 , wherein a size of the extracted region of interest is proportional to one of a width of the object of interest or a height of the object of interest.

7. The electronic device according to claim 1 , wherein

a width of the extracted region of interest is proportional to a width of the object of interest, and

a height of the extracted region of interest is proportional to a height of the object of interest.

8. The electronic device according to claim 1 , wherein

the circuitry is further configured to generate a plurality of test images based on the resize of the extracted region of interest in accordance with a plurality of scaling sizes, and

each test image of the plurality of test images comprises the extracted region of interest.

9. The electronic device according to claim 8 , wherein

the circuitry is further configured to generate a plurality of test masks of the object of interest based on the application of the trained DNN on the generated plurality of test images, and

each test mask of the plurality of test masks corresponds to a different scaling size of the plurality of scaling sizes.

10. The electronic device according to claim 9 , wherein the circuitry is further configured to:

resize the generated of plurality of test masks; and

stitch the resized plurality of test masks on a corresponding template frame of a plurality of template frames.

11. The electronic device according to claim 10 , wherein

the circuitry is further configured to determine a similarity score between each stitched test mask of the stitched plurality of test masks and a reference mask for the input image, and

the reference mask is a ground truth mask for the input image.

12. The electronic device according to claim 11 , wherein the circuitry is further configured to select a scaling size from the plurality of scaling sizes as the threshold scaling size based on a maximum similarity score among the determined similarity scores for the stitched plurality of test masks.

13. The electronic device according to claim 11 , wherein

the circuitry is further configured to compute an average intersection-over-union (IOU) between each stitched test mask of the stitched plurality of test masks and the reference mask, and

the computed average IOU corresponds to the determined similarity score.

14. The electronic device according to claim 1 , wherein a size of the generated final mask is equal to a size of the resized region of interest.

15. The electronic device according to claim 1 , wherein the circuitry is further configured to resize the final mask such that a size of the resized final mask equals a size of the extracted region of interest.

16. The electronic device according to claim 15 , wherein

the circuitry is further configured to generate an output mask based on a stitch of the resized final mask on a template frame,

the template frame corresponds to a binary background frame for the output mask, and

a size of the output mask is equal to a size of the input image.

17. The electronic device according to claim 1 , wherein the circuitry is configured to:

select the input image from a sequence of input images of a video; and

generate, for each input image of the sequence of input images, the final mask of the object of interest at an output rate, wherein the final mask is generated such that the output rate is time synchronized with a frame rate of the video.

18. The electronic device according to claim 1 , wherein the circuitry is further configured to segment the object of interest from the input image based on the generated final mask.

19. A method, comprising:

in an electronic device:

generating a pilot mask of an input image based on application of a trained deep neural network (DNN) on the input image, wherein the input image comprises an object of interest;

extracting, from the input image, a region of interest that includes the object of interest, wherein the region of interest is extracted based on the generated pilot mask of the input image;

resizing the extracted region of interest such that a size of the resized region of interest equals a threshold scaling size; and

generating a final mask of the object of interest based on application of the trained DNN on the resized region of interest.

20. The method according to claim 19 , further comprising:

detecting the object of interest in the input image based on the generated pilot mask; and

extracting the region of interest from the input image based on the detection of the object of interest.