Methods and apparatuses, and computing devices for segmenting object

1. A method for segmenting an object, comprising:

selecting, for an image to be processed, a plurality of local candidate regions according to two or more different preset scales respectively;

performing image segmentation processing on two or more local candidate regions, to predict and obtain binary segmentation masks of the two or more local candidate regions;

performing image classification processing on the two or more local candidate regions, to predict and obtain object classes to which the two or more local candidate regions belong respectively; and

fusing the two or more local candidate regions according to an overlapped area of the binary segmentation masks of the two or more the local candidate regions and the object classes to which the two or more local candidate regions belong, to obtain an object segmentation image.

2. The method according to claim 1 , further comprising:

before the selecting, for an image to be processed, a plurality of local candidate regions according to two or more different preset scales respectively,

performing convolution layer and/or pooling layer processing by a convolutional neural network, to obtain an intermediate result of the convolutional neural network;

wherein the selecting a plurality of local candidate regions according to two or more different preset scales respectively further comprises: selecting a local candidate region generation layer by using the intermediate result of the convolutional neural network, and selecting, according to the two or more different preset scales respectively, the plurality of local candidate regions on a feature map corresponding to the local candidate region generation layer through a sliding window.

3. The method according to claim 2 , further comprising:

after selecting the plurality of local candidate regions,

performing deconvolution layer and/or unpooling layer processing to unify the local candidate regions of different sizes to a fixed size.

4. The method according to claim 1 , wherein the performing image classification processing on two or more local candidate regions, to predict and obtain object classes to which the local candidate regions belong further comprises:

in case that a center of a local candidate region is located within a calibrated object, a proportion of an area of the calibrated object in the local candidate region to an area of the calibrated object is greater than a first threshold, and a proportion of the area of the calibrated object in the local candidate region to an area of the local candidate region is greater than a second threshold, determining, that an object class to which the local candidate region belongs is a class of the calibrated object.

5. The method according to claim 1 , further comprising:

before fusing the two or more local candidate regions,

establishing a loss training function to train a loss of image classification and image segmentation.

6. The method according to claim 1 , wherein fusing the two or more local candidate regions further comprises:

determining an overlapped area of binary segmentation masks of two adjacent local candidate regions; and

in response to the overlapped area being greater than a preset threshold, the two adjacent local candidate regions belonging to a same object class, and neither of the two adjacent local candidate regions being assigned as an object, generating a new object and assigning the two adjacent local candidate regions as the new object.

7. The method according to claim 1 , wherein fusing the two or more local candidate regions further comprises:

determining an overlapped area of binary segmentation masks of two adjacent local candidate regions; and

in response to the overlapped area being greater than a preset threshold, the two adjacent local candidate regions belonging to a same object class, and one of the two adjacent local candidate regions being assigned as an object, merging the two adjacent local candidate regions and assigning the other local candidate region as the object.

8. The method according to claim 1 , wherein fusing the two or more local candidate regions further comprises:

determining an overlapped area of binary segmentation masks of two adjacent local candidate regions; and

in response to the overlapped area being greater than a preset threshold, the two adjacent local candidate regions belonging to a same object class, and the two adjacent local candidate regions being assigned as two objects, merging the two objects.

9. An apparatus for segmenting an object, comprising:

a processor; and

a memory for storing instructions executable by the processor;

wherein the processor is configured to:

select, for an image to be processed, a plurality of local candidate regions according to two or more different preset scales respectively;

perform image segmentation processing on two or more local candidate regions, to predict and obtain binary segmentation masks of the two or more local candidate regions;

perform image classification processing on the two or more local candidate regions, to predict and obtain object classes to which the two or more local candidate regions belong respectively; and

fuse the two or more local candidate regions according to an overlapped area of the binary segmentation masks of the two or more the local candidate regions and the object classes to which the two or more local candidate regions belong, to obtain an object segmentation image.

10. The apparatus according to claim 9 , the processor is further configured to perform convolution layer and/or pooling layer processing by a convolutional neural network, to obtain an intermediate result of the convolutional neural network; and

the processor is further configured to: select a local candidate region generation layer by using the intermediate result of the convolutional neural network, and select, according to the two or more different preset scales respectively, the plurality of local candidate regions on a feature map corresponding to the local candidate region generation layer according to the two or more different preset scales through a sliding window.

11. The apparatus according to claim 10 , wherein the processor is further configured to: perform deconvolution layer and/or unpooling layer processing to unify the local candidate regions in different sizes to a fixed size.

12. The apparatus according to claim 9 , wherein the processor is further configured to: determine that an object class to which the local candidate region belongs is a class of the calibrated object, in case that a center of a local candidate region is located within a calibrated object, a proportion of an area of the calibrated object in the local candidate region to an area of the calibrated object is greater than a first threshold, and a proportion of the area of the calibrated object in the local candidate region to an area of the local candidate region is greater than a second threshold.

13. The apparatus according to claim 9 , the processor is further configured to establish a loss training function to train a loss of image classification and image segmentation.

14. The apparatus according to claim 9 , wherein the processor is further configured to:

determine an overlapped area of binary segmentation masks of two adjacent local candidate regions; and

in response to the overlapped area being greater than the preset threshold, the two adjacent local candidate regions belonging to a same object class, and neither of the two adjacent local candidate regions being assigned as an object, generate a new object and assign the two adjacent local candidate regions as the new object.

15. The apparatus according to claim 9 , wherein the processor is further configured to:

determine an overlapped area of binary segmentation masks of two adjacent local candidate regions; and

in response to the overlapped area being greater than the preset threshold, the two adjacent local candidate regions belonging to the same object class, and one of the two adjacent local candidate regions being evaluated as an object, merge the two adjacent local candidate regions and evaluate the other local candidate region as the object,

in response to the overlapped area being greater than a preset threshold, the two adjacent local candidate regions belonging to a same object class, and one of the two adjacent local candidate regions being assigned as an object, merge the two adjacent local candidate regions and assign the other local candidate region as the object.

16. The apparatus according to claim 9 , wherein the processor is further configured to:

determine an overlapped area of binary segmentation masks of two adjacent local candidate regions; and

in response to the overlapped area being greater than a preset threshold, the two adjacent local candidate regions belonging to a same object class, and the two adjacent local candidate regions being assigned as two objects, merge the two objects.

17. A non-transitory computer-readable medium having stored therein instructions that, when executed by a processor, causes to the processor to:

select, for an image to be processed, a plurality of local candidate regions according to two or more different preset scales respectively;

perform image segmentation processing on two or more local candidate regions, to predict and obtain binary segmentation masks of the two or more local candidate regions;

perform image classification processing on the two or more local candidate regions, to predict and obtain object classes to which the two or more local candidate regions belong respectively; and

fuse the two or more local candidate regions according to an overlapped area of the binary segmentation masks of the two or more the local candidate regions and the object classes to which the two or more local candidate regions belong, to obtain an object segmentation image.

18. A program product comprising non-transitory computer readable instructions that cause a processor to execute the method of claim 1 .