Method and system of image segmentation refinement for image processing

1. A computer-implemented method of image segmentation refinement for image processing, comprising:

obtaining pixel data of at least one image;

determining a border area cooperatively formed between at least two initially formed image area segments on the image; and

determining a location shift to apply to one or more points forming at least one contour in the border area and that separates the segments comprising using an iterative active contour-type algorithm having a plurality of weighted forces each of a different treatment that generates the force in a different way and applied to the points to modify the position of the points wherein one or more dynamic weights change while performing iterations to adjust a value of at least one weighted force of at least one individual iteration.

2. The method of claim 1 comprising modifying at least one dynamic weight over a plurality of the iterations to weaken or strengthen at least one weighted force over the plurality of iterations.

3. The method of claim 2 comprising changing a smoothness dynamic weight iteration to iteration to permit less bending of the contour over time.

4. The method of claim 1 comprising changing a balloon dynamic weight of individual iterations by changing the sign of the weight.

5. The method of claim 1 comprising determining at least one characteristic of contours that is a general arrangement of the contour at at least one border area on the image, and depending on the at least one characteristic, determining at least one of:

which one or more of the forces to apply to a contour by the active contour-type algorithm,

which way to apply at least one weighted force to a contour by the active contour-type algorithm, or

both.

6. The method of claim 5 wherein the characteristics include a position on the image between desired segments, a border area between areas of the image that should be merged into the same segment, and a border of a hole area substantially entirely enclosed within a segment and that has incorrect image data.

7. The method of claim 6 wherein a diffusion force is applied or not applied to the contour depending on which characteristic the contour has.

8. The method of claim 6 wherein determining which way to apply at least one force comprises determining whether to apply a weighted ballooning force to shrink or grow an object on the image.

9. The method of claim 5 wherein determining which way to apply at least one force comprises determining which initial segments of the image are used to form a Gaussian mixture model and depending on the characteristic of the contour.

10. The method of claim 5 comprising re-sampling points to re-define contours at multiple individual iterations, wherein the points indicated by the results of the re-sampling are used to form the weighted forces of the individual iterations.

11. The method of claim 1 wherein one of the forces is a region force that is partially based on at least one Gaussian mixture model (GMM) to determine segmentation assignment of the point(s).

12. The method of claim 11 comprising forming at least one GMM based on multiple clusters, each cluster representing a characteristic of pixel data of pixels in a segment.

13. The method of claim 11 comprising determining a GMM for each segment cooperatively forming a border area.

14. The method of claim 13 comprising using the GMMs to each provide a probability that a point should be within a segment of the GMM by comparing a color of the point to colors of the segment.

15. The method of claim 14 wherein the color of the point is interpolated from color values of a group of pixels near the point.

16. The method of claim 14 comprising using the multiple probabilities to adjust an initial region weight value to form the region force.

17. The method of claim 16 comprising using a difference between two probabilities of the GMMs to adjust the initial region weight.

18. The method of claim 1 comprising validating that a point should be moved to a new location indicated by the location shift comprising determining whether the new location is in the border area.

19. The method of claim 18 comprising shifting the point to the new location indicated by the iterations when the new location is within the border area.

20. The method of claim 12 comprising at least one of:

wherein each cluster is based on a different color or luminance in the segment; and

wherein each cluster is based on the most used colors in the segment.

21. A computer-implemented system of image segmentation refinement for image processing, comprising:

at least one display;

at least one memory

at least one processor communicatively coupled to the display and the memory; and

an image segmentation unit operated by the at least one processor and to:

obtain pixel data of at least one image;

determine a border area cooperatively formed between at least two initially formed image area segments on the image; and

determine a location shift to apply to one or more points forming at least one contour in the border area and that separates the segments comprising using an iterative active contour-type algorithm having a plurality of weighted forces each of a different treatment that generates the force in a different way and applied to the points to modify the position of the points wherein one or more dynamic weights change while performing iterations to adjust a value of at least one weighted force of at least one individual iteration.

22. The system of claim 21 wherein the image segmentation unit is to:

modify at least one dynamic weight over a plurality of the iterations to weaken or strengthen at least one weighted force over the plurality of iterations;

change a smoothness dynamic weight iteration to iteration to permit less bending of the contour over time;

change a balloon dynamic weight of individual iterations by changing the sign of the weight;

determine at least one characteristic of contours at at least one border area on the image, and depending on the at least one characteristic, determining at least one of:

which one or more of the forces to apply to a contour by the active contour-type algorithm,

which way to apply at least one weighted force to a contour by the active contour-type algorithm, or

both;

wherein the characteristics include a position on the image between desired segments, a border area between areas of the image that should be merged into the same segment, and a border of a hole area substantially entirely enclosed within a segment and that has incorrect image data;

wherein a diffusion force is applied or not applied to the contour depending on which characteristic the contour has;

wherein determining which way to apply at least one force comprises determining whether to apply a weighted ballooning force to shrink or grow an object on the image;

wherein determining which way to apply at least one force comprises determining which initial segments of the image are used to form a Gaussian mixture model and depending on the characteristic of the contour;

re-sample points to re-define contours at multiple individual iterations, wherein the points indicated by the results of the re-sampling are used to form the weighted forces of the individual iterations;

wherein one of the forces is a region force that is partially based on at least one Gaussian mixture model (GMM) to determine segmentation assignment of the point(s).

23. A non-transitory computer-readable medium having stored thereon instructions that when executed cause a computing device to:

obtain pixel data of at least one image;

determine a border area cooperatively formed between at least two initially formed image area segments on the image; and

determine a location shift to apply to one or more points forming at least one contour in the border area and that separates the segments comprising using an iterative active contour-type algorithm having a plurality of weighted forces each of a different treatment that generates the force in a different way and applied to the points to modify the position of the points wherein one or more dynamic weights change while performing iterations to adjust a value of at least one weighted force of at least one individual iteration.

24. The method of claim 1 wherein the treatments comprise at least two of:

an internal smoothing force that limits curvature of the contour comprising pushing points toward an average position between previous and succeeding points along the contour,

an external gradient force that bends the contour toward other content structure on the image comprising pushing points toward color gradients,

a region force that indicates whether a pixel should be at the border and uses a Gaussian mixture model,

a balloon force that expands and contracts holes, and

a diffusion force that reduces sensitivity of the contour to noise.

25. The method of claim 24 wherein at least four of the treatments are provided with each treatment having an adjustable one of the weights to adjust the strength of a force of a treatment relative to the strength of the forces of the other treatments.