Identifying image abnormalities using an appearance model
View Patent ↗The identification of known normal structures within an image is preferably accomplished using an appearance model. Specifically, an active appearance model, which encapsulates a complete model of the shape and global texture variations of an object from a collection of samples, is utilized to define normal structures within an image by restricting training samples supplied to the active appearance model during a training phase to those that do not contain abnormal structures. Accordingly, the trained appearance model represents only normal variations in the object of interest. When another image with abnormalities is presented to the system, the appearance model cannot synthesize the abnormal structures which show up as errors in a residual image. Accordingly, the errors in the residual image represent potential abnormalities.
1. A method of detecting abnormalities in an input image of an object, the method comprising:
receiving the input image of the object at a processing system;
receiving, at the processing system, a sample normal image of a normal object formed using an appearance model, wherein the appearance model is synthesized from a training set of normal images that depict normal objects containing no abnormalities, and wherein the appearance model is synthesized using a texture model defining a texture distribution for the normal objects and a shape model defining a shape distribution for the normal objects;
determining, by the processing system, at least one difference between the input image and the sample normal image;
modifying, by the processing system, the sample normal image based at least in part on the at least one difference between the input image and the sample normal image;
ceasing modification of the sample normal image based on a stopping criterion being met, wherein the stopping criteria is calculated based on a threshold decrease in the at least one difference between the input image and the sample normal image between consecutive iterations of the determining the at least one difference and the modifying the sample normal image; and
identifying, by the processing system, an abnormality in the input image, wherein the abnormality is indicated by an area of the input image that does not conform to a corresponding area of the sample normal image.
2. The method as claimed in claim 1 , wherein the appearance model is generated based on a shape model applied to the training set of normal images.
3. The method as claimed in claim 1 , wherein the appearance model is generated based on a texture model applied to the training set of normal images.
4. The method as claimed in claim 1 , wherein the appearance model is generated based on a shape model and a texture model applied to the training set of normal images.
5. The method as claimed in claim 1 , wherein the appearance model is an active appearance model.
6. The method as claimed in claim 1 , wherein the normal objects are the same class as the object in the input image.
7. The method as claimed in claim 1 , further comprising extracting a plurality of texture tuples from the training set of normal images, wherein the extracting the plurality of texture tuples comprises normalizing shape contours of the objects in the training set of normal images to a mean shape contour of all objects in the training set of normal images, and wherein the plurality of texture tuples all have a same shape for the object.
8. The method as claimed in claim 1 , wherein synthesis of the appearance model comprises applying principal component analysis jointly to both the texture model and the shape model.
9. The method as claimed in claim 1 , further comprising modifying the sample normal images until the threshold amount of error has been satisfied.
10. An apparatus comprising:
a memory configured to store a sample normal image of a normal object formed using an appearance model, wherein the appearance model is synthesized from a training set of normal images that depict normal objects containing no abnormalities, and wherein the appearance model is synthesized using a texture model defining a texture distribution for the normal objects and a shape model defining a shape distribution for the normal Objects; and
an image abnormality processing unit configured to:
determine at least one difference between an input image and the sample normal image;
modify the sample normal image based at least in part on the at least one difference between the input image and the sample normal image;
cease modification of the sample normal image based on a stopping criterion being met, wherein the stopping criteria is calculated based on a threshold decrease in the at least one difference between the input image and the sample normal image between consecutive iterations of the determining the at least one difference and the modifying the sample normal image; and
identify an abnormality in the input image, wherein the abnormality is indicated by an area of the input image that does not conform to a corresponding area of the sample normal image.
11. The apparatus as claimed in claim 10 , wherein the image abnormality processing unit is further configured to generate the appearance model based on a shape model applied to the training set of normal images.
12. The apparatus as claimed in claim 10 , wherein the image abnormality processing unit is further configured to generate the appearance model based on a texture model applied to the training set of normal images.
13. The apparatus as claimed in claim 10 , wherein the image abnormality processing unit is further configured to generate the appearance model based on a shape model and a texture model applied to the training set of normal images.
14. The apparatus as claimed in claim 10 , wherein the appearance model is an active appearance model.
15. The apparatus as claimed in claim 10 , further comprising a data entry device configured to receive the input image of the object.
16. A non-transitory computer-readable medium having instructions stored thereon that, upon execution by a computing device, cause the computing device to perform operations comprising:
receiving the input image of the object;
receiving a sample normal image of a normal object formed using an appearance model, wherein the appearance model is synthesized from a training set of normal images that depict normal objects containing no abnormalities, and wherein the appearance model is synthesized using a texture model defining a texture distribution for the normal objects and a shape model defining a shape distribution for the normal objects;
determining at least one difference between the input image and the sample normal image;
modifying the sample normal image based at least in part on the at least one difference between the input image and the sample normal image;
ceasing modification of the sample normal image based on a stopping criterion being met, wherein the stopping criteria is calculated based on a threshold decrease in the at least one difference between the input image and the sample normal image between consecutive iterations of the determining the at least one difference and the modifying the sample normal image; and
identifying an abnormality in the input image, wherein the abnormality is indicated by an area of the input image that does not conform to a corresponding area of the sample normal image.
17. The non-transitory computer-readable medium as claimed in claim 16 , wherein synthesis of the appearance model comprises applying principal component analysis jointly to both the texture model and the shape model.