Graphical object classification

1. A graphical object classification system, comprising:

a processor; and

a non-transitory processor-readable medium comprising:

an acquisition module to access a two dimensional bitmap representation of a graphical object generated from a parsed screenshot of a graphical user interface (GUI);

a signature generation module to generate an appearance-invariant signature of the graphical object based on the bitmap representation of the graphical object, wherein the appearance-invariant signature comprises a composite of two or more signature vectors that vary little in response to changes in shape, size, orientation, and visual layer appearance of the graphical object, and each of the signature vectors includes attributes based on operations on the bitmap representation of the graphical object comprising integration along a first dimension, differentiation of the integration along a different second dimension, and domain transformation of the differentiation into at least one of Fourier magnitude components and log-domain transforms; and

a classification module to classify the graphical object based on the appearance-invariant signatures, wherein the graphical object is classified into a class and the graphical object after a change in appearance is classified into the same class when the respective appearance-invariant signatures before and after the change share substantially common attributes.

2. The system of claim 1 , wherein the signature generation module is operable to:

generate a first signature vector comprising a first collection of attributes from the bitmap representation of the graphical object;

generate a second signature vector comprising a second collection of attributes from the bitmap representation of the graphical object; and

composite the first signature vector and the second signature vector into multiple portions.

3. The system of claim 1 , wherein the signature generation module is operable to:

generate a first signature vector comprising a first collection of attributes from the bitmap representation of the graphical object based on a first integral along a first dimension of the bitmap representation of the graphical object, a first derivative of the first integral along a second dimension of the bitmap representation of the graphical object different from the first dimension, and a first domain transform of the first derivative;

generate a second signature vector comprising a second collection of attributes from the representation of the graphical object based on a second integral along the second dimension, a second derivative of the second integral along the first dimension, and a second domain transform of the second derivative; and

composite the first signature vector and the second signature vector into multiple portions.

4. The system of claim 1 , wherein the signature generation module is operable to include gradient information generated from the bitmap representation of the graphical object, wherein the gradient information is a histogram of gradient angles of the graphical object calculated from the bitmap representation of the graphical object and the gradient information is an additional signature vector composited to the appearance-invariant signature.

5. The system of claim 1 , wherein color information related to the graphical object is generated from the bitmap representation of the graphical object and the color information is an additional signature vector composited to the appearance-invariant signature.

6. The system of claim 1 , wherein the classification module is a support vector machine (SVM) based classification module, wherein the SVM has been configured using a supervised training procedure in which groups of appearance-invariant signatures for each of a variety of classes were provided to the SVM to generate support vectors for use by the SVM to discriminate between appearance-invariant signatures of graphical objects from the variety of classes.

7. The system of claim 1 , wherein the classification module is operable to:

compare the bitmap representation of the graphical object with a template;

determine that the bitmap representation of the graphical object does not satisfy the template; and

provide the appearance-invariant signature to a support vector machine module in response to the determining.

8. The system of claim 1 , wherein the acquisition module applies an edge-detection filter to the GUI and selects portions of the GUI for which the edge-detection filter generates rectangular features.

9. A processor-readable medium storing code representing instructions to cause a processor to perform a process, the process comprising:

receiving a plurality of two dimensional bitmap graphical object representations of a parsed screenshot of a graphical user interface (GUI), each bitmap graphical object representation associated with a graphical object from a plurality of graphical objects, each graphical object associated with an object class from a plurality of object classes;

generating a plurality of appearance-invariant signatures, each appearance-invariant signature based on the bitmap graphical object representation of a graphical object from the plurality of graphical objects and associated with that graphical object, wherein the appearance-invariant signature comprises a composite of two or more signature vectors that vary little in response to changes in shape, size, orientation, and visual layer appearance of the graphical object, and each of the signature vectors includes attributes based on operations on the bitmap representation of the graphical object comprising integration along a first dimension, differentiation of the integration along a different second dimension, and domain transformation of the differentiation into at least one of Fourier magnitude components and log-domain transforms; and

classifying each graphical object into an object class from the plurality of object classes based on a composite of two or more of the appearance-invariant signature associated with that graphical object, wherein each graphical object is classified into the object class and the respective graphical object after a change in appearance is classified into the same object class when the respective appearance-invariant signatures before and after the change share substantially common attributes.

10. The processor-readable medium of claim 9 , wherein the generating a plurality of appearance-invariant signatures includes:

generating for each appearance-invariant signature a first signature vector of attributes from the bitmap graphical object representation of the graphical object associated with that appearance-invariant signature;

generating for each appearance-invariant signature a second signature vector of attributes from the bitmap graphical object representation of the graphical object associated with that appearance-invariant signature; and

compositing the first signature vector and the second signature vector for each appearance-invariant signature within that appearance-invariant signature into multiple portions.

11. The processor-readable medium of claim 9 , wherein each appearance-invariant signature includes a representation of a plurality of gradients generated from the bitmap graphical object representation of the graphical object associated with that appearance-invariant signature, wherein each of the plurality of gradients is a histogram of gradient angles of the graphical object calculated from the bitmap graphical object representation and the gradient information is an additional signature vector composited to the appearance-invariant signature.

12. The processor-readable medium of claim 9 , wherein the classifying includes providing the appearance-invariant signature associated with each graphical object to a support vector machine (SVM) module, wherein the SVM has been configured using a supervised training procedure in which groups of appearance-invariant signatures for each of a variety of classes were provided to the SVM to generate support vectors for use by the SVM to discriminate between appearance-invariant signatures of graphical objects from the variety of classes.

13. The processor-readable medium of claim 9 , further comprising:

generating color information related to the graphical object from the bitmap representation of the graphical object; and

compositing the color information to the appearance-invariant signature as an additional signature vector.

14. The processor-readable medium of claim 9 , wherein accessing the bitmap representation of a graphical object includes applying an edge-detection filter to the GUI and selecting portions of the GUI for which the edge-detection filter generates rectangular features.

15. A graphical object classification method, the process comprising:

accessing a two dimensional bitmap representation of a graphical object from a parsed screenshot of a graphical user interface (GUI);

generating a plurality of signature vectors from the representation, wherein each of the signature vectors comprises attributes that vary little in response to changes in shape, size, orientation, and visual layer appearance of the graphical object, and each of the signature vectors includes attributes based on operations on the bitmap representation of the graphical object from the group comprising integration along a first dimension, differentiation of the integration along a different second dimension, and domain transformation of the differentiation into at least one of Fourier magnitude components and log-domain transforms;

compositing each signature vector from the plurality of signature vectors to define an appearance-invariant signature of the graphical object; and

classifying each graphical object based on the appearance-invariant signature, wherein the graphical object is classified into a class and the graphical object after a change in appearance is classified into the same class when the respective appearance-invariant signatures before and after the change share substantially common attributes.

16. The method of claim 15 , further comprising:

including color information generated from the bitmap representation of the graphical object; and

compositing the color information as an additional signature vector to the appearance-invariant signature.

17. The method of claim 15 , wherein compositing each signature vector from the plurality of signature vectors to define the appearance-invariant signature of the graphical object comprises concatenating a first signature vector associated with a first axis of the graphical object, a second signature vector associated with a second axis of the graphical object, a third signature vector associated with a plurality of gradient information generated from the bitmap representation of the graphical object, and a fourth signature vector associated with a plurality of color information generated from the bitmap representation of the graphical object.

18. The method of claim 15 , wherein generating the plurality of signature vectors includes:

generating a first signature vector from a first integral of the bitmap representation in the first dimension, generating a first derivative of the first integral in the second dimension, and generating a first domain transform of the first derivative;

generating a second signature vector from a second integral of the bitmap representation in the second dimension, generating a second derivative of the second integral in the first dimension, and generating a second domain transform of the second derivative; and

compositing the first signature vector and the second signature vector for the appearance-invariant signature into multiple portions.

19. The method of claim 18 , wherein each dimension of the plurality of signature vectors is associated with an axis of the graphical object.

20. The method of claim 15 , further comprising:

including gradient information generated from the bitmap representation, wherein the gradient information is a histogram of gradient angles of the graphical object calculated from the bitmap representation of the graphical object and the gradient information is an additional signature vector composited to the appearance-invariant signature.