Unsupervised writer style adaptation for handwritten word spotting
View Patent ↗A word spotting system includes a semi-continuous hidden Markov model configured to model a handwritten word of interest. A writing segments extractor is configured to extract writing segments generally comprising images of handwritten character strings from a received document image. A word model adaptation processor is configured to adjust a shared pool of Gaussians of the semi-continuous hidden Markov model respective to the extracted writing segments. A modeler is configured to model extracted writing segments using the semi-continuous hidden Markov model with the adjusted shared pool of Gaussians to identify whether each modeled writing segment matches the handwritten word of interest.
1. A word spotting method comprising:
defining a word model having first parameters and second parameters;
extracting writing segments generally comprising images of handwritten character strings from a received document image;
adjusting the first parameters of the word model respective to the extracted writing segments without adjusting the second parameters of the word model; and
performing word spotting on the received document image using the word model with the adjusted first parameters and the unadjusted second parameters to identify whether a handwritten word of interest is present in the received document image;
wherein at least the extracting, the adjusting, and the performing of word spotting are performed by a computer.
2. The word spotting method as set forth in claim 1 , wherein the word model is a semi-continuous hidden Markov model, the first parameters are parameters of a shared pool of Gaussians of the semi-continuous hidden Markov model, and the second parameters are mixture weights of the semi-continuous hidden Markov model.
3. The word spotting method as set forth in claim 2 , wherein the first parameters are mean and covariance parameters of the shared pool of Gaussians of the semi-continuous hidden Markov model.
4. The word spotting method as set forth in claim 2 , wherein the performing is repeated for a plurality of different handwritten words of interest to identify whether each handwritten word of interest is present in the received document image.
5. The word spotting method as set o h in claim 2 , wherein the word spotting comprises:
modeling extracted writing segments using the semi-continuous hidden Markov model with the adjusted first parameters and the unadjusted second parameters to identify whether each modeled writing segment matches the handwritten word of interest, wherein the handwritten word of interest comprises a string of a plurality of characters.
6. The word spotting method as set forth in claim 5 , further comprising:
computing a sequence of features vectors for each modeled writing segment, the modeling being respective to the computed sequences of features vectors;
wherein the computing is performed by the computer.
7. The word spotting method as set forth in claim 1 , further comprising:
printing or displaying the received document image conditional upon the word spotting indicating that the handwritten word of interest is present in the received document image.
8. The word spotting method as set forth in claim 1 , wherein the word spotting comprises:
modeling extracted writing segments using the word model with the adjusted first parameters and the unadjusted second parameters to identify whether each modeled writing segment matches the handwritten word of interest, wherein the handwritten word of interest comprises a string of a plurality of characters.
9. The word spotting method as set forth in claim 8 , wherein the first parameters principally characterize word parts and the second parameters principally characterize combinations of word parts.
10. The word spotting method as set forth in claim 8 , further comprising:
computing a sequence of features vectors for each modeled writing segment, the modeling being respective to the computed sequences of features vectors;
wherein the computing is performed by the computer.
11. The word spotting method as set forth in claim 1 , further comprising:
performing a fast-rejection classification of the extracted writing segments prior to performing word spotting.
12. A word spotting system comprising:
a semi-continuous hidden Markov model configured to model a handwritten word of interest;
a writing segments extractor configured to extract writing segments generally comprising images of handwritten character strings from a received document image;
a word model adaptation processor configured to adjust a shared pool of Gaussians of the semi-continuous hidden Markov model respective to the extracted writing segments; and
a modeler configured to model extracted writing segments using the semi-continuous hidden Markov model with the adjusted shared pool of Gaussians to identify whether each modeled writing segment matches the handwritten word of interest.
13. The word spotting system as set forth in claim 12 , wherein the word model adaptation processor does not adjust mixture weight parameters of the semi-continuous hidden Markov model.
14. The word spotting system as set forth in claim 12 , wherein the word model adaptation processor is configured to adjust mean and covariance parameters of the shared pool of Gaussians of the semi-continuous hidden Markov model respective to the extracted writing segments.
15. The word spotting system as set forth in claim 12 , further comprising:
a user interface configured to display an indication of whether the received document image includes the handwritten word of interest based on the output of the modeler.
16. The word spotting system as set forth in claim 12 , further comprising:
a moving window features vector generator configured to compute a sequence of features vectors for each modeled writing segment, the modeler being configured to model the sequence of features vectors using the semi-continuous hidden Markov model with the adjusted shared pool of Gaussians.
17. A non-transitory storage medium storing instructions readable and executable by a computer to perform a word spotting method including (i) extracting writing segments generally comprising images of handwritten character strings from a received document image (ii) adjusting some but not all parameters of a word model respective to the extracted writing segments and (iii) comparing the adjusted word model with extracted writing segments to identify whether a handwritten word of interest is present in the received document image.
18. The non-transitory storage medium as set forth in claim 17 , wherein parameters characterizing word parts are adjusted by the adjusting while parameters characterizing combinations of word parts are not adjusted by the adjusting.
19. The non-transitory storage medium as set forth in claim 18 , wherein the word model is a semi-continuous hidden Markov model, the parameters characterizing word parts are parameters of a shared pool of Gaussians of the semi-continuous hidden Markov model, and the parameters characterizing combinations of word parts are mixture weights of the semi-continuous hidden Markov model.
20. The non-transitory storage medium as set forth in claim 19 , wherein the word spotting method further includes computing a sequence of features vectors for each modeled writing segment, the comparing including comparing the computed sequences of features vectors with the adjusted word model.
21. The non-transitory storage medium as set forth in claim 19 , wherein the word spotting method further includes printing or displaying the received document image or an indication thereof conditional upon the word spotting indicating that the handwritten word of interest is present in the received document image.
22. The word spotting method as set forth in claim 1 , further comprising:
prior to performing the word spotting, performing normalization pre-processing of those extracted writing segments not rejected by the fast-rejection classification, said normalization pre-processing including normalizing at least one of slant, skew, text height, and aspect ratio, the normalization pre-processing being performed by the computer.