Digital ink database searching using handwriting feature synthesis
A system and method for ink database searching using handwriting feature synthesis is disclosed which allows a digital ink database to be searched using a text-based query. Using a writer-specific handwriting model derived from a handwriting recognition system or suitable training procedure, a text query is converted into feature vectors that are similar to the feature vectors that would have been extracted had the author of the digital ink database written the text query by hand. The feature vectors are then used to search the database.
1. A method of searching a digital ink database, the digital ink database being associated with a predetermined author, the method including the steps of:
receiving an ASCII text query entered through a computer keyboard;
mapping the ASCII text query to a set of feature vectors using a handwriting model obtained from the author associated with the digital ink database using handwriting recognition results, the mapping step comprising the sub-steps of;
performing a search of a lexicon consisting of a table of letter sequences mapped to feature vectors, and determining a sequence of lexicon entries which contain letter sequences that, when combined, produce the text query, the lexicon being part of the handwriting modal, and
obtaining a set of feature vectors corresponding to the sequence of lexicon entries;
using the set of feature vectors to search the digital ink database; and
displaying the results of the search to the user,
wherein the handwriting recognition results are obtained by the steps of:
sampling handwriting;
smoothing the handwriting using a filter;
performing slant correction;
using a zone estimation algorithm to perform height normalization;
using feature extraction to perform segmentation into sub-strokes and generation of feature vectors;
using feature reduction of the set of feature vectors;
performing vector quantization to cluster the feature vectors and generate code word vectors; and
searching a dictionary for the most likely word from the code word vectors.
2. The method as claimed, in claim 1 , wherein the lexicon includes multiple feature vectors for each letter sequence.
3. The method as claimed in claim 1 , wherein the lexicon includes an end of word letter sequences and associated feature vectors.
4. The method as claimed in claim 1 , wherein if more than one sequence of lexicon entries is determined contain letter sequences that can be combined to produce the text query, then the sequence of lexicon entries having the least number of entries is used.
5. Apparatus for searching a digital ink database, the digital ink database being associated with a predetermined author, the apparatus including:
a keyboard for receiving an ASCII text query;
means for mapping the ASCII text query to a set of feature vectors using a handwriting model obtained from the author associated with the digital ink database using handwriting recognition results, the mapping is performed by performing a search of a lexicon consisting of a table of letter sequences mapped to feature vectors, determining a sequence of lexicon entries which contain letter sequences that, when combined, produce the text query, the lexicon being part of the handwriting model, and obtaining a set of feature vectors corresponding to the sequence of lexicon entries; and
means for searching the digital ink database using the set of feature vectors; and
an output device to display the results of the search to the user,
wherein the handwriting recognition results are obtained by the steps of:
sampling handwriting;
smoothing the handwriting using a filter;
performing slant correction;
using a zone estimation algorithm to perform height normalization;
using feature extraction to perform segmentation into sub-strokes and generation of feature vectors;
using feature reduction of the set of feature vectors;
performing vector quantization to cluster the feature vectors and generate code word vectors; and
searching a dictionary for the most likely word from the code word vectors.