System and method for performing analysis on word variants
View Patent ↗A computer-readable medium stores a first lexicon data structure for lexicon words. The first data structure includes a host form variant field containing a host form variant such as a clitic host form variant, a host form field containing the host form of the host form variant (only present if the forms differ) such as a clitic host verbal form, and a verification field indicative of whether the host form variant is a valid word. The first data structure also includes a segment association field containing data or segmentation bits associating the host form variant with certain types of attachment entries in the lexicon, which also contain data or segmentation bits, to define valid combinations between the host form variant and at least one of the attachment entries in the lexicon. A second lexicon data structure for each of the attachment entries in the lexicon is also stored.
1. A method of annotating verb-clitic form segments in a lexicon, the method comprising:
defining, for a segment which is a clitic pair form segment, final segment data indicative of whether the segment must appear in a final position of any verb-clitic words formed using the segment;
annotating the segment using the defined final segment data;
defining, for the segment, segment association data indicative of valid combinations of the segment with other types of segments to form verb-clitic words;
annotating the segment using the segment association data;
defining, for the segment, word break data indicative of a word break location in the clitic pair form segment; and
annotating the segment using the word break data.
2. The method of claim 1 , wherein the step of defining segment association data further comprises defining the segment association data such that it is indicative of classes of clitic hosts variants to which the clitic pair form segment can appropriately be attached.
3. A method of combining first and second verb-clitic form segments from a lexicon to form a verb-clitic word, the method comprising:
determining whether absence of final segment data associated with the first verb-clitic form segment indicates that the first verb-clitic form segment cannot be a final segment of the verb-clitic word;
determining whether final segment data associated with the second verb-clitic form segment indicates that the second verb-clitic form segment must be the final segment of the verb-clitic word; and
combining the first and second verb-clitic form segments from the lexicon to form the verb-clitic word only if it is determined that the first verb-clitic form segment cannot be the final segment and that the second verb-clitic form segment must be the final segment.
4. The method of claim 3 , and further comprising:
determining from clitic class data associated with the respective first and second verb-clitic form segments whether the first and second verb-clitic form segments share at least one of a plurality of different clitic class associations; and
combining the first and second verb-clitic form segments from the lexicon to form the verb-clitic word only if it is determined that the first and second verb-clitic form segments share at least one of the plurality of different clitic class associations.
5. A tangible computer storage medium having stored thereon computer readable instructions for performing steps of a method of annotating verb-clitic form segments in a lexicon, comprising:
defining, for a segment which is a clitic host variant, final segment data indicative of whether the segment must appear in a final position of any verb-clitic words formed using the segment;
annotating the segment using the defined final segment data;
defining, for the segment, segment association data indicative of valid combinations of the segment with other types of segments to form verb-clitic words;
annotating the segment using the segment association data;
defining, for the segment, verification data indicative of whether the clitic host variant must be combined with a clitic segment to form a valid word; and
annotating the segment using the verification data.
6. The tangible computer storage medium of claim 5 , and wherein the step of defining segment association data further comprises defining the segment association data such that it is indicative of classes of clitic hosts variants to which the clitic form segment can appropriately be attached.