Detecting segmentation errors in an annotated corpus

1 . A computer-implemented method to obtain a segmentation error rate of an annotated corpus, the method comprising:

processing the annotated corpus with a computer to ascertain segmentation variations therein;

presenting segmentation variations to a language analyzer with the computer to identify segmentation errors in the segmentation variations; and

counting a number of segmentation errors and calculating a segmentation error rate for the corpus.

2 . The computer-implemented method of claim 1 wherein presenting segmentation variations includes presenting segmentation variations with some adjacent context.

3 . The computer-implemented method of claim 1 wherein calculating the segmentation error rate includes a calculation based on the number of errors counted and the number of segmentations in the corpus.

4 . A computer-implemented method for locating segmentation errors in an annotated corpus, the method comprising:

obtaining sets of segmentation variation instances of multi-character words from the corpus with a computer, each set comprising more than one segmentation variation instance of a word in the corpus;

rendering each segmentation variation instance to a language analyzer with the computer to identify if the segmentation variation instance is a segmentation error; and

receiving an indication if the segmentation variation instance is a segmentation error.

5 . The computer-implemented method of claim 1 wherein rendering segmentation variations includes presenting segmentation variations with some adjacent context.

6 . The computer-implemented method of claim 1 wherein obtaining sets of segmentation variation instances comprises compiling a list of the words for each set in a list.

7 . The computer-implemented method of claim 6 and further comprising compiling each of the segmentation variation instances in a list.

8 . The computer-implemented method of claim 7 and further comprising compiling each of the segmentation errors in a list.

9 . A system for locating segmentation errors in an annotated corpus, the system comprising:

an extracting module configured to extract segmentation variations from the corpus and compile a list of segmentation variations instances for each of the segmentation variations having two or more segmentation variations for a given word;

a rendering module configured to render each segmentation variation instance and receive an indication from an analyzer as to whether the segmentation variation instance is a segmentation error.

10 . The system of claim 9 wherein the rendering module is configured to render each segmentation variation instance with adjacent context.

11 . The system of claim 10 wherein the rendering module is configured to calculate a segmentation error rate for the corpus based on the segmentation errors identified.