IP Library › Granted Patent US 7,236,980
Granted Patent B2
US 7,236,980 · App. 10/375,004 · Granted Jun 26, 2007

Unicode normalization

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,236,980
App. No.
10/375,004
Granted
Jun 26, 2007
Kind
B2
Abstract

A system and process for achieving faster Unicode normalizations is described. Various approaches are highlighted that when used alone or in combination eliminate a number of time consuming routines of conventional normalization approaches. Some of the approaches include late normalization, avoiding second table lookups, compressed classes and use of flags, assigning classes to base characters, complicated characters, and flags.

Claims (82)

1. A computer-implemented process for normalizing Unicode comprising the following steps:

examining characters represented in a Unicode string;

determining if each examined character is in a quick lookup table;

normalizing each examined character with a previous character for each examined character not in the quick lookup table;

for each examined character in the quick lookup table, skip examining the next character represented in the Unicode string; and

for each examined character normalized with a previous character, outputting the normalized form to the computer output.

2. A computer-implemented process for normalizing Unicode comprising the following steps:

for each character in a Unicode string;

examining the character;

skipping the examined character if the examined character is a low Unicode character;

otherwise, looking up the examined character in a table and skipping the examined character if possible;

otherwise, skipping the examined character if the examined character is one of special case characters or if the examined character is a base character;

if the examined character has not been skipped, performing a full normalization on the examined character; and

for each examined character that has undergone full normalization, outputting the normalized form to the computer output.

3. The computer-implemented process according to claim 2 , wherein the step of performing a full normalization on the examined character further includes the steps of:

looking up the special properties of a previous base character;

ordering all characters between the examined character and the base character;

skipping normalization if any of the characters between the examined character and the base character are non-combinable characters;

if none of the characters between the examined character and the base character are non-combinable characters, attempting to combine the examined character with the base character.

4. The computer-implemented process according to claim 3 , wherein the step of performing a full normalization on the examined character further includes the steps of:

performing special case normalization for the examined character and the base character; and

remembering character information for potential future processing.

5. The computer-implemented process according to claim 3 , wherein the step of performing a full normalization on the examined character further includes the step of:

remembering character information for potential future processing.

6. A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode when executed by a processor, said instructions comprising the following steps:

examining characters represented in Unicode string;

determining if each examined character is in a quick lookup table;

normalizing each examined character with a previous character for each examined character not in the quick lookup table;

for each examined character in the quick lookup table, skip examining the next character represented in the Unicode string; and

for each examined character normalized with a previous character, outputting the normalized form to the computer output.

7. A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode when executed by a processor, said instructions comprising the following steps:

for each character in a Unicode string;

examining the examined character;

skipping the examined character if the examined character is a low Unicode character;

otherwise, looking up the examined character in a table and skipping the examined character if possible;

otherwise, skipping the examined character if the examined character is one of special case characters or if the examined character is a base character;

if the examined character has not been skipped, performing a full normalization on the examined character; and

for each examined character that has undergone full normalization, outputting the normalized form to the computer output.

8. The computer readable medium according to claim 7 , wherein the step of performing a full normalization on the examined character further includes the steps of:

looking up the special properties of a previous base character;

ordering all characters between the examined character and the base character;

skipping normalization if any of the characters between the examined character and the base character are non-combinable characters;

if none of the characters between the examined character and the base character are non-combinable characters, attempting to combine the examined character with the base character.

9. The computer readable medium according to claim 8 , wherein the step of performing a full normalization on the examined character further includes the steps of:

performing special case normalization for the examined character and the base character; and

remembering character information for potential future processing.

10. The computer readable medium according to claim 8 , wherein the step of performing a full normalization on the examined character further includes the step of:

remembering character information for potential future processing.

11. A computer system for normalizing Unicode, said system comprising:

an input for receiving characters;

a processor for normalizing the characters, said processor performing the steps of:

examining characters represented in a Unicode string;

determining if each examined character is in a quick lookup table;

normalizing each examined character with a previous character for each examined character not in the quick lookup table; and

for each examined character in the quick lookup table, skip examining the next character represented in the Unicode string;

an output for outputting a normalized form of the received characters.

12. A computer system for normalizing Unicode, said system comprising:

an input for receiving characters;

a processor for normalizing the characters, said processor performing the steps of:

for each character in a Unicode string;

examining the character;

skipping the examined character if the examined character is a low Unicode character;

otherwise, looking up the examined character in a table and skipping the examined character if possible;

otherwise, skipping the examined character if the examined character is one of special case characters or if the examined character is a base character;

if the examined character has not been skipped, performing a full normalization on the examined character;

an output for outputting a normalized form of said received characters.

13. The computer system according to claim 12 , wherein the step of performing a full normalization on the examined character performed by the processor further includes the steps of:

looking up the special properties of a previous base character;

ordering all characters between the examined character and the base character;

skipping normalization if any of the characters between the examined character and the base character are non-combinable characters;

if none of the characters between the examined character and the base character are non-combinable characters, attempting to combine the examined character with the base character.

14. The computer system according to claim 13 wherein the step of performing a flail normalization on the examined character performed by the processor further includes the steps of:

performing special case normalization for the examined character and the base character; and

remembering character information for potential future processing.

15. The computer system according to claim 13 , wherein the step of performing a full normalization on the examined character performed by the processor further includes the step of:

remembering character information for potential future processing.

16. A computer system for normalizing Unicode comprising the following steps:

means for examining characters represented in a Unicode string;

means for determining if each examined character is in a quick lookup table;

means for normalizing each examined character with a previous character when the examined character is not in the quick lookup table;

means for skipping examining the next character represented in the Unicode string, when the examined character is in the quick lookup table; and

means for outputting to the computer output the normalized form of an examined character with a previous character, when the examined character is not in the quick lookup table.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034541/0477 →