System and methods for improving data compression
An improved data compression system and method is disclosed. The data compression method uses smaller fixed bit words to represent streams of digital data. The fixed bit words are sent instead of the streams of digital data. A table relating the fixed bit words and streams of digital data is provided as a dictionary. Each entry has a node which is indexed in a stored index. As additional fixed bit words are established for new streams of digital data, older fixed bit words are deleted from the dictionary. The older fixed bit words are prevented from being deleted if they are used for subsequent streams of data. There are a number of methods to determine how the older fixed bit words are retained which are disclosed.
1. A method of compressing digital data for transmission over a communications channel, the method comprising:
determining strings of digital data;
assigning a fixed size code word to a string of digital data;
storing the fixed size code word and corresponding string of digital data in a dictionary as a node;
storing subsequent fixed size code words as new nodes in the dictionary representing subsequent different strings of digital data ahead of the node storing the first fixed sized code word;
examining a subsequent part of digital data for the string of digital data; and
deleting the fixed size code word's node from the dictionary and adding the fixed size code word's node to the dictionary when a match is made between the string of digital data and the subsequent part of the digital data.
2. The method of claim 1 further comprising:
sending the fixed size code word in place of the string of digital data to a receiver; and
translating the fixed sized code word into the string of digital data;
wherein the receiver maintains a duplicate dictionary.
3. The method of claim 1 wherein the fixed sized code word node in the dictionary is deleted using a cyclic index.
4. The method of claim 1 further comprising storing a history index which records when the nodes are used to represent subsequent strings of data.
5. The method of claim 4 wherein the deletion and addition of the original fixed sized code word node includes:
establishing a priority queue to record nodes in the order they are used to represent subsequent strings of data;
updating the history index of the original fixed sized code word node and moving the record of the node to the start of priority queue.
6. The method of claim 5 wherein the deletion and addition of the fixed sized code word node includes:
establishing a second priority queue to record empty nodes in the order that they are deleted; and
moving the record of the deleted node to the end of the second priority queue.
7. The method of claim 1 wherein the deletion and addition of the original fixed sized code word node includes:
ordering the nodes in the dictionary in a binary tree by the time they are used to represent subsequent strings of data; and
moving the node representing the original fixed sized code word to the top of the order in the binary tree when the code word is used to represent a subsequent string of data.
8. The method of claim 7 wherein the deletion and addition of the original fixed sized code word node includes:
establishing a heap structure ordering records representing the deleted nodes in the dictionary; and
adding a record of a node which is deleted to the end of the heap structure.
9. The method of claim 1 wherein the communications channel is a wireless link.
10. The method of claim 1 wherein the communications channel is a fiber optic cable.
11. The method of claim 1 wherein the communications channel is a phone line.
12. A data compressor which compresses a digital stream of data, the compressor comprising:
an encoder which matches strings of digital data with fixed size code words;
a transmitter coupled to the encoder which sends the fixed size code word;
a memory coupled to the encoder to store a dictionary with nodes representing strings of digital data, the fixed size code word and an index relating to the frequency that the fixed size code words are sent, wherein the fixed size code words are deleted from the dictionary as additional fixed sized code words are added; and
a processor which, when a string is matched, creates a copy of the node representing the string, creates a new node to store the copy of the node, and deletes the node representing the string from the dictionary.
13. The data compressor of claim 12 wherein the memory includes a linked list with entries to record nodes in the order they are used to represent subsequent strings of data; and
wherein the entry relating to a node is updated by moving the entry to the start of the linked list when the node is used.
14. The data compressor of claim 13 wherein the memory includes a second linked list with entries to record nodes which have been deleted; and
wherein a new entry is created at the end of the second linked list for each node deleted.
15. The data compressor of claim 12 wherein the nodes are maintained by a binary tree structure.