Methods for encoding and decoding data
The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.
1. A method for processing data, the data residing in a data field, comprising:
recognizing any characteristic, attribute, or parameter of the data;
selecting an encoder associated with the recognized characteristic, attribute, or parameter of the data;
compressing the data with the selected encoder to create compressed data wherein the compressing achieves a compression ratio of over 10:1 on the data; and
broadcasting the compressed data to a plurality of clients.
2. The method of claim 1 , wherein the compressing and broadcasting occur over a time period which is less than a time to broadcast the data in uncompressed form.
3. The method of claim 1 , further comprising:
decompressing the compressed data, and wherein the compressing, the broadcasting, and the decompressing occur over a time period which is less than a time to broadcast the data in uncompressed form.
4. The method of claim 1 , wherein multiple messages are contained in a data packet.
5. The method of claim 1 , wherein the data comprises financial data transmitted in sequence.
6. The method of claim 1 , wherein the data field includes stock information.
7. The method of claim 1 , wherein the data field includes options information.
8. The method of claim 1 , wherein the compressing is lossy.
9. The method of claim 1 , wherein the broadcasting comprises:
broadcasting the compressed data utilizing a User Datagram Protocol (UDP).
10. The method of claim 1 , wherein the compressing is performed on a server and wherein the compressed data is broadcast from the server.
11. The method of claim 1 , wherein the recognizing includes analyzing the data within the data field and excludes analyzing based on a descriptor that is indicative of the recognized characteristic, attribute, or parameter of the data within the data field.
12. A method for processing data, the data residing in data fields, comprising:
recognizing any characteristic, attribute, or parameter of the data;
selecting an encoder associated with the recognized characteristic, attribute, or parameter of the data;
compressing the data with the selected encoder utilizing at least one state machine to provide compressed data having a compression ratio of over 4:1; and
point-to-point transmitting the compressed data to a client;
wherein the compressing and the transmitting occur over a period of time which is less than a time to transmit the data in an uncompressed form.
13. The method of claim 12 , further comprising:
decompressing the compressed data at the client; and
wherein the compressing, the transmitting, and the decompressing occur over a period of time that is less than the time to transmit the data in uncompressed form.
14. The method of claim 12 , wherein a data packet that includes the data fields also includes multiple messages.
15. The method of claim 12 , wherein the compressed data is transmitted as part of one or more data blocks of financial data transmitted in sequence.
16. The method of claim 12 , wherein the data field includes stock information.
17. The method of claim 12 , wherein the data field includes options information.
18. The method of claim 12 , wherein the compressing is lossy.
19. The method of claim 12 , wherein the transmitting comprises:
transmitting the data utilizing a User Diagram Protocol (UDP).
20. The method of claim 12 , wherein the compressing is performed on a server, and wherein the compressed data is transmitted from the server.
21. The method of claim 12 , wherein the recognizing includes analyzing the data within the data fields and excludes analyzing based on a descriptor that is indicative of the recognized characteristic, attribute, or parameter of the data within the data fields.
22. A method for processing data, the data residing in data fields, comprising:
selecting an encoder associated with a data field from among a plurality of data fields in a packet of one or more data blocks of financial data in sequence, the selecting being performed at least in part by utilizing a list indicative of data fields and their associated encoders;
encoding data in each data field from among the plurality of data fields to provide one or more encoded data blocks of financial data, wherein the size of encoded data in each data field from among the plurality of data fields is at least one-fourth the size of the data before the encoding; and
broadcasting the encoded one or more data blocks in sequence to a plurality of clients, wherein the encoding and the broadcasting occur over a period of time which is less than a time to broadcast the data in each data field from among the plurality of data fields in an unencoded form.
23. The method of claim 22 , wherein the encoded one or more data blocks of financial data include date information.
24. The method of claim 22 , wherein the encoded one or more data blocks of financial data include sequence information.
25. The method of claim 22 , wherein the encoded one or more data blocks of financial data include multiple messages.
26. The method of claim 22 , wherein the encoded one or more data blocks of financial data broadcast in sequence include stock information.
27. The method of claim 22 , wherein the encoded one or more data blocks of financial data broadcast in sequence include options information.
28. The method of claim 22 , wherein the encoding comprises:
lossless encoding the data in each data field from among the plurality of data fields.
29. The method of claim 22 , wherein the broadcasting comprises: broadcasting the encoded one or more data blocks in sequence utilizing a User Datagram Protocol (UDP).
30. The method of claim 22 , wherein the encoding is performed on a server, and wherein the encoded one or more data blocks broadcast in sequence is broadcast from the server.