Method and system for facilitating access to recorded data

1. A system configured to access recorded data for detecting market abuses amongst market transactions, the system comprising:

a processor;

a primary memory; and

a secondary memory, wherein the system is configured to:

receive data via an interface of the system;

separate the received data into data subsets;

apply a compression technique to compress each data subset such that each data subset includes compressed payload data;

assign identifiers to each compressed data subset to create data units;

establish an index on the basis of the identifiers assigned to each compressed data subset, wherein the index indicates a physical location of each data unit in the primary memory and the secondary memory, the index having one or more index entries, and each index entry comprising:

an identifier from the assigned identifiers;

timing parameters corresponding to at least a point in time when an order was placed at an exchange and a point in time when a transaction associated with the order closed; and

memory address parameters of memory locations that contain the data units associated with the order;

search at least one of the primary memory and the secondary memory, for the data units, using the established index;

decompress the data units associated with the order within a window of time using the timing parameters; and

present a windowing view of an orderbook using the decompressed data units based on the window of time associated with the timing parameters.

2. The system of claim 1 , wherein the memory address parameters include a first start address and a first end address.

3. The system of claim 1 , wherein

the primary memory includes a cache memory; and

the secondary memory includes a persistent storage memory.

4. The system of claim 3 , wherein the system is further caused to search in both the primary memory, by accessing the cache memory for real-time analysis, and the second memory, by accessing the persistent storage memory for historical analysis, using the established index.

5. The system of claim 3 , wherein the primary memory is configured to store the data units created by the processing device, the stored data units being transferred from the primary memory when a primary memory utilization degree reaches an upper threshold value.

6. The system of claim 5 , wherein the system is further caused to transfer the data units to the secondary memory until the primary memory utilization degree falls to a lower threshold value, wherein a next transfer of data units from the primary memory to the secondary memory is performed when the primary memory utilization degree again reaches the upper threshold value.

7. The system of claim 3 , wherein the primary memory is configured to store the created data units, the stored data units being transferred from the primary memory to the secondary memory when the primary memory receives a memory flush instruction.

8. The system of claim 1 , wherein the system is further caused to locate a group of compressed data subsets based on an interval of time associated with the timing window defined by the timing parameters associated with each index entry and decompress each data subset in the group of compressed data subsets thereby generating decompressed payload data.

9. A method for accessing recorded data, comprising:

at an information processing system having at least a processor, a primary memory, and a secondary memory:

receiving data via an interface of the information processing system;

separating the received data into data subsets;

applying a compression technique to compress each data subset such that each data subset includes compressed payload data;

assigning identifiers to each compressed data subset to create data units;

establishing an index on the basis of the identifiers assigned to each compressed data subset, wherein the index indicates a physical location of each data unit in the primary memory and the secondary memory, the index having one or more index entries, and each index entry comprising:

an identifier from the assigned identifiers; and

timing parameters corresponding to at least a point in time when an order was placed at an exchange and a point in time when a transaction associated with the order closed;

searching at least one of the primary memory and the secondary memory, for the data units, using the established index; and

decompressing the data units associated with the order within a window of time using the timing parameters.

10. The method of claim 9 , wherein each index entry further comprises memory address parameters of memory locations that contain the data units associated with the order.

11. The method of claim 10 , wherein the memory address parameters include a first start address, a first end address, a second start address, and a second end address.

12. The method of claim 9 , further comprising locating a group of compressed data subsets based on an interval of time associated with the timing window defined by the timing parameters associated with each index entry and decompressing each data subset in the group of compressed data subsets thereby generating decompressed payload data.

13. The method of claim 9 , further comprising searching the primary memory for real time analysis and searching the secondary memory for historical analysis using the established index.

14. A system, comprising:

one or more terminal devices; and

an information processing system having at least a primary memory and a secondary memory, wherein the information processing system is configured to:

receive data via an interface of the information processing system;

separate the received data into data subsets;

apply a compression technique to compress each data subset such that each data subset includes compressed payload data;

assign identifiers to each compressed data subset to create data units;

establish an index on the basis of the identifiers assigned to each compressed data subset, wherein the index indicates a physical location of each data unit in the primary memory and the secondary memory, the index having one or more index entries, and each index entry comprising:

an identifier from the assigned identifiers; and

timing parameters corresponding to at least a point in time when an order was placed at an exchange and a point in time when a transaction associated with the order closed;

search at least one of the primary memory and the secondary memory, for the data units, using the established index; and

decompress the data units associated with the order within a window of time using the timing parameters.

15. The system of claim 14 , wherein each index entry further comprises memory address parameters of memory locations that contain the data units associated with the order.

16. The system of claim 15 , wherein the memory address parameters include a first start address and a first end address.

17. The system of claim 14 , wherein the information processing system is further configured to locate a group of compressed data subsets based on an interval of time associated with the timing window defined by the timing parameters associated with each index entry and decompress each data subset in the group of compressed data subsets thereby generating decompressed payload data.

18. The system of claim 14 , wherein the information processing system is further configured to search the primary memory for real time analysis and search the secondary memory for historical analysis using the established index.

19. The method of claim 9 , further comprising:

presenting a windowing view of an orderbook using the decompressed data units based on the window of time associated with the timing parameters.

20. The system of claim 14 , wherein the information processing system is further configured to:

present a windowing view of an orderbook using the decompressed data units based on the window of time associated with the timing parameters.