METHOD OF STORING AND ACCESSING DATA
A method of storing and accessing data comprising receiving an initial data block, dividing the initial data block into a plurality of initial subblocks, each initial subblock being the same size, storing each subblock in accordance with a data schema, generating a set of calculated subblocks, each calculated subblock being generated from a plurality of initial subblocks, such that each calculated subblock is the same size as an initial subblocks, storing the calculated subblocks in accordance with the data schema, and generating a reference structure, the reference structure comprising a pointer to each of the stored subblocks.
1 . A method of storing and accessing data comprising;
receiving an initial data block,
dividing the initial data block into a plurality of initial subblocks, each initial subblock being the same size,
storing each subblock in accordance with a data schema,
generating a set of calculated subblocks, each calculated subblock being generated from a plurality of initial subblocks, such that each calculated subblock is the same size as an initial subblocks,
storing the calculated subblocks in accordance with the data schema,
and generating a reference structure, the reference structure comprising a pointer to each of the stored subblocks.
2 . A method according to claim 1 where each calculated subblock is generated from four initial subblocks.
3 . A method according to claim 2 comprising generating a further set of further calculated subblocks, each further calculated subblock being generated from one of;
a plurality of calculated subblocks, or
the plurality of initial subblocks from which the calculated subblocks are generated;
wherein each of the further calculated subblocks is the same size as an initial subblock,
the method further comprising storing the further calculated subblocks in accordance with the data schema and adding a pointer to each of the stored further calculated subblocks to the reference structure.
4 . A method according to claim 3 where each further calculated subblock is based on four calculated subblocks or 16 initial subblocks.
5 . A method according to claim 3 comprising performing the steps of generating additional sets of calculated subblocks based on the further calculated subblocks.
6 . A method according to claim 1 wherein the reference structure comprises a quad-tree, each node corresponding to a subblock comprising the pointer to the stored subblocks and, where the subblock comprises a calculated subblock, a pointer to each node of the tree relating to a subblock from which the calculated subblock was generated.
7 . A method according to claim 1 wherein the data comprises gridded data.
8 . A method according to claim 1 comprising obtaining the initial data block by selectively obtaining data from a data source comprising an index file, the method comprising reading the index file to identify required data within the data source, and downloading the required data accordingly.
9 . A method according to claim 1 wherein the data comprises meteorological data.