System and method for on-demand indexing
A system and method for on-demand indexing in a data management system is described. An index is generated when it is requested, such as when a database operation requires access to the index. If the index is loaded in memory, the index is retrieved from memory. Otherwise, the index is generated on-demand. A priority configuration identifies at least one priority index which is generated and loaded in memory. The priority configuration can identify priority indexes either directly or indirectly, such as by a threshold parameter.
1. A method of on-demand generation of an index of records in a data management system comprising:
receiving a priority configuration including information identifying at least one priority index associated with records in said data management system;
generating said at least one priority index prior to receiving any request for the priority index, and loading said at least one priority index in said memory;
receiving a request for a first index;
determining whether said first index is currently loaded in memory, wherein at least some indexes are currently loaded in memory and other potential indexes are not currently loaded in memory;
retrieving said first index from said memory when said first index is currently loaded in said memory;
generating said first index by accessing said records when said first index is not currently loaded in said memory;
retrieving a Δ index associated with said first index, wherein said Δ index comprises at least one entry reflecting an addition or modification of a record affecting said first index;
updating said first index using said Δ index to reflect changes identified by said at least one entry of said Δ index; and
providing said first index.
2. The method of claim 1 , further comprising:
automatically identifying at least one threshold priority index using a threshold parameter;
generating said at least one threshold priority index prior to receiving any request for the threshold priority index; and
loading said at least one threshold priority index in said memory;
wherein said priority configuration further includes said threshold parameter.
3. The method of claim 2 , wherein said threshold parameter indicates a prior frequency of index use.
4. The method of claim 1 , wherein said priority configuration is defined by a user via a graphical user interface.
5. The method of claim 1 , further comprising:
automatically generating said priority configuration based on a prior frequency of index use.
6. A non-transitory computer-readable medium storing instructions and data configured to perform an automated process when executed by a processor, said automated process comprising
receiving a priority configuration including information identifying at least one priority index associated with records in a data management system;
generating said at least one priority index prior to receiving any request for the priority index, and loading said at least one priority index in said memory;
receiving a request for a first index;
determining whether said first index is currently loaded in memory, wherein at least some indexes are currently loaded in memory and other potential indexes are not currently loaded in memory;
retrieving said first index from said memory when said first index is currently loaded in said memory;
generating said first index by accessing said records when said first index is not currently loaded in said memory;
retrieving a Δ index associated with said first index, wherein said Δ index comprises at least one entry reflecting an addition or modification of a record affecting said first index;
updating said first index using said Δ index to reflect changes identified by said at least one entry of said Δ index; and
providing said first index.
7. The medium of claim 6 , wherein said automated process further includes:
using at least one threshold parameter to automatically identify any threshold priority indexes;
generating said any threshold priority indexes prior to receiving any request for the threshold priority index; and
loading said any threshold priority indexes in said memory,
wherein said priority configuration further includes said at least one threshold parameter.
8. A computer system comprising:
a computer;
a non-transitory computer-readable medium coupled to said computer, said computer-readable medium storing instructions and data configured to perform an automated process when executed by said computer, said automated process comprising:
receiving a priority configuration including information identifying at least one priority index;
generating said at least one priority index prior to receiving any request for the priority index;
loading said at least one priority index in memory;
receiving a request for a first index;
determining whether said first index is currently loaded in said memory, wherein at least some indexes are currently loaded in memory and other potential indexes are not currently loaded in memory;
retrieving said first index from said memory when said first index is currently loaded in said memory;
generating said first index by accessing said records when said first index is not currently loaded in said memory;
retrieving a Δ index associated with said first index, wherein said Δ index comprises at least one entry reflecting an addition or modification of a record affecting said first index;
updating said first index using said Δ index to reflect changes identified by said at least one entry of said Δ index; and
providing said first index.
9. The computer system of claim 8 , wherein said automated process further comprises:
using at least one threshold parameter to automatically identify any threshold priority indexes;
generating said any threshold priority indexes prior to receiving any request for the threshold priority index; and
loading said any threshold priority indexes in said memory,
wherein said priority configuration further includes said at least one threshold parameter.
10. The computer system of claim 8 , wherein said request for said first index is received from another process running on said computer system.