Managing memory resources in a network environment in order to handle querying of logical data structures
View Patent ↗A memory management module operates to receive a query that identifies a set of data of a logical data structure. The logical data structure is stored with memory resources that are provided on multiple service-level tiers of a network environment, where the multiple service-level tiers may include at least a first tier of memory resources and a second tier of memory resources. From the query, a subset of data is determined that is stored with the second tier of memory resources. The memory management module responds to detecting the condition or event by causing transfer of the subset of data items from the second tier of memory resources to the first tier of memory resources.
1. A method for managing data storage for query processing in a network environment, the method being implemented by one or more processors and comprising:
(a) maintaining a metadata representation of a logical data structure that is queryable by entities of the network environment, the logical data structure spanning a first set of logical partitions on a first tier of memory resources and a second set of logical partitions on a second tier of memory resources, the logical data structure being optimized to store data for responses to queries in the first tier of memory resources;
(b) detecting a condition or event for providing data stored with the second tier of memory resources in response to queries; and
(c) in response to detecting the condition or event, transferring data from the second tier of memory resources to the first tier of memory resources.
2. The method of claim 1 , wherein (b) includes receiving an incoming query that references data stored with the second tier of memory resources, and wherein (c) is performed in response to receiving the incoming query.
3. The method of claim 2 , further comprising responding to the incoming query using data stored with the first tier of memory resources, data stored with the first tier of memory resources including the data transferred from the second tier of memory resources.
4. The method of claim 3 , wherein (c) includes generating an internal query for the referenced data stored with the second tier of memory resources in response to receiving the incoming query.
5. The method of claim 4 , wherein the incoming query references data stored with the first tier of memory resources and with the second tier of memory resources, and wherein the method further comprises returning a result of the incoming query using (i) data that was originally stored with the first tier of memory resources and (ii) data transferred to the first tier of memory resources from the second tier of memory resources.
6. The method of claim 1 , wherein (b) includes detecting a scheduled event or condition.
7. The method of claim 1 , wherein (b) includes detecting an overall demand for data of the logical data structure.
8. The method of claim 1 , wherein (b) monitoring activity in relation to the logical data structure.
9. The method of claim 1 , further comprising determining when the first tier of memory resources has insufficient resources to handle one or more queries, and then dynamically allocating additional memory resources to the first tier of memory resources in response.
10. A non-transitory computer-readable medium that stores instructions for managing data storage for query processing in a network environment, wherein the instructions, when executed by one or more processors, cause the one or more processors to perform operations that include:
maintaining a metadata representation of a logical data structure that is queryable by entities of the network environment, the logical data structure spanning a first set of logical partitions on a first tier of memory resources and a second set of logical partitions on a second tier of memory resources, the logical data structure being optimized to store data for responses to queries in the first tier of memory resources;
detecting a condition or event for providing data stored with the second tier of memory resources in response to queries; and
in response to detecting the condition or event, transferring data from the second tier of memory resources to the first tier of memory resources.
11. The non-transitory computer-readable medium of claim 10 , wherein instructions for detecting the condition includes receiving an incoming query that references data stored with the second tier of memory resources, and wherein operations for transferring data is performed in response to receiving the incoming query.
12. The non-transitory computer-readable medium of claim of claim 11 , further comprising instructions for performing operations that include responding to the incoming query using data stored with the first tier of memory resources, wherein the data stored with the first tier of memory resources including the data transferred from the second tier of memory resources.
13. The non-transitory computer-readable medium of claim of claim 12 , wherein instructions for transferring data includes generating an internal query for the referenced data stored with the second tier of memory resources in response to receiving the incoming query.
14. The non-transitory computer-readable medium of claim 13 , wherein the incoming query references data stored with the first tier of memory resources and with the second tier of memory resources, and wherein the instructions provide for operations that include returning a result of the incoming query using (i) data that was originally stored with the first tier of memory resources and (ii) data transferred to the first tier of memory resources from the second tier of memory resources.
15. The non-transitory computer-readable medium of claim 10 , further comprising instructions for performing operations that include, wherein (b) includes detecting a scheduled event or condition.
16. The non-transitory computer-readable medium of claim 10 , further comprising instructions for performing operations that include, wherein (b) includes detecting an overall demand for data of the logical data structure.
17. The non-transitory computer-readable medium of claim 10 , further comprising instructions for performing operations that include, wherein (b) monitoring activity in relation to the logical data structure.
18. The non-transitory computer-readable medium of claim 10 , further comprising instructions for performing operations that include, further comprising determining when the first tier of memory resources has insufficient resources to handle one or more queries, and then dynamically allocating additional memory resources to the first tier of memory resources in response.