Temporal optimization of data operations using distributed search and server management

1. A method, comprising:

configuring one or more host groups using a processor, each of the one or more host groups being associated with a class and a range of time characteristics;

using a manifest file, by the processor, to determine allocation of data associated with the one or more host groups among one or more shards associated with a server configured to store the data, the data being retrieved from one or more social media applications or networks in data communication with a platform configured to the data in one or more data streams that identifies other data types at parsing by the platform, the data and the data types being stored on the one or more shards based on a time characteristic associated with the data, the time characteristic being used to identify a partition associated with the one or more shards on which is stored the data;

determining one or more stripes associated with the one or more shards distributed among the one or more host groups, a distribution of the one or more stripes, the one or more shards, and the one or more host groups being determined by the processor based on an association with the class and the range of time characteristics;

receiving a query at the processor to retrieve the data, the data comprising social data and metadata associated with the social data;

evaluating, by the processor, the query to identify the time characteristic associated with the data;

identifying a location from which to retrieve the data, the location being identified by the processor processing the query to compare the time characteristic to the range of time characteristics associated with at least one of the one or more host groups, and by identifying a class associated with the one or more host groups at the location where the data is stored; and

rewriting the query to run on at least one of the one or more host groups at the location using a distributed search platform, the processor targeting the another query at a host group associated with the class, wherein a storage device associated with the one or more host groups is configured to transfer the data at a first rate and another storage device associated with the one or more host groups is configured to transfer the data a second rate, the data being transferred from the storage device to the another storage device to identify a location from which to retrieve the data, the location being identified by comparing the time characteristic to the range of time characteristics and the time characteristic exceeds the range of time characteristics.

2. The method of claim 1 , wherein the distributed search platform comprises Solr.

3. The method of claim 1 , wherein the time characteristic comprises a time associated with the social data.

4. The method of claim 1 , wherein the time characteristic comprises a time at which the social data was generated.

5. The method of claim 1 , wherein the one or more host groups are logically configured in one or more tiers, each of the one or more tiers being established based on the class.

6. The method of claim 1 , wherein the query is rewritten to run against a plurality of classes of storage devices.

7. The method of claim 1 , wherein the one or more host groups comprise at least one physical storage device.

8. The method of claim 1 , wherein the one or more host groups comprise at least one logical storage device.

9. The method of claim 1 , wherein the location comprises one or more shards distributed across a plurality of storage devices comprising physical and virtual storage devices.

10. The method of claim 1 , wherein at least one storage device associated with the one or more host groups is configured to transfer the data at a first rate and another storage device associated with the one or more host groups is configured to transfer the data a second rate.

11. The method of claim 1 , wherein a storage device associated with the one or more host groups is configured to transfer the data at a first rate and another storage device associated with the one or more host groups is configured to transfer the data a second rate, the data being transferred from the storage device to the another storage device during identification of a location from which to retrieve the data, the location being identified by comparing the time characteristic to the range of time characteristics and the time characteristic is within the range of time characteristics.

12. A non-transitory computer readable medium having one or more computer program instructions configured to perform a method, the method comprising:

configuring one or more host groups, each of the one or more host groups being associated with a class and a range of time characteristics;

using a manifest file to determine allocation of data associated with the one or more host groups among one or more shards associated with a server configured to store the data, the data being retrieved from one or more social media applications or networks in data communication with a platform configured to the data in one or more data streams that identifies other data types at parsing by the platform, the data and the data types being stored on the one or more shards based on a time characteristic associated with the data, the time characteristic being used to identify a partition associated with the one or more shards on which is stored the data;

determining one or more stripes associated with the one or more shards distributed among the one or more host groups, a distribution of the one or more stripes, the one or more shards, and the one or more host groups being determined based on an association with the class and the range of time characteristics;

receiving a query to retrieve the data, the data comprising social data and metadata associated with the social data;

evaluating the query to identify the time characteristic associated with the data;

identifying a location from which to retrieve the data, the location being identified by comparing the time characteristic to the range of time characteristics associated with at least one of the one or more host groups, and by identifying a class associated with the one or more host groups at the location where the data is stored; and

rewriting the query to run on at least one of the one or more host groups at the location using a distributed search platform, the another query being targeted at a host group associated with the class, wherein a storage device associated with the one or more host groups is configured to transfer the data at a first rate and another storage device associated with the one or more host groups is configured to transfer the data a second rate, the data being transferred from the storage device to the another storage device to identify a location from which to retrieve the data, the location being identified by comparing the time characteristic to the range of time characteristics and the time characteristic exceeds the range of time characteristics.