System and method for optimizing the storage of data
The disclosure describes systems and methods for optimizing the storage of data collected from a plurality of communication channels including social data, spatial data, temporal data and logical data. Based on user data collected from past interactions with the network, clusters of objects that include common information are combined into single objects in order to save storage space and reduce the number of objects managed by the system. The combination method may also be used to dynamically identify events or conditions which are related to a plurality of entities.
1. A method for storing information objects (IOs) comprising:
generating, via at least one processor of a computing device, a plurality of user-related first IOs, each user-related first IO containing data describing a communication transmitted via one of a plurality of communication channels, the data in each user-related first IO identifying a user that is capable of communicating via at least one of the plurality of communication channels and being at a location at a time;
temporarily storing the plurality of user-related first IOs in a computer-readable storage medium in communication with the computing device;
analyzing, via at least one processor of the computing device, the plurality of user-related first IOs to identify a subset of user-related first IOs comprising data identifying locations of users within a selected distance from a first location and a determined time within a first time period;
generating, via at least one processor of the computing device, an event IO comprising data identifying the locations within the selected distance from the first location, the determined time within the first time period and each of the users identified in the subset of user-related first IOs;
storing the event IO in a datastore in communication with said computing device; and
deleting, via at least one processor of the computing device, the subset of user-related first IOs.
2. The method of claim 1 further comprising:
defining the first time period based upon a number of users within the determined time at the first location.
3. The method of claim 1 further comprising:
defining the first distance based on a number of users within the selected distance from the first location.
4. The method of claim 1 further comprising:
storing, in the event IO, any additional data stored in each of the subset of user-related first IOs.
5. The method of claim 1 further comprising:
generating an event description for the event IO based on information contained in the subset of user-related first IOs.
6. The method of claim 5 further comprising:
generating a plurality of user-related second IOs, each user-related second IO containing data describing a communication transmitted via one of a plurality of communication channels, the data in each user-related second IO containing a future event definition containing an event time and an event title;
searching the plurality of user-related second IOs for future event definitions with an event time within the first time period; and
generating the event description based on titles of future event definitions having an event time within the first time period.
7. The method of claim 6 further comprising:
selecting one of the event titles from the future event definitions having an event time within the first time period.
8. The method of claim 5 further comprising:
generating the event description based on the first location.
9. The method of claim 2 further comprising:
generating the event IO if the number of users within the selected distance from the first location is greater than a predetermined threshold.
10. The method of claim 1 further comprising:
for at least one communication detected, identifying a sending device that transmitted the communication, a transmission time the transmission was sent and a device location of the sending device at the time the transmission was sent;
determining that the sending device is a proxy for a location of a first user; and
generating a user-related first IO identifying the first user being at the device location at the transmission time.
11. A method for storing information objects (IOs) comprising:
monitoring a plurality of communications transmitted via one of a plurality of communication channels on a network;
identifying, via a computing device, a subset of related communications based on an analysis of the content of the plurality of communications;
generating, via the computing device, a user-related information object (IO) from the subset of related communications, the user-related IO containing data from the plurality of communications; and
storing the user-related IO as a record of the subset of related communications in a datastore in communication with the computing device.
12. The method of claim 11 further comprising:
retrieving non-redundant data obtained from each of the subset of related communications from the user-related IO; and
storing the non-redundant data.
13. The method of claim 11 further comprising:
determining that the subset of related communications contains content indicating that a plurality of users were co-located during a specified time period; and
generating an event IO containing the content from the subset of related communications, wherein the event IO identifies the plurality of users, an event location and the specified time period.
14. The method of claim 11 further comprising:
determining that the subset of related communications contains additional content indicating that a plurality of users were performing a specified activity during the specified time period;
generating an activity IO containing the additional content of the subset of related communications, the activity IO identifies the specified activity, the plurality of users and the specified time period.
15. The method of claim 11 further comprising:
determining that the subset of related communications contain information indicating that a plurality of locations were associated with a specified condition during the specified time period;
generating a condition IO containing the information from the subset of related communications, the condition IO identifying the specified condition, the plurality of locations and the specified time period.
16. The method of claim 12 further comprising:
storing a communication identifier in the user-related IO for each communication in the subset of related communications and the associated communication channel.
17. A system for optimizing the storage of user-related information objects (IOs) comprising:
a plurality of processors;
a correlation engine, implemented by at least one of said plurality of processors, connected via at least one communication channel to a plurality of computing devices, that identifies relationships between the user-related IOs and associated real-world entities (RWEs) including the plurality of computing devices;
storage connected to the correlation engine that stores at least one of social data, spatial data, temporal data and logical data associated with the RWEs; and
a data optimization engine implemented by at least one of said plurality of processors that determines redundant data contained with the user-related IOs based upon the correlated relationships between the user-related IOs and the RWEs identified by the correlation engine, and generates a combined user-related IO with at least a portion of the user-related IOs.
18. The system of claim 17 , wherein the combined user-related IO is stored on a data backbone of a network through which the at least one communication channel communicates.
19. The system of claim 18 , wherein the redundant data with the user-related IOs are deleted upon the combined user-related IO accumulating all of the data.