System and method for a data extraction and backup database
View Patent ↗Methods and systems for storing information extracted from a file are presented. These methods and systems can be used to store content and metadata extracted from a file, and to associate the content and metadata so a holistic image of the file may be maintained. Additionally, these methods and systems may allow the location of a file to be stored and associated with the content or metadata of the file. Methods, systems and databases of this type may be especially useful in avoiding duplication of data by allowing the content and metadata of files to be compared to previously stored content and metadata.
1. A method of extracting original data from at least one file that resides on one or more different mediums operating on one or more different platforms, wherein the original data is extracted and stored at a target location without unnecessary replication of duplicate data, comprising the steps of:
obtaining content data and metadata relating to the original data of the at least one file, wherein the at least one file is stored in a first environment created on the one or more different mediums operating in the one or more different platforms;
in a second environment created on the one or more different mediums operating in the one or more different platforms and the target location, storing the content data and the metadata of the at least one file, wherein the content data is associated with the metadata, and wherein the second environment is different from the first environment;
obtaining an original location corresponding to the at least one file, the original location indicative of where the at least one file is stored in the first environment;
in the second environment, storing the original location in a location table, wherein the location table includes the following:
a link to the content data stored in a content hash table, indexed in response to the original location indicative of where the at least one file was stored in the first environment; and
a link to the metadata stored in a metadata hash table, indexed in response to the original location where the file was stored in the first environment;
reconstituting at least a piece of the at least one file by accessing at least one of the following in the location table:
the link to the content data, in response to the original location; and
the link to the metadata, in response to the original location; and
storing the content data in the content hash table and the metadata in the metadata hash table, wherein the storing operation further comprises the steps of;
generating a digital signature from the content data; and
utilizing the digital signature to compare the content data of the least one file that is being extracted with content data already resident at the target location to avoid replication of content data and only storing content data that is not duplicative of the content data that is already resident at the target location.
2. The method of claim 1 , wherein the step of storing the content and metadata further comprises the steps of:
generating a digital signature from the metadata;
storing the content in an entry in the content hash table, wherein the content's digital signature is an index into the content hash table, so that the content's digital signature is the link to the content; and
storing the metadata in an entry in the metadata hash table, wherein the metadata's digital signature is an index into the metadata hash table, so that the metadata's digital signature is the link to the metadata.
3. The method of claim 2 , wherein at least one or both of the digital signatures is generated using a hashing algorithm.
4. The method of claim 3 , wherein the hashing algorithm is the SHA1 secure hashing algorithm.
5. The method of claim 2 , wherein the entry in the content hash table comprises the content data and the link to the metadata.
6. The method of claim 5 , wherein the entry in the metadata hash table comprises the metadata and the link to the content data.
7. The method of claim 2 , wherein the location table is a location hash table, and wherein storing the location comprises the steps of:
generating a digital signature from the location; and
storing the original location in an entry in the location hash table, wherein the location's digital signature is an index into the location hash table, so that the location hash table is indexed in response to the original location by indexing with the location's digital signature.
8. A data processing system comprising at least one computer and one or more data sources that are either stand alone or networked, the data processing system for storing components of at least one file that was stored in a first environment at the one or more data sources, the data processing system utilizing code embodied within the data processing system configured to perform operations, the operations comprising:
obtaining content data and metadata relating to the original data of the at least one file, wherein the at least one file is stored in a first environment created on the one or more different mediums operating in the one or more different platforms;
in a second environment created on the one or more different mediums operating in the one or more different platforms and the target location, storing the content data and the metadata of the at least one file, wherein the content data is associated with the metadata, and wherein the second environment is different from the first environment;
obtaining an original location corresponding to the at least one file, the original location indicative of where the at least one file is stored in the first environment;
in the second environment, storing the original location in a location table, wherein the location table includes the following:
a link to the content data stored in a content hash table, indexed in response to the original location indicative of where the at least one file was stored in the first environment; and
a link to the metadata stored in a metadata hash table, indexed in response to the original location where the file was stored in the first environment;
reconstituting at least a piece of the at least one file by accessing at least one of the following in the location table:
the link to the content data, in response to the original location; and
the link to the metadata, in response to the original location; and
storing the content data in the content hash table and the metadata in the metadata hash table, wherein the storing operation further comprises the steps of;
generating a digital signature from the content data; and
utilizing the digital signature to compare the content data of the least one file that is being extracted with content data already resident at the target location to avoid replication of content data and only storing content data that is not duplicative of the content data that is already resident at the target location.
9. The data processing system of claim 8 , wherein the operation of storing the content and metadata further comprises the operations of:
generating a digital signature from the metadata;
storing the content in an entry in the content hash table, wherein the content's digital signature is an index into the content hash table, so that the content's digital signature is the link to the content; and
storing the metadata in an entry in the metadata hash table, wherein the metadata's digital signature is an index into the metadata hash table, so that the metadata's digital signature is the link to the metadata.
10. The data processing system of claim 9 , wherein at least one or both of the digital signatures is generated using a hashing algorithm.
11. The data processing system of claim 10 , wherein the hashing algorithm is the SHA1 secure hashing algorithm.
12. The data processing system of claim 9 , wherein the entry in the content hash table comprises the content data and the link to the metadata.
13. The data processing system of claim 12 , wherein the entry in the metadata hash table comprises the metadata and the link to the content data.
14. The data processing system of claim 9 , wherein the location table is a location hash table, and wherein storing the location comprises the operations of:
generating a digital signature from the location; and
storing the original location in an entry in the location hash table, wherein the location's digital signature is an index into the location hash table, so that the location hash table is indexed in response to the original location by indexing with the location's digital signature.
15. A non-transitory computer readable medium including instructions configured to utilize code for storing information related to at least one file, wherein the code is embodied within the data processing system, the code comprising instructions configured to perform operations including:
obtaining content data and metadata relating to the original data of the at least one file, wherein the at least one file is stored in a first environment created on the one or more different mediums operating in the one or more different platforms;
in a second environment created on the one or more different mediums operating in the one or more different platforms and the target location, storing the content data and the metadata of the at least one file, wherein the content data is associated with the metadata, and wherein the second environment is different from the first environment;
obtaining an original location corresponding to the at least one file, the original location indicative of where the at least one file is stored in the first environment;
in the second environment, storing the original location in a location table, wherein the location table includes the following:
a link to the content data stored in a content hash table, indexed in response to the original location indicative of where the at least one file was stored in the first environment; and
a link to the metadata stored in a metadata hash table, indexed in response to the original location where the file was stored in the first environment;
reconstituting at least a piece of the at least one file by accessing at least one of the following in the location table:
the link to the content data, in response to the original location; and
the link to the metadata, in response to the original location; and
storing the content data in the content hash table and the metadata in the metadata hash table, wherein the storing operation further comprises the steps of;
generating a digital signature from the content data; and
utilizing the digital signature to compare the content data of the least one file that is being extracted with content data already resident at the target location to avoid replication of content data and only storing content data that is not duplicative of the content data that is already resident at the target location.
16. The computer readable medium of claim 15 , wherein the operation of storing the content and metadata further comprises the operations of:
generating a digital signature from the metadata;
storing the content in an entry in the content hash table, wherein the content's digital signature is an index into the content hash table, so that the content's digital signature is the link to the content; and
storing the metadata in an entry in the metadata hash table, wherein the metadata's digital signature is an index into the metadata hash table, so that the metadata's digital signature is the link to the metadata.
17. The computer readable medium of claim 16 , wherein at least one or both of the digital signatures is generated using a hashing algorithm.
18. The computer readable medium of claim 17 , wherein the hashing algorithm is the SHA1 secure hashing algorithm.
19. The computer readable medium of claim 16 , wherein the entry in the content hash table comprises the content data and the link to the metadata.
20. The computer readable medium of claim 19 , wherein the entry in the metadata hash table comprises the metadata and the link to the content data.
21. The computer readable medium of claim 16 , wherein the location table is a location hash table, and wherein storing the location comprises the operations of:
generating a digital signature from the location; and
storing the original location in an entry in the location hash table, wherein the location's digital signature is an index into the location hash table, so that the location hash table is indexed in response to the original location by indexing with the location's digital signature.