Granular restore view using out-of-band continuous metadata collection
Disclosed are systems and methods for using a metadata agent to collect metadata from a client computer system and including receiving the metadata associated with files used by applications executing in the client computer system. The metadata belong to objects stored in a volume associated with the client computer system. The metadata is collected in real time by the metadata agent executing in the client computer system as a background task. The metadata is stored in a metadata base associated with a metadata server. A request to search the metadata is received from a backup server associated with the metadata server. The search request includes search criteria. A search engine associated with the metadata server searches the metadata in the metadata database to find a match based on the search criteria. A search result is transmitted from the metadata server to the backup server which uses the search result to search for objects in the backup data.
1. A computer-implemented method comprising:
receiving, by a metadata server, incremental metadata changes associated with objects in a client computer system, the incremental metadata changes collected in real time by a metadata agent executing in the client computer system;
storing, by the metadata server, the incremental metadata changes in a metadata database associated with the metadata server;
receiving, by the metadata server and from a backup server, a request to search the metadata database for metadata of the objects associated with a particular backup point, the request including information enabling the metadata server to construct search criteria, the information including at least point-in-time information;
consolidating, by the metadata server, the incremental metadata changes to form a single consolidated metadata change in response to receiving the request to search the metadata database from the backup server and in response to determining a number of the incremental metadata changes collected exceeds a number of backups performed during a time period, the single consolidated metadata change applying all of the incremental metadata changes;
associating, by the metadata server, the single consolidated metadata change with backup metadata associated with the backup server coupled to the metadata server in response to receiving the request to search the metadata database from the backup server; and
searching, by the metadata server, the metadata database based on a search criteria, the search criteria being generated by the metadata server using the information included in the received request from the backup server.
2. The method of claim 1 , further comprising providing a search result to the backup server when there is a match in the metadata database based on the search criteria.
3. The method of claim 1 , wherein said associating is performed in real time.
4. The method of claim 3 , wherein the incremental metadata changes are received from the metadata agent over a communication channel independent of a communication channel used to transmit backup data from the client computer system to the backup server.
5. The method of claim 4 , wherein the backup metadata is associated with the backup data and include information about the backup data.
6. The method of claim 5 , wherein the incremental metadata changes received from the metadata agent includes at least host name information associated with the client computer system, volume and path information associated with a volume, and path information associated with the objects.
7. A computer program product comprising computer-readable program code to be executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code including instructions to:
receive, by a metadata server, incremental metadata changes associated with objects in a client computer system, the incremental metadata changes collected in real time by a metadata agent executing in the client computer system;
store, by the metadata server, the incremental metadata changes in a metadata database associated with the metadata server;
receive, by the metadata server and from a backup server, a request to search the metadata database for metadata of the objects associated with a particular backup point, the request including information enabling the metadata server to construct search criteria, the information including at least point-in-time information;
consolidate, by the metadata server, the incremental metadata changes to form a single consolidated metadata change in response to receiving the request to search the metadata database from the backup server and in response to determining a number of the incremental metadata changes collected exceeds a number of backups performed during a time period, the single consolidated metadata change applying all of the incremental metadata changes;
associate, by the metadata server, the single consolidated metadata change with backup metadata associated with the backup server coupled to the metadata server in response to receiving the request to search the metadata database from the backup server; and
search, by the metadata server, the metadata database based on a search criteria, the search criteria being generated by the metadata server using the information from the received request from the backup server.
8. The computer program product of claim 7 , wherein the program code includes further instructions to provide a search result to the backup server when there is a match in the metadata database based on the search criteria.
9. The computer program product of claim 7 , wherein said associating is performed in real time.
10. The computer program product of claim 9 , wherein the incremental metadata changes are received from the metadata agent over a communication channel independent of a communication channel used to transmit backup data from the client computer system to the backup server.
11. The computer program product of claim 10 , wherein the backup metadata is associated with the backup data and include information about the backup data.
12. The computer program product of claim 11 , wherein the incremental metadata changes received from the metadata agent includes at least host name information associated with the client computer system, volume and path information associated with a volume, and path information associated with the objects.
13. A system comprising:
a processor-based application, which when executed on a computer having one or more processors, will cause the one or more processors to:
receive, by a metadata server, incremental metadata changes associated with objects in a client computer system, the incremental metadata changes collected in real time by a metadata agent executing in the client computer system;
store, by the metadata server, the incremental metadata changes in a metadata database associated with the metadata server;
receive, by the metadata server and from a backup server, a request to search the metadata database for metadata of the objects associated with a particular backup point, the request including information enabling the metadata server to construct search criteria, the information including at least point-in-time information;
consolidate, by the metadata server, the incremental metadata changes to form a single consolidated metadata change in response to receiving the request to search the metadata database from the backup server and in response to determining a number of the incremental metadata changes collected exceeds a number of backups performed during a time period, the single consolidated metadata change applying all of the incremental metadata changes;
associate, by the metadata server, the single consolidated metadata change with backup metadata associated with the backup server coupled to the metadata server in response to receiving the request to search the metadata database from the backup server; and
search, by the metadata server, the metadata database based on a search criteria, the search criteria being generated by the metadata server using the information from the received request from the backup server.
14. The system of claim 13 , wherein the one or more processors is further configured to provide a search result to the backup server when there is a match in the metadata database based on the search criteria.
15. The system of claim 13 , wherein said associating is performed in real time.
16. The system of claim 15 , wherein the incremental metadata changes are received from the metadata agent over a communication channel independent of a communication channel used to transmit backup data from the client computer system to the backup server.
17. The system of claim 16 , wherein the backup metadata is associated with the backup data and include information about the backup data, and wherein the incremental metadata changes received from the metadata agent includes at least host name information associated with the client computer system, volume and path information associated with a volume, and path information associated with the objects.