Method and system of retrieving data in a data file
A method and system for requesting information from a server are provided. The method may include sending, by a client to a server, an HTTP GET request to retrieve at least a portion of at least one derived data file, the at least one derived data file being dynamically generated by the server in response to the request using a data file virtualization policy and at least a portion of a data file. The data file virtualization policy and the data file may be stored in a storage accessible by the server, and the dynamic generation may comprise an aligned unbuffered direct read operation from the stored data file of a data size that is aligned to an integer multiple of the storage's block size. The at least the portion of the at least one derived data file may then be received by the client from the server.
1. A method of requesting information from a server, the method being implemented by one or more processors configured to execute instructions, which, when executed, cause the one or more processors to:
send, by a client to a server, a hypertext transfer protocol (HTTP) GET request to retrieve at least a portion of at least one derived data file, wherein the at least one derived data file is dynamically generated by the server in response to the request using a data file virtualization policy and at least a portion of a data file, the data file virtualization policy and the data file being stored in a storage accessible by the server, and wherein the dynamic generation comprises an aligned unbuffered direct read operation from the stored data file of a data size that is aligned to an integer multiple of the storage's block size; and
receive, by the client from the server, at least the portion of the at least one derived data file.
2. The method according to claim 1 , wherein the stored data file is stored in the storage among one or more other data files, the stored data file and the one or more other data files being of a same type, and wherein the at least one derived data file is not stored in the storage.
3. The method according to claim 2 , wherein the data file virtualization policy is associated in a one-to-many relationship with the stored data file and the one or more other data files.
4. The method according to claim 1 , wherein the one or more processors are further caused to:
send, by the client to the server, a request for a list of supported derived data types and a list of supported parameters for each of the supported derived data types, wherein the parameters allow a client to control an output of the at least one derived data file;
receive, by the client from the server, the list of supported parameters; and
send, by the client, an intent to request data of at least a subset of the supported derived data types.
5. The method according to claim 1 , wherein the data file virtualization policy comprises:
a name of the stored data file,
a data type of the stored data file,
a Boolean flag set to either True or False, True indicating an intent of the client to request for derived data, and
a list of derived data types that the client intends to retrieve.
6. The method according to claim 5 , wherein the at least one derived data file comprises a plurality of dynamically generated derived data files, a number of the plurality of derived data files being based on the intent of the client to retrieve them.
7. The method according to claim 1 , wherein the stored data file comprises Wide-Area Motion Imagery (WAMI) data, a type of the stored data file being a WAMI Frame.
8. The method according to claim 1 , wherein the one or more processors are further caused to:
send, by the client to the server, a request for a list of supported derived data types;
receive, by the client, the list of supported derived data types; and
generate, by the client, the data file virtualization policy using the list of supported derived data types.
9. The method according to claim 1 , wherein the at least one derived data file is listed in a directory listing generated by the server, and
wherein the directory listing is accessible to the client such that the at least one derived data file is presented to the client as if the at least one derived data file is stored in the storage.
10. The method according to claim 9 , wherein the one or more processors are further caused to:
send, by the client to the server, an HTTP 1.1 WebDAV PROPFIND request for the directory listing.
11. A system, comprising:
a client computer system comprising one or more processors and a network connection to a server computer system, the client computer system being configured to:
send, to a server computer system, a hypertext transfer protocol (HTTP) GET request to retrieve at least a portion of at least one derived data file, wherein the at least one derived data file is dynamically generated by the server computer system in response to the request using a data file virtualization policy and at least a portion of a data file, the data file virtualization policy and the data file being stored in a storage accessible by the server computer system, and wherein the dynamic generation comprises an aligned unbuffered direct read operation from the stored data file of a data size that is aligned to an integer multiple of the storage's block size; and
receive, from the server computer system, at least the portion of the at least one derived data file.
12. The system according to claim 11 , wherein the stored data file is stored in the storage among one or more other data files, the stored data file and the one or more other data files being of a same type, and wherein the at least one derived data file is not stored in the storage.
13. The system according to claim 12 , wherein the data file virtualization policy is associated in a one-to-many relationship with the stored data file and the one or more other data files.
14. The system according to claim 11 , wherein the client computer system is further configured to:
send, to the server computer system, a request for a list of supported derived data types and a list of supported parameters for each of the supported derived data types, wherein the parameters allow a client to control an output of the at least one derived data file;
receive, from the server computer system, the list of supported parameters; and
send an intent to request data of at least a subset of the supported derived data types.
15. The system according to claim 11 , wherein the data file virtualization policy comprises:
a name of the stored data file,
a data type of the stored data file,
a Boolean flag set to either True or False, True indicating an intent of the client computer system to request for derived data, and
a list of derived data types that the client computer system intends to retrieve.
16. The system according to claim 15 , wherein the at least one derived data file comprises a plurality of dynamically generated derived data files, a number of the plurality of derived data files being based on the intent of the client computer system to retrieve them.
17. The system according to claim 11 , wherein the stored data file comprises Wide-Area Motion Imagery (WAMI) data, a type of the stored data file being a WAMI Frame.
18. The system according to claim 11 , wherein the client computer system is further configured to:
send, to the server computer system, a request for a list of supported derived data types;
receive the list of supported derived data types; and
generate the data file virtualization policy using the list of supported derived data types.
19. The system according to claim 11 , wherein the at least one derived data file is listed in a directory listing generated by the server computer system, and
wherein the directory listing is accessible to the client computer system such that the at least one derived data file is presented to the client computer system as if the at least one derived data file is stored in the storage.
20. The system according to claim 19 , wherein the client computer system is further configured to:
send, to the server computer system, an HTTP 1.1 WebDAV PROPFIND request for the directory listing.