Optimization of path selection for transfers of files
View Patent ↗A system may include a file transfer manager that determines a file for transfer from a source location to a target location, the file being associated with file metadata characterizing the file, and with an organization. The file transfer manager may include an orchestrator that determines at least two transfer paths for the transfer, including at least a first transfer path utilizing a private wide area network (WAN) of the organization and a second transfer path utilizing a publicly available data hosting service, access transfer metadata characterizing the at least two transfer paths, and access organizational metadata characterizing organizational transfer path usage factors. The file transfer manager also may include a heuristics engine configured to execute path decision logic using the file metadata, the transfer metadata, and the organizational metadata, to select a selected transfer path from the at least two transfer paths.
1. A system for network path selection for transfers of files within a computing network, including instructions stored on a non-transitory computer readable storage medium and executable by at least one processor, the system comprising:
a file transfer manager implemented at a first data center, wherein the file transfer manager is configured to cause the at least one processor to transfer a file from a source computing device to a target computing device, the source computing device being communicatively coupled to the first data center, the target computing device being communicatively coupled to a second data center, wherein the file is associated with file metadata, and with an organization, the file metadata including one or more checksum values and information about the file, the file metadata including security-related metadata and transfer latency metadata,
wherein the file transfer manager is configured to cause the at least one processor to:
intercept, via an application programming interface (API), an operating system call issued at the source computing device, the intercepted operating system call including a file transfer request for transferring the file;
determine whether or not the file already exists at the target computing device based on the one or more checksum values;
compute network metadata about the network conditions of a private wide area network (WAN) and of a plurality of publicly available data hosting services in response to the determination that the file does not already exist at the target computing device, the network metadata including information representing available network bandwidth on the private WAN and available network bandwidth on the plurality of publicly available data hosting services, wherein the private WAN is a computer network administered by the organization, and each of the plurality of publicly available data hosting services is a network that is not administered by the organization; and
programmatically select either the private WAN or one of the plurality of publicly available data hosting services for transferring the file by executing path decision logic, wherein the path decision logic includes decision criteria specifying weights applied to the security-related metadata and the transfer latency metadata, wherein the security-related metadata is associated with a higher weight than the transfer latency metadata,
wherein, in response to the selection of one of the plurality of publicly available data hosting services for transferring the file, the file transfer manager is configured to cause the at least one processor to:
upload the file to the selected publicly available data hosting service;
transfer configuration data to the second data center, the configuration data including information for accessing the selected publicly available data hosting service, the configuration data including a digital certificate;
instruct the second data center to download the file from the selected publicly available data hosting service using the configuration data; and
instruct the second data center to transfer the file from the second data center to the target computing device.
2. The system of claim 1 , wherein the file transfer manager is configured to cause the at least one processor to instruct the selected publicly available data hosting service to select one of a plurality of data centers of the selected publicly available data hosting service based on the plurality of data centers' proximity to the second data center, wherein the file is uploaded to the selected data center of the selected publicly available data hosting service.
3. The system of claim 1 , wherein the file metadata includes a description of contents of the file, a size of the file, a usage or access history of the file, and an owner of the file.
4. The system of claim 1 , wherein the file metadata includes specified security levels, quality of service levels, priority levels related to individual members or classes of member of the organization associated with the file, and at least one constraint on file transfers.
5. The system of claim 1 , wherein the file transfer manager is configured to cause the at least one processor to update at least a portion of the file metadata.
6. The system of claim 1 , wherein the configuration data includes credential information associated with the organization for use in accessing an account provided by the selected publicly available data hosting service.
7. A method comprising:
transferring, by a file transfer manager implemented at a first data center, a file from a source computing device to a target computing device, the source computing device being communicatively coupled to the first data center, the target computing device being communicatively coupled to a second data center, wherein the file is associated with file metadata, and with an organization, the file metadata including one or more checksum values and information about the file, the file metadata including security-related metadata and transfer latency metadata, wherein the transferring includes:
intercepting, via an application programming interface (API), an operating system call issued at the source computing device, the intercepted operating system call including a file transfer request for transferring the file;
determining whether or not the file already exists at the target computing device based on the one or more checksum values;
computing network metadata about the network conditions of a private wide area network (WAN) and of a plurality of publicly available data hosting services in response to the determination that the file does not already exist at the target computing device, the network metadata including information representing available network bandwidth on the private WAN and available network bandwidth on the plurality of publicly available data hosting services, wherein the private WAN is a computer network administered by the organization, and each of the plurality of publicly available data hosting services is a network that is not administered by the organization;
programmatically selecting the private WAN or one of the plurality of publicly available data hosting services for transferring the file by executing path decision logic, wherein the path decision logic includes decision criteria specifying weights applied to the security-related metadata and the transfer latency metadata, wherein the security-related metadata is associated with a higher weight than the transfer latency metadata; and
wherein, in response to the selection of one of the plurality of publicly available data hosting services for transferring the file, the method further includes the file transfer manager for:
uploading the file to the selected publicly available data hosting service;
transferring configuration data to the second data center, the configuration data including a digital certificate, wherein the configuration data including information for accessing the selected publicly available data hosting service;
instructing the second data center to download the file from the selected publicly available data hosting service using the configuration data; and
instructing the second data center to transfer the file from the second data center to the target computing device.
8. The method of claim 7 , further comprising:
instructing the selected publicly available data hosting service to select one of a plurality of data centers of the selected publicly available data hosting service based on the plurality of data centers' proximity to the second data center, wherein the file is uploaded to the selected data center of the selected publicly available data hosting service.
9. A computer program product including instructions recorded on a non-transitory computer readable storage medium and configured to cause at least one processor to:
transfer, by a file transfer manager implemented at a first data center, a file from a source computing device to a target computing device, the source computing device being communicatively coupled to the first data center, the target computing device being communicatively coupled to a second data center, wherein the file is associated with file metadata, and with an organization, the file metadata including one or more checksum values and information about the file, the file metadata including security-related metadata and transfer latency metadata, wherein the instructions to transfer include:
intercept, via an application programming interface (API), an operating system call issued at the source computing device, the intercepted operating system call including a file transfer request for transferring the file;
determine whether or not the file already exists at the target computing device based on the one or more checksum values;
compute network metadata about network conditions of a private wide area network (WAN) and of a plurality of publicly available data hosting services in response to the determination that the file does not already exist at the target computing device, the network metadata including information representing available network bandwidth on the private WAN and available network bandwidth on the plurality of publicly available data hosting services, wherein the private WAN is a computer network administered by the organization, and each of the plurality of publicly available data hosting services is a network that is not administered by the organization;
programmatically select the private WAN or one of the plurality of publicly available data hosting services for transferring the file by executing path decision logic, wherein the path decision logic includes decision criteria specifying weights applied to the security-related metadata and the transfer latency metadata, wherein the security-related metadata is associated with a higher weight than the transfer latency metadata; and
wherein, in response to the selection of one of the plurality of publicly available data hosting services for transferring the file, the file transfer manager further configured to:
upload the file to the selected publicly available data hosting service,
transfer configuration data to the second data center, wherein the configuration data including information for accessing the selected publicly available data hosting service,
instruct the second data center to download the file from the selected publicly available data hosting service using the configuration data, and
instruct the second data center to transfer the file from the second data center to the target computing device, the configuration data including a digital certificate.
10. The computer program product of claim 9 , wherein the private WAN includes geographically-distributed data centers.
11. The computer program product of claim 9 , further comprising:
instruct the selected publicly available data hosting service to select one of a plurality of data centers of the selected publicly available data hosting service based on the plurality of data centers' proximity to the second data center, wherein the file is uploaded to the selected data center of the selected publicly available data hosting service.