Distributed file systems on content delivery networks
A method, article of manufacture, and apparatus for accessing a distributed file system using a content delivery network is discussed. An IO request for an object is received at a metadata server comprising metadata mapping a plurality of objects to an object store. An object identifier associated with the object and a CDN domain associated with the object store are determined. A URL is generated from the object identifier and the CDN domain. The URL is then returned to the client.
1. A method for accessing data using a content delivery network (CDN) that is associated with object stores configured to store the data, the CDN including a plurality of servers and a plurality of datacenters, the method comprising:
receiving a request associated with an object at a metadata server in a communication from a client, wherein the metadata server comprises metadata mapping a plurality of objects to an object store;
determining an object identifier associated with the requested object and a CDN domain;
generating a URL to the CDN domain from the object identifier and the CDN domain;
returning the URL to the client; and
communicating with a server of the CDN domain to perform an IO request using the URL generated from the object identifier and the CDN domain, wherein the server of the CDN domain serves as a cache and a proxy to the object store and performs the IO request to the object store when the requested object is not cached at the server of the CDN domain.
2. The method of claim 1 , wherein the server of the CDN domain is associated with a specific object store, further comprising requesting the requested object, by the server of the CDN domain, from the specific object store when the requested object is not cached at the server of the CDN domain.
3. The method of claim 1 , wherein communicating the CDN domain comprises communicating with the server of the CDN domain, wherein the server of the CDN domain performs the request without accessing the object store when the requested object is cached by the server and wherein the server accesses the object store for the requested object when the requested object is not cached by the server.
4. The method of claim 1 , wherein the server is configured to act as a proxy and a cache for the requested object.
5. The method of claim 1 , further comprising forwarding the IO request to a remote object store when the requested object is not cached at the server of the CDN domain.
6. The method of claim 1 , further comprising returning the requested to the client through the server of the CDN domain.
7. The method of claim 1 , further comprising determining an optimal object store for the requested object.
8. The method of claim 7 , further comprising storing the requested object at the optimal object store associated with the server of the CDN domain when the requested object was not previously stored at the optimal object store.
9. The method of claim 1 , further comprising caching the requested object at the server of the CDN domain.
10. The method of claim 1 , further comprising writing an object to the CDN through the server of the CDN network, wherein the server of the CDN network caches the object being written.
11. The method of claim 1 , wherein the object store appears as a single object store but includes a plurality of physically separated datacenters.
12. A non-transitory computer readable medium comprising computer executable instructions for accessing data using a content delivery network (CDN) that is associated with object stores configured to store the data, the CDN including a plurality of servers and a plurality of datacenters, the instructions comprising:
receiving a request associated with an object at a metadata server in a communication from a client, wherein the metadata server comprises metadata mapping a plurality of objects to an object store;
determining an object identifier associated with the requested object and a CDN domain;
generating a URL to the CDN domain from the object identifier and the CDN domain;
returning the URL to the client; and
communicating with a server of the CDN domain to perform an IO request using the URL generated from the object identifier and the CDN domain, wherein the server of the CDN domain serves as a cache and a proxy to the object store and performs the IO request to the object store when the requested object is not cached at the server of the CDN domain.
13. The non-transitory computer readable medium of claim 12 , wherein the at least some of the plurality of servers of CDN domain is associated with a with a specific object store.
14. The non-transitory computer readable medium of claim 12 , wherein the server of the CDN domain is associated with a specific object store, further comprising requesting the requested object, by the server of the CDN domain, from the specific object store when the requested object is not cached at the server of the CDN domain.
15. The non-transitory computer readable medium of claim 12 , wherein the specific object store is determined to be an optimal object store for a client requesting the requested object.
16. The non-transitory computer readable medium of claim 12 , wherein communicating the CDN domain comprises communicating with the server of the CDN domain, wherein the server of the CDN domain performs the request without accessing the object store when the requested object is cached by the server and wherein the server accesses the object store for the requested object when the requested object is not cached by the server.
17. The non-transitory computer readable medium of claim 12 , wherein the server is configured to act as a proxy and a cache for the requested object.
18. The non-transitory computer readable medium of claim 12 , the instructions further comprising forwarding the IO request to a remote object store when the requested object is not cached at the server of the CDN domain, and returning the requested to the client through the server of the CDN domain.
19. The non-transitory computer readable medium of claim 12 , the instructions further comprising determining an optimal object store for the requested object, caching the requested object at the server of the CDN domain, storing the requested object at the optimal object store associated with the server of the CDN domain when the requested object was not previously stored at the optimal object store.
20. The non-transitory computer readable medium of claim 12 , further comprising writing an object to the CDN through the server of the CDN network, wherein the server of the CDN network caches the object being written.