Highly available private cloud service

1. One or more non-transitory machine-readable storage mediums storing one or more sequences of instructions, which when executed by one or more processors, cause:

a cloud service provider maintaining a first cluster of servers which stores a set of user interfaces, wherein each node of the first cluster comprises a first distributed management layer that manages, monitors and, administers corrections to nodes of the first cluster and facilitates communication with nodes of a second cluster of servers,

a cloud service consumer maintaining the second cluster of servers which store sets of user data for customers of said cloud service consumer, wherein each node of the second cluster comprises a second distributed management layer that manages, monitors, administers corrections to nodes of the second cluster and facilitates communication with nodes of the first cluster, and

upon a first node in the first cluster receiving a request from an end-user of a cloud-based service, the first node (a) communicating via HTTPS with the second distributed management layer to obtain a particular set of user data required in responding to said request, (b) preparing a response that comprises a particular user interface from said set of user interfaces and said particular set of user data, and (c) transmitting said response to said end-user.

2. The one or more non-transitory computer-readable storage mediums of claim 1 , wherein the cloud service consumer is a first cloud service consumer in a plurality of cloud service consumers that each maintain a separate cluster of servers, and wherein each separate cluster of servers, maintained by a particular cloud service consumer, comprises a separate distributed management layer that manages, monitors, administers corrections to nodes of the separate cluster of servers maintained by a particular cloud service consumer and facilitates communication with nodes of the first cluster.

3. The one or more non-transitory computer-readable storage mediums of claim 1 , wherein the second cluster of servers communicates with the first cluster of servers by performing:

a first software process, executing on said second cluster of servers, issues a request to said first cluster of servers for an authentication token and an URL address over which to communicate with said second cluster of servers;

the first software process obtaining an authentication token and a URL address from said first cluster of servers, wherein said URL address and said authentication token are dynamically determined in response to said first cluster of servers receiving said request from said first software process; and

a second software process, executing on said second cluster of servers, communicating with said first cluster of servers using said URL address and said authentication token.

4. The one or more non-transitory computer-readable storage mediums of claim 1 , wherein the first cluster of servers communicates a communication pertaining to a particular account with the second cluster of servers by performing:

at the first cluster of servers, identifying an open HTTPS response request issued from a software process, executing on said second cluster of servers, associated with said particular account; and

sending said communication to said software process using said HTTPS response request.

5. The one or more non-transitory computer-readable storage mediums of claim 1 , wherein the distributed management layer of the first cluster of servers is configured to ensure at least a specified set of services are executing in the first cluster of servers.

6. The one or more non-transitory computer-readable storage mediums of claim 1 , wherein the distributed management layer of the first cluster of servers performs:

monitor execution of a least a specified set of services executing in the first cluster of servers;

probing each of the specified set of services to detect an operational problem therein; and

upon detecting an operational problem in a particular service of said specified set of services, automatically resolving the operational problem to ensure the particular service is executing in accordance with a previously established standard of operation.

7. The one or more non-transitory computer-readable storage mediums of claim 1 , wherein said distributed management layer of the first cluster of servers comprises one or more services that each may execute on one or more nodes of the first cluster, and wherein said distributed management layer continuously monitors the one or more nodes of the first cluster to identify upon which particular node, of the one or more nodes of the first cluster, each of the one or more services should execute.

8. A system, comprising:

one or more processors; and

one or more non-transitory computer-readable storage mediums storing one or more sequences of instructions, which when executed, cause:

a cloud service provider maintaining a first cluster of servers which stores a set of user interfaces, wherein each node of the first cluster comprises a first distributed management layer that manages, monitors and, administers corrections to nodes of the first cluster and facilitates communication with nodes of a second cluster of servers,

a cloud service consumer maintaining the second cluster of servers which store sets of user data for customers of said cloud service consumer, wherein each node of the second cluster comprises a second distributed management layer that manages, monitors, administers corrections to nodes of the second cluster and facilitates communication with nodes of the first cluster, and

upon a first node in the first cluster receiving a request from an end-user of a cloud-based service, the first node (a) communicating via HTTPS with the second distributed management layer to obtain a particular set of user data required in responding to said request, (b) preparing a response that comprises a particular user interface from said set of user interfaces and said particular set of user data, and (c) transmitting said response to said end-user.

9. The system of claim 8 , wherein the cloud service consumer is a first cloud service consumer in a plurality of cloud service consumers that each maintain a separate cluster of servers, and wherein each separate cluster of servers, maintained by a particular cloud service consumer, comprises a separate distributed management layer that manages, monitors, administers corrections to nodes of the separate cluster of servers maintained by a particular cloud service consumer and facilitates communication with nodes of the first cluster.

10. The system of claim 8 , wherein the second cluster of servers communicates with the first cluster of servers by performing:

a first software process, executing on said second cluster of servers, issues a request to said first cluster of servers for an authentication token and an URL address over which to communicate with said second cluster of servers;

the first software process obtaining an authentication token and a URL address from said first cluster of servers, wherein said URL address and said authentication token are dynamically determined in response to said first cluster of servers receiving said request from said first software process; and

a second software process, executing on said second cluster of servers, communicating with said first cluster of servers using said URL address and said authentication token.

11. The system of claim 8 , wherein the first cluster of servers communicates a communication pertaining to a particular account with the second cluster of servers by performing:

at the first cluster of servers, identifying an open HTTPS response request issued from a software process, executing on said second cluster of servers, associated with said particular account; and

sending said communication to said software process using said HTTPS response request.

12. The system of claim 8 , wherein the distributed management layer of the first cluster of servers is configured to ensure at least a specified set of services are executing in the first cluster of servers.

13. The system of claim 8 , wherein the distributed management layer of the first cluster of servers performs:

monitor execution of a least a specified set of services executing in the first cluster of servers;

probing each of the specified set of services to detect an operational problem therein; and

upon detecting an operational problem in a particular service of said specified set of services, automatically resolving the operational problem to ensure the particular service is executing in accordance with a previously established standard of operation.

14. The system of claim 8 , wherein said distributed management layer of the first cluster of servers comprises one or more services that each may execute on one or more nodes of the first cluster, and wherein said distributed management layer continuously monitors the one or more nodes of the first cluster to identify upon which particular node, of the one or more nodes of the first cluster, each of the one or more services should execute.

15. A method, comprising:

a cloud service provider maintaining a first cluster of servers which stores a set of user interfaces, wherein each node of the first cluster comprises a first distributed management layer that manages, monitors and, administers corrections to nodes of the first cluster and facilitates communication with nodes of a second cluster of servers,

a cloud service consumer maintaining the second cluster of servers which store sets of user data for customers of said cloud service consumer, wherein each node of the second cluster comprises a second distributed management layer that manages, monitors, administers corrections to nodes of the second cluster and facilitates communication with nodes of the first cluster, and

upon a first node in the first cluster receiving a request from an end-user of a cloud-based service, the first node (a) communicating via HTTPS with the second distributed management layer to obtain a particular set of user data required in responding to said request, (b) preparing a response that comprises a particular user interface from said set of user interfaces and said particular set of user data, and (c) transmitting said response to said end-user.

16. The method of claim 15 , wherein the cloud service consumer is a first cloud service consumer in a plurality of cloud service consumers that each maintain a separate cluster of servers, and wherein each separate cluster of servers, maintained by a particular cloud service consumer, comprises a separate distributed management layer that manages, monitors, administers corrections to nodes of the separate cluster of servers maintained by a particular cloud service consumer and facilitates communication with nodes of the first cluster.

17. The method of claim 15 , wherein the second cluster of servers communicates with the first cluster of servers by performing:

a first software process, executing on said second cluster of servers, issues a request to said first cluster of servers for an authentication token and an URL address over which to communicate with said second cluster of servers;

the first software process obtaining an authentication token and a URL address from said first cluster of servers, wherein said URL address and said authentication token are dynamically determined in response to said first cluster of servers receiving said request from said first software process; and

a second software process, executing on said second cluster of servers, communicating with said first cluster of servers using said URL address and said authentication token.

18. The method of claim 15 , wherein the first cluster of servers communicates a communication pertaining to a particular account with the second cluster of servers by performing:

at the first cluster of servers, identifying an open HTTPS response request issued from a software process, executing on said second cluster of servers, associated with said particular account; and

sending said communication to said software process using said HTTPS response request.

19. The method of claim 15 , wherein the distributed management layer of the first cluster of servers is configured to ensure at least a specified set of services are executing in the first cluster of servers.

20. The method of claim 15 , wherein the distributed management layer of the first cluster of servers performs:

monitor execution of a least a specified set of services executing in the first cluster of servers;

probing each of the specified set of services to detect an operational problem therein; and

upon detecting an operational problem in a particular service of said specified set of services, automatically resolving the operational problem to ensure the particular service is executing in accordance with a previously established standard of operation.

21. The method of claim 15 , wherein said distributed management layer of the first cluster of servers comprises one or more services that each may execute on one or more nodes of the first cluster, and wherein said distributed management layer continuously monitors the one or more nodes of the first cluster to identify upon which particular node, of the one or more nodes of the first cluster, each of the one or more services should execute.