Elastic scalability of a content transformation cluster
Content transformations can include transformation of content items in a CMS repository from a source format to a target format. Such transformations can be performed using a transformation node cluster having multiple nodes, each of which is configured for a specific content transformation type. Router nodes can receive requests for content items and route content items to transformation nodes having a proper content transformation type to either transform a requested content item to the target format or perform an intermediate transformation as part of a transformation chain. A transformation node cluster can be dynamically configurable based on estimates of expected loads for the various types of transformations. Systems, methods, and articles of manufacture are also described.
1. A computer-implemented method for scaling cloud-based content transformations, the method comprising:
estimating, by at least one computing system comprising computer hardware based on transformation usage data, an expected load for each of a plurality of transformation types for content item requests from one or more client machines relating to content items maintained in a content management system repository, each transformation type of the plurality of transformation types transforming a first content format to a second content format differing from the first content format, the transformation usage data comprising historical data pertaining to content transformation requests requested from the transformation node cluster;
configuring, by the at least one computing system, a transformation node cluster comprising a plurality of nodes, the configuring comprising designating each of two or more subsets of the plurality of nodes for executing one of the plurality of content transformation types, each of the two or more subsets having a designated number of nodes of the plurality of nodes, the number of nodes based on the estimated expected load for the one of the plurality of transformation types for which that subset is designated; and
assigning, by the at least one computing system, one or more router nodes within the plurality of nodes, wherein the one or more router nodes, as a result of the assigning perform operations comprising:
receiving a content item request from the one or more client machines,
identifying a required transformation type specified in the content item request, and
routing the content item request to an appropriate subset of the two or more subsets in the configured transformation node cluster, the content transformation for which the appropriate subset is designated matching the required transformation type specified in the content item request.
2. The computer-implemented method in accordance with claim 1 , further comprising re-configuring the transformation node cluster, the re-configuring comprising changing the designated number of nodes of the plurality of nodes for at least one of the two or more subsets based on a changed estimate of the expected load.
3. The computer-implemented method in accordance with claim 1 , wherein the operations performed by the one or more router nodes further comprise:
identifying a current content format of a content item referenced in a received request of the content item requests and a target content format of the content item specified in the received request; and
determining one or more required transformation types for the content item from the plurality of content transformation types.
4. The computer-implemented method in accordance with claim 3 , wherein the one or more required transformation types comprise a transformation chain comprising a plurality of transformation types, and wherein the appropriate subset has a first transformation type to transform the content item to a first intermediate content item having a first intermediate format.
5. The computer-implemented method in accordance with claim 4 , wherein the operations performed by the one or more router nodes further comprise:
routing the first intermediate content item to a second appropriate subset, a second content transformation for which the second appropriate subset is designated matching a second required transformation type of the one or more required transformation types for transforming the first intermediate content item to either a second intermediate content item having a second intermediate format or to a target content item having the target format.
6. The computer-implemented method in accordance with claim 1 , wherein at least one of the one or more router nodes is also included in one of the two or more subsets of the plurality of nodes.
7. The computer-implemented method in accordance with claim 1 , wherein the estimating further comprises use of at least one of a predictive model, machine learning, and a neural network to make predictions based on the historical data.
8. The computer-implemented method in accordance with claim 1 , wherein the stored data comprise stored callbacks provided by the one or more router nodes to client machines in response to previously completed content item requests.
9. The computer-implemented method in accordance with claim 8 , wherein the stored callbacks comprise at least one of references to requested content items, arrays of transformed content item references referencing intermediate content items created in transformation chains to produce transformed content items in response to the completed content item requests, listings of options specified in the content item requests, listings of operations performed to convert content items to target content formats, and identifiers for transformation nodes that participated in the transformation chains.
10. The computer-implemented method in accordance with claim 1 , wherein the transformation usage data further comprises at least one of a type and number of business process applications supported by the content management system repository, usage factors defined for content item transformations expected to be called by the business process applications, and an absolute or relative amount of mobile device usage supported by the content management system repository.
11. The computer-implemented method in accordance with claim 1 , wherein the estimating further comprises acting on at least one human input parameter relating to expected transformation demands.
12. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
estimating, based on transformation usage data, an expected load for each of a plurality of transformation types for content item requests from one or more client machines relating to content items maintained in a content management system repository, each transformation type of the plurality of transformation types transforming a first content format to a second content format differing from the first content format, the transformation usage data comprising historical data pertaining to content transformation requests requested from the transformation node cluster;
configuring, by the at least one computing system, a transformation node cluster comprising a plurality of nodes, the configuring comprising designating each of two or more subsets of the plurality of nodes for executing one of the plurality of content transformation types, each of the two or more subsets having a designated number of nodes of the plurality of nodes, the number of nodes based on the estimated expected load for the one of the plurality of transformation types for which that subset is designated; and
assigning, by the at least one computing system, one or more router nodes within the plurality of nodes, wherein the one or more router nodes, as a result of the assigning perform operations comprising:
receiving a content item request from the one or more client machines,
identifying a required transformation type specified in the content item request, and
routing the content item request to an appropriate subset of the two or more subsets in the configured transformation node cluster, the content transformation for which the appropriate subset is designated matching the required transformation type specified in the content item request.
13. The computer program product in accordance with claim 12 , wherein the operations further comprise re-configuring the transformation node cluster, the re-configuring comprising changing the designated number of nodes of the plurality of nodes for at least one of the two or more subsets based on a changed estimate of the expected load.
14. The computer program product in accordance with claim 12 , wherein the stored data comprise stored callbacks provided by the one or more router nodes to client machines in response to previously completed content item requests.
15. The computer program product in accordance with claim 14 , wherein the stored callbacks comprise at least one of references to requested content items, arrays of transformed content item references referencing intermediate content items created in transformation chains to produce transformed content items in response to the completed content item requests, listings of options specified in the content item requests, listings of operations performed to convert content items to target content formats, and identifiers for transformation nodes that participated in the transformation chains.
16. The computer program product in accordance with claim 12 , wherein the transformation usage data further comprises at least one of a type and number of business process applications supported by the content management system repository, usage factors defined for content item transformations expected to be called by the business process applications, and an absolute or relative amount of mobile device usage supported by the content management system repository.
17. A system comprising:
computer hardware configured to perform operations comprising:
estimating, based on transformation usage data, an expected load for each of a plurality of transformation types for content item requests from one or more client machines relating to content items maintained in a content management system repository, each transformation type of the plurality of transformation types transforming a first content format to a second content format differing from the first content format, the transformation usage data comprising historical data pertaining to content transformation requests requested from the transformation node cluster;
configuring, by the at least one computing system, a transformation node cluster comprising a plurality of nodes, the configuring comprising designating each of two or more subsets of the plurality of nodes for executing one of the plurality of content transformation types, each of the two or more subsets having a designated number of nodes of the plurality of nodes, the number of nodes based on the estimated expected load for the one of the plurality of transformation types for which that subset is designated; and
assigning, by the at least one computing system, one or more router nodes within the plurality of nodes, wherein the one or more router nodes, as a result of the assigning perform operations comprising:
receiving a content item request from the one or more client machines,
identifying a required transformation type specified in the content item request, and
routing the content item request to an appropriate subset of the two or more subsets in the configured transformation node cluster, the content transformation for which the appropriate subset is designated matching the required transformation type specified in the content item request.
18. A system as in claim 17 , wherein the computer hardware comprises
a programmable processor; and
a machine-readable medium storing instructions that, when executed by the processor, cause the programmable processor to perform at least some of the operations.