Loading models on nodes having multiple model service frameworks
This disclosure relates to model loading. In one aspect, a method includes determining, based on a preset execution script and resource information of multiple execution nodes, loading-tasks corresponding to the execution nodes. Each execution node is deployed on a corresponding cluster node. Loading requests are sent to the execution nodes, thereby causing the execution nodes to start execution processes based on the corresponding loading requests. The execution processes start multiple model service frameworks on each cluster node. Multiple models are loaded onto each of the model service frameworks. Each loading request includes loading-tasks corresponding to the execution node to which the loading request was sent. The execution processes include a respective execution process for each model service framework.
1. A computer-implemented method, comprising:
receiving, by an execution node from a control node, a loading request comprising loading tasks corresponding to execution nodes, wherein the loading tasks are determined by the control node based on a preset execution script and resource information for each of a plurality of execution nodes, and wherein different execution nodes are deployed on different cluster nodes; and
starting, by the execution node, multiple execution processes based on the loading request, wherein the multiple execution processes start multiple model service frameworks and multiple models are loaded onto each of the model service frameworks.
2. The computer-implemented method of claim 1 , wherein the execution processes are in one-to-one correspondence with the model service frameworks.
3. The computer-implemented method of claim 1 , further comprising:
detecting that a target execution process has been lost; and
in response to detecting that the target execution process has been lost, re-establishing the target execution process.
4. The computer-implemented method of claim 1 , wherein:
the loading request further comprises a declaration about an execution process in the preset execution script; and
starting the multiple execution processes based on the loading request comprises starting the multiple execution processes based on the declaration about the execution process in the preset execution script.
5. The computer-implemented method of claim 1 , wherein:
the loading request further comprises a declaration about a model service framework in the preset execution script; and
starting multiple model service frameworks by the multiple execution processes comprises starting the multiple model service frameworks based on the declaration about the model service framework in the preset execution script.
6. The computer-implemented method of claim 1 , wherein each cluster node comprises at least one of a physical machine, a virtual machine, or a container.
7. The computer-implemented method of claim 1 , wherein the resource information for each particular execution node comprises a least one of a quantity of central processing unit (CPU) cores of a particular cluster node on which the particular execution node is located or remaining memory capacity of the particular cluster node on which the particular execution node is located.
8. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
receiving, by an execution node from a control node, a loading request comprising loading tasks corresponding to execution nodes, wherein the loading tasks are determined by the control node based on a preset execution script and resource information for each of a plurality of execution nodes, and wherein different execution nodes are deployed on different cluster nodes; and
starting, by the execution node, multiple execution processes based on the loading request, wherein the multiple execution processes start multiple model service frameworks and multiple models are loaded onto each of the model service frameworks.
9. The non-transitory, computer-readable medium of claim 8 ,
wherein the execution processes are in one-to-one correspondence with the model service frameworks.
10. The non-transitory, computer-readable medium of claim 8 , wherein the operations comprise:
detecting that a target execution process has been lost; and
in response to detecting that the target execution process has been lost, re-establishing the target execution process.
11. The non-transitory, computer-readable medium of claim 8 , wherein:
the loading request further comprises a declaration about an execution process in the preset execution script; and
starting the multiple execution processes based on the loading request comprises starting the multiple execution processes based on the declaration about the execution process in the preset execution script.
12. The non-transitory, computer-readable medium of claim 8 , wherein:
the loading request further comprises a declaration about a model service framework in the preset execution script; and
starting multiple model service frameworks by the multiple execution processes comprises starting the multiple model service frameworks based on the declaration about the model service framework in the preset execution script.
13. The non-transitory, computer-readable medium of claim 8 , wherein each cluster node comprises at least one of a physical machine, a virtual machine, or a container.
14. A computer-implemented system, comprising:
one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:
receiving, by an execution node from a control node, a loading request comprising loading tasks corresponding to execution nodes, wherein the loading tasks are determined by the control node based on a preset execution script and resource information for each of a plurality of execution nodes, and wherein different execution nodes are deployed on different cluster nodes; and
starting, by the execution node, multiple execution processes based on the loading request, wherein the multiple execution processes start multiple model service frameworks and multiple models are loaded onto each of the model service frameworks.
15. The computer-implemented system of claim 14 , wherein the execution processes are in one-to-one correspondence with the model service frameworks.
16. The computer-implemented system of claim 14 , wherein the operations comprise:
detecting that a target execution process has been lost; and
in response to detecting that the target execution process has been lost, re-establishing the target execution process.
17. The computer-implemented system of claim 14 , wherein:
the loading request further comprises a declaration about an execution process in the preset execution script; and
starting the multiple execution processes based on the loading request comprises starting the multiple execution processes based on the declaration about the execution process in the preset execution script.
18. The computer-implemented system of claim 14 , wherein:
the loading request further comprises a declaration about a model service framework in the preset execution script; and
starting multiple model service frameworks by the multiple execution processes comprises starting the multiple model service frameworks based on the declaration about the model service framework in the preset execution script.
19. The computer-implemented system of claim 14 , wherein each cluster node comprises at least one of a physical machine, a virtual machine, or a container.
20. The computer-implemented system of claim 14 , wherein the resource information for each particular execution node comprises a least one of a quantity of CPU cores of a particular cluster node on which the particular execution node is located or remaining memory capacity of the particular cluster node on which the particular execution node is located.