Granted Patent B2

US 12,613,752 · App. 17/819,345 · Granted Apr 28, 2026

Inventors: Pinyu Xiao (Guangzhou City, CN); Lurong Pan (Vestavia Hill, AL)

G06F9/5072G06F9/5083G06F2209/5022

Abstract

Claims (55)

1 . A data processing method based on a cloud platform comprising:

obtaining task processing requests submitted by several target users through a distributed system, wherein the task processing requests are requests for processing scientific computing tasks;

determining whether a number of the obtained task processing requests reaches a preset capacity expansion threshold, and if so, generate a capacity expansion request of workload;

performing capacity expansion processing on a computing node according to the capacity expansion request;

redeploying the workload based on the expanded computing node after the capacity expansion processing, so as to execute the scientific computing task based on the redeployed workload; and

setting the workload as a set of metadata, wherein the metadata includes the image address of a docker container to be run and the computing resource requirements required to run the workload.

2 . The data processing method based on a cloud platform according to claim 1 , wherein when obtaining task processing requests submitted by several target users, the data processing method further includes:

generating task events corresponding to the task processing requests; and

determining whether the number of obtained task processing requests reaches the preset capacity expansion threshold, including determining whether the number of generated task events reaches the preset capacity expansion threshold.

3 . The data processing method based on a cloud platform according to claim 2 , wherein determining whether the number of generated task events reaches the preset capacity expansion threshold is determined by monitoring the generated task events.

4 . The data processing method based on a cloud platform according to claim 2 , further comprises:

generating a first working message corresponding to the task event is generated based on the task event; and

determining whether the number of generated task events reaches the preset capacity expansion threshold includes determining whether the number of generated first work messages reaches the preset capacity expansion threshold.

5 . The data processing method based on a cloud platform according to claim 1 , wherein when the task processing request is obtained, the data processing method further comprises:

generating a second working message corresponding to the task processing request; and

determining whether the number of acquired task processing requests reaches the preset capacity expansion threshold includes determining whether the number of generated second working messages reaches the preset capacity expansion threshold.

6 . The data processing method based on a cloud platform according to claim 1 , further comprises:

according to an execution result of the scientific computing task executed by the workload in operation, the task state of the scientific computing task is changed in real time, or the scientific computing task is created as multiple subtasks.

7 . The data processing method based on a cloud platform according to claim 1 , further comprises:

determining whether the workload is running completely, and if so, the computing node that deploys the workload will enter an idle state.

8 . The data processing method based on a cloud platform according to claim 7 , wherein after determining that the computing node that deploys the workload enters an idle state, the computing node that deploy the workload is shut down and recycled after it is determined that the computing node deploying the workload is in an idle state, the data processing method further comprises:

shutting down and recycling the computing node deploying the workload.

9 . The data processing method based on a cloud platform according to claim 1 , wherein obtaining task processing requests submitted by several target users through the distributed system comprises:

when several target users submit task processing requests to a distributed database, obtaining the task processing requests based on data changes of the distributed database.

10 . A data processing system based on a cloud platform comprising:

at least one processor implementing:

a distributed task scheduling system;

a cloud elastic scaling system; and

a task execution system,

wherein the distributed task scheduling system comprises a task module,

wherein the cloud elastic scaling system comprises a workload scaling module and a computing node scaling module, and

wherein the task execution system comprises several computing nodes; and

wherein the task module is used for a target user to submit a task processing request of a scientific computing task, and determine whether the number of obtained task processing requests reaches a preset capacity expansion threshold, and if so, generate a workload capacity expansion request;

wherein the workload scaling module is used to trigger the computing node scaling module to expand the computing node according to the capacity expansion request; and

wherein the computing node scaling module is used to expand the computing node according to the capacity expansion request;

redeploy the workload based on the expanded computing nodes after the capacity expansion processing, so as to execute the scientific computing task based on the redeployed workload and

set the workload as a set of metadata, wherein the metadata includes the image address of a docker container to be run and the computing resource requirements required to run the workload.

11 . The data processing system based on a cloud platform according to claim 10 , wherein the distributed task scheduling system further comprises:

an event module, wherein

the event module is used for monitoring task events and determining whether the number of task events reaches a preset capacity expansion threshold, and if so, triggering the workload scaling module to expand the computing nodes, and

wherein the task events are events corresponding to task processing requests.

12 . The data processing system based on a cloud platform according to claim 10 , wherein the distributed task scheduling system further comprises:

a work message module, wherein the work message module is used for monitoring the length change of a message queue to determine whether to trigger the workload scaling module to expand the computing node, and

wherein the message queue is used for storing work information corresponding to the task processing request.

13 . The data processing system based on a cloud platform according to claim 10 , wherein the computing node scaling module is further used for detecting whether a target computing node enters an idle state, and/or shutting down and recycling the idle target computing node.

14 . The data processing system based on a cloud platform according to claim 10 , wherein the distributed task scheduling system is further used for changing a task state corresponding to the scientific computing task or creating the scientific computing task into multiple subtasks according to an execution result of the scientific computing task executed by the workload.

15 . The data processing system based on a cloud platform according to claim 10 , wherein the task module further comprises:

a distributed database; and

a data change capture unit, wherein

the distributed database is used for several target users to submit task processing requests for scientific computing tasks, and

wherein the data change capture unit acquires the task processing request based on the data change of the distributed database.

16 . An electronic device, comprising:

at least one processor; and

a memory communicatively connected with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and

wherein the instructions are executed by the at least one processor to enable the at least one processor to execute the data processing method according to any one of claims 1-6, or 7-9 .

Assignments (4)

Continuity (1)

References Cited (43)