Batch computing system and associated method

1 . A batch computing system, comprising:

a memory, configured to store a first shared model parameter set corresponding to a plurality of first tasks in common and a task-specific parameter set corresponding to each first task in the plurality of first tasks, wherein the plurality of first tasks are generated by fine tuning the first shared model;

a task manager, configured to access parameters stored in the memory; and

an inference computer, comprising:

a preprocessing module, configured to, in a service stage, receive a plurality of task requests, parse the plurality of task requests to derive a data length and a designated task for each task request, and enable the task manager to read a task-specific parameter set and a shared model parameter set of the designated task of each task request;

a batch assignment module, configured to assign a plurality of first task requests of the plurality of task requests that are corresponding to the first shared model to a plurality of computation batches; and

an inference execution module, configured to perform a common computation for a plurality of designated tasks of first task requests in each computation batch in batch according to the first shared model parameter set, and perform task-specific computation operations for the plurality of designated tasks of the first task requests in each computation batch,

wherein, the batch assignment module first assembles a plurality of first task requests of the plurality of first task requests corresponding to a same first task and having similar data lengths to a same sub-computation batch, and then assembles a plurality of sub-computation batches having similar data lengths and different designated tasks to a computation batch of the plurality of computation batches.

2 . The batch computing system of claim 1 , wherein the task manager comprises a registration module, configured to, in a registration stage, receive the plurality of first tasks, establish a plurality of first task identifications (IDs) of the plurality of first tasks, and use the plurality of first task IDs as indexes to record the first shared model and task-specific computations corresponding to the plurality of first tasks.

3 . The batch computing system of claim 2 , wherein the registration module is further configured to, in the registration stage, receive a plurality of second tasks generated by fine tuning a second shared model, establish a plurality of second task IDs of the plurality of second tasks, and use the plurality of second task IDs as indexes to record the second shared model and task-specific computations corresponding to the plurality of second tasks.

4 . The batch computing system of claim 3 , wherein the registration module is further configured to, in the registration stage, establish a model ID corresponding to the first shared model and a model ID corresponding to the second shared model, and record a model ID corresponding to each task ID according to a shared model corresponding to each task of the plurality of first tasks and the plurality of second tasks.

5 . The batch computing system of claim 3 , wherein the registration module is further configured to, in the registration stage, establish a plurality of specific computation IDs corresponding to a plurality of task-specific computations, and record a specific computation ID corresponding to each task ID according to a task-specific computation corresponding to each task of the plurality of first tasks and the plurality of second tasks.

6 . The batch computing system of claim 2 , wherein:

each task request of the plurality of task requests comprises a designated task ID of a designated task and an input data requesting an inference service.

7 . The batch computing system of claim 6 , wherein:

in the service stage, the preprocessing module transfers a plurality of task IDs of the plurality of task requests to the task manager; and

the task manager further comprises a load module, configured to, in the service stage, read a shared model parameter set and a task-specific parameter set of the designated task of each task request from the memory according to the plurality of task IDs.

8 . The batch computing system of claim 1 , wherein the inference execution module further comprises a plurality of specific computation modules, configured to perform a plurality of task-specific computation operations corresponding to the plurality of first tasks.

9 . An inference service method, comprising:

in a registration stage, storing a first shared model parameter set corresponding to a plurality of first tasks in common and a task-specific parameter set of each first task of the plurality of first tasks in a memory, wherein the plurality of first tasks are generated by fine tuning a first shared model;

in a service stage, receiving a plurality of task requests;

parsing the plurality of task requests to derive a data length and a designated task of each of the plurality of task requests;

reading a task-specific parameter set and a shared model parameter set of the designated task of each of the plurality of task requests from the memory;

assigning a plurality of first task requests of the plurality of task requests corresponding to the first shared model to a plurality of computation batches;

performing, in a manner of batch, a common computation of a plurality of designated tasks of first task requests in each of the plurality of computation batches according to the first shared model parameter set; and

performing a plurality of task-specific computation operations according to a plurality of task-specific parameter sets of the plurality of designated tasks of the first task requests in each of the plurality of computation batches,

wherein the step of assigning the plurality of first task requests of the plurality of task requests to the plurality of computation batches comprises:

assembling a plurality of first task requests of the plurality of first task requests that are corresponding to a same first task and having similar data lengths to a same sub-computation batch; and

assembling a plurality of sub-computation batches having similar data lengths and different designated tasks to a same computation batch of the plurality of computation batches.

10 . The method of claim 9 , further comprising:

in the registration stage, receiving the plurality of first tasks;

establishing a plurality of first task identifications (ID) of the plurality of first tasks; and

using the plurality of first task IDs as indexes to record a shared model and task-specific computations corresponding to the plurality of first tasks.

11 . The method of claim 10 , further comprising:

in the registration stage, receiving a plurality of second tasks generated by fine tuning a second shared model;

establishing a plurality of second task IDs of the plurality of second tasks; and

using the plurality of second task IDs as indexes to record a shared model and task-specific computations corresponding to the plurality of second tasks.

12 . The method of claim 11 , further comprising:

in the registration stage, establishing a model ID corresponding to the first shared model and a model ID corresponding to the second shared model; and

recording a model ID corresponding to each task ID according to a shared model corresponding to each task of the plurality of first tasks and the plurality of second tasks.

13 . The method of claim 11 , further comprising:

in the registration stage, establishing a plurality of specific computation IDs corresponding to a plurality of task-specific computations; and

recording a specific computation ID corresponding to each task ID according to a task-specific computation corresponding to each task of the plurality of first tasks and the plurality of second tasks.

14 . The method of claim 10 , wherein:

each task request of the plurality of task requests comprises a designated task ID of a designated task and an input data requesting an inference service.