General machine learning model, and model file generation and parsing method
Disclosed are a general-purpose machine learning model generation method and apparatus, and a computer device and a storage medium. The method comprises: acquiring task parameters of a machine learning task (S 1201 ), performing classification processing on the task parameters to obtain task instructions and model parameters (S 1202 ), aggregating the task instructions and the model parameters according to a data type to obtain stack data and heap data (S 1203 ), and integrating the stack data and the heap data to obtain a general-purpose machine learning model (S 1204 ). By means of the method, compiled results of a corresponding general-purpose model in the running of an algorithm can be directly executed, which avoids repetitive compilation, thus greatly improving the efficiency of machine learning algorithm implementation and shortening the time from compilation to obtaining execution results.
1. A method for parsing a general-purpose machine learning secondary model file, wherein the general-purpose machine learning secondary model file comprises a general-purpose machine learning model and a secondary model directory, and the method comprises:
obtaining a general-purpose machine learning model file, wherein the obtaining the general-purpose machine learning model file includes:
obtaining an identification code of the general-purpose machine learning model file,
detecting whether the identification code complies with a preset rule, and
based on a detection that the identification code complies with the preset rule, reading a secondary model directory in the general-purpose machine learning model file,
wherein the reading the secondary model directory includes:
obtaining a check code of the general-purpose machine learning model file, and
checking whether the check code is consistent with a preset standard code, and based on a checking result that the check code is inconsistent with the preset standard code, performing an error correction operation,
wherein the checking whether the check code is consistent with the preset standard code includes:
obtaining an error correction code,
performing an error correction operation on the general-purpose machine learning model file according to the error correction code to obtain an error-corrected model file,
checking whether a check code of the error-corrected general-purpose machine learning model file is consistent with the preset standard code, and
based on a checking result that the check code of the error-corrected general-purpose machine learning model file is consistent with the preset standard code, reading a secondary model directory in the general-purpose machine learning model file;
reading a secondary model directory in the general-purpose machine learning model file;
reading a target secondary model according to the secondary model directory; and
restoring the target secondary model to obtain a target general-purpose machine learning model.
2. The method of claim 1 , wherein the step of reading a target secondary model according to the secondary model directory includes:
obtaining a storage offset of a target secondary model in the general-purpose machine learning model file, and
reading the target general-purpose machine learning model according to the storage offset.
3. The method of claim 1 , comprising:
reading hardware parameter information in the general-purpose machine learning model; and
generating hardware matching information according to the hardware parameter information.
4. The method of claim 1 , comprising:
classifying and disassembling the general-purpose machine learning model to obtain stack area data and heap area data, and
computing the stack area data, the heap area data, and input data to obtain output data.
5. A device for parsing a general-purpose machine learning model file, wherein the general-purpose machine learning model file comprises a general-purpose machine learning model and a secondary model directory, and the device comprises:
a central processing unit (CPU); and a memory;
a file obtainer, a model distributor, a directory parser, and a model reader, wherein the directory parser is connected to the file obtainer, the model distributor, and the model reader, respectively;
wherein, the file obtainer is configured to obtain a general-purpose machine learning model file, wherein the file obtainer includes a file header checker; wherein the file header checker is configured to obtain an identification code of the general-purpose machine learning model file, and detect whether the identification code complies with a preset rule,
based on a detection result that the identification code does not comply with the preset rule, the file header checker is configured to read a model directory in the general-purpose machine learning model file,
the file header checker is further configured to obtain a check code of the general-purpose machine learning model file, and check whether the check code is consistent with a preset standard code, based on a checking result the check code is inconsistent with the preset standard code, the file header checker is configured to perform an error correction operation, wherein the file obtainer includes a file tailer corrector, wherein the file tailer corrector is configured to:
obtain an error correction code,
perform error correction on the general-purpose machine learning model file according to the error correction code to obtain an error-corrected general-purpose machine learning model file,
check whether a check code of the error-corrected general-purpose machine learning model file is consistent with the preset standard code, and
based on a checking result that the check code of the error-corrected general-purpose machine learning model file is consistent with the preset standard code, read a secondary model directory in the general-purpose machine learning model file;
the model distributor is configured to read a secondary model directory in the general-purpose machine learning model file, read a target secondary model according to the secondary model directory, and restore the target secondary model to obtain a target general-purpose machine learning model;
the directory parser is configured to read a model directory in the general-purpose machine learning model file; and
the model reader is configured to read a target general-purpose machine learning model according to the model directory.
6. The device of claim 5 , wherein the model reader is further configured to obtain an offset of a target secondary machine learning model in the general-purpose machine learning model file, and read the target general-purpose machine learning model according to the offset.
7. The device of claim 5 , wherein the device for parsing a general-purpose machine learning model file further includes a model distributor, wherein the model distributor is connected to the directory parser.
8. The device of claim 5 , wherein the device further includes a hardware matcher, wherein the hardware matcher is connected to the model reader, and the hardware matcher is configured to read hardware parameter information in the general-purpose machine learning model, and match corresponding hardware in the device pool according to the hardware parameter information.