General machine learning model, and model file generation and parsing method
Disclosed are a general machine learning model generation method and apparatus, and a computer device and a storage medium. The method comprises: acquiring task parameters of a machine learning task (S 1201 ); performing classification processing on the task parameters to obtain task instructions and model parameters (S 1202 ); aggregating the task instructions and the model parameters according to a data type to obtain stack data and heap data (S 1203 ); and integrating the stack data and the heap data to obtain a general machine learning model (S 1204 ). By means of the method, compiled results of a corresponding general model in the running of an algorithm can be directly executed, which avoids repetitive compilation, thus greatly improving the efficiency of machine learning algorithm implementation and shortening the time from compilation to obtaining execution results.
1. A method for parsing a general-purpose machine learning model file, wherein the general-purpose machine learning model file comprises a target general-purpose machine learning model and a model directory, and the method comprises:
obtaining a general-purpose machine learning model file;
reading a model directory in the general-purpose machine learning model file;
reading a target general-purpose machine learning model according to the model directory;
wherein the obtaining the general-purpose machine learning model file includes obtaining an identification code of the general-purpose machine learning model file from a file header checker,
detecting whether the identification code complies with a preset rule, and
based on a determination that the identification code complies with the preset rule, reading a model directory in the general-purpose machine learning model file by the file header checker,
wherein if the identification code complies with the preset rule, the reading a model directory in the general-purpose machine learning model file includes:
obtaining a check code of the general-purpose machine learning model file,
checking whether the check code is consistent with a preset standard code, and based on a checking result that the check code is inconsistent with the preset standard code, performing an error correction operation,
wherein the checking whether the check code is consistent with the preset standard code, and if the check code is inconsistent with the preset standard code, performing the error correction operation include:
obtaining an error correction code by a file tailer corrector,
performing an error correction operation on the general-purpose machine learning model file according to the error correction code to obtain an error-corrected model file by the file tailer corrector,
checking whether a check code of the error-corrected general-purpose machine learning model file is consistent with the preset standard code, and
based on a checking result that the check code of the error-corrected general-purpose machine learning model file is consistent with the preset standard code, reading a model directory in the general-purpose machine learning model file;
reading hardware parameter information in the general-purpose machine learning model by the hardware matcher;
generating hardware matching information according to the hardware parameter information by the hardware matcher; and
matching the corresponding hardware in a device pool based on the hardware parameter information by the hardware matcher.
2. The method of claim 1 , wherein the step of reading a corresponding general-purpose machine learning model according to the model directory includes:
obtaining a storage offset of a target general-purpose machine learning model in the general-purpose machine learning model file, and
reading the target general-purpose machine learning model according to the storage offset.
3. The method of claim 1 , comprising
classifying and disassembling the general-purpose machine learning model to obtain stack area data and heap area data; and
computing the stack area data, the heap area data, and input data to obtain output data.
4. A device for parsing a general-purpose machine learning model file, comprising:
a file obtainer, a directory parser, and a model reader, wherein the directory parser is connected to the file obtainer, and the model reader, respectively; wherein
the file obtainer is configured to obtain a general-purpose machine learning model file;
the directory parser is configured to read a model directory in the general-purpose machine learning model file;
the model reader is configured to read a target general-purpose machine learning model according to the model directory,
wherein the file obtainer includes a file header checker;
wherein the file header checker is configured to obtain an identification code of the general-purpose machine learning model file, and detect whether the identification code complies with a preset rule,
wherein based on a determination that the identification code of the file header does not comply with the preset rule, the file header checker is configured to read a model directory in the general-purpose machine learning model file,
wherein the file header checker is further configured to obtain a check code of the general-purpose machine learning model file, and check whether the check code is consistent with a preset standard code, based on a checking result that the check code is inconsistent with the preset standard code, the file header checker is configured to perform an error correction operation,
wherein the file obtainer further includes a file tailer corrector,
wherein the file tailer corrector is configured to obtain an error correction code, perform error correction on the general-purpose machine learning model file according to the error correction code to obtain an error-corrected model file, and check whether a check code of the error-corrected model file is consistent with the preset standard code,
wherein based on a checking result that the check code of the error-corrected model file is consistent with the preset standard code, the file tailer corrector is configured to read the model directory in the general-purpose machine learning model file, and
a hardware matcher, wherein the hardware matcher is connected to the model reader, and the hardware matcher is configured to read hardware parameter information in the general-purpose machine learning model and match corresponding hardware in a device pool according to the hardware parameter information.
5. The device of claim 4 , wherein the model reader is further configured to obtain an offset of a target general-purpose machine learning model in the general-purpose machine learning model file, and
read the target general-purpose machine learning model according to the offset.
6. The device of claim 4 , wherein the device for parsing a general-purpose machine learning model file further includes a model distributor, wherein the model distributor is connected to the directory parser.