Merging models on an edge server
Systems and methods are provided for merging models for use in an edge server under the multi-access edge computing environment. In particular, a model merger selects a layer of a model based on a level of memory consumption in the edge server and determines sharable layers based on common properties of the selected layer. The model merger generates a merged model by generating a single instantiation of a layer that corresponds to the sharable layers. A model trainer trains the merged model based on training data for the respective models to attain a level of accuracy of data analytics above a predetermined threshold. The disclosed technology further refreshes the merged model upon observing a level of data drift that exceeds a predetermined threshold. The refreshing of the merged model includes detaching and/or splitting consolidated sharable layers of sub-models in the merged model. By merging models, the disclosed technology reduces memory footprints of models used in the edge server, rectifying memory scarcity issues in the edge server.
1 . A computer-implemented method for merging at least a part of a plurality of models, the method comprising:
selecting, based on memory consumption of a first layer of a first model in a memory of an edge server, the first layer of the first model as a first sharable layer, wherein the plurality of models comprises the first model and a second model;
determining, based on layer properties of the second model, a second layer of the second model in the memory of the edge server as a second sharable layer, wherein the second layer of the second model indicates one or more same layer properties as the first layer of the first model;
generating a third model as a merged model of the first model and the second model, wherein the third model comprises a third layer as a common layer representing a single instantiation of both the first layer and the second layer; and
transmitting the third model to the edge server, thereby causing the edge server to replace a respective instantiation of the first model and the second model in the memory of the edge server with the third model, and further causing the edge server to execute the third model to perform data analytics on the edge server, wherein the third model consumes less memory on the edge server than a combination of both the first model and the second model.
2 . The computer-implemented method of claim 1 , wherein the first model and the second model correspond to video analytic models.
3 . The computer-implemented method of claim 1 , wherein each of the first layer and the second layer is based on properties including one or more of:
input size,
output size,
kernel size, or
stride length.
4 . The computer-implemented method of claim 1 , wherein the first model and the second model are distinct.
5 . The computer-implemented method of claim 1 , the method further comprising:
receiving data drift information associated with a first sub-model of the third model;
generating, based on the data drift information associated with the first sub-model of the third model, a fourth model, wherein the fourth model includes the first sub-model;
updating the third model by detaching the first sub-model from the third model; and
transmitting the third model and the fourth model.
6 . The computer-implemented method of claim 1 , the method further comprising:
training the third model based on training data associated with the first model and the second model.
7 . The computer-implemented method of claim 6 , wherein training the third model comprises determining that the third model meets an accuracy threshold for performing the data analytics.
8 . The computer-implemented method of claim 1 , wherein the first layer and the second layer are not located in corresponding locations within the first model and the second model, respectively.
9 . A system for merging a plurality of models for use in an edge server, the system comprising:
a processor; and
a memory storing computer-executable instructions that when executed by the processor cause the system to:
select, based on memory consumption of a first layer of a first model in a memory of the edge server, the first layer of the first model as a first sharable layer, wherein the plurality of models comprises the first model and a second model;
determine, based on layer properties of the second model, a second layer of the second model in the memory of the edge server as a second sharable layer, wherein the second layer of the second model indicates one or more same layer properties as the first layer of the first model;
generate a third model as a merged model for the first model and the second model, wherein the third model comprises a third layer as a common layer representing a single instantiation of both the first layer and the second layer; and
transmit the third model to the edge server, thereby causing the edge server to replace a respective instantiation of the first model and the second model in the memory of the edge server with the third model, and further causing the edge server to execute the third model to perform data analytics on the edge server, wherein the third model consumes less memory on the edge server than a combination of both the first model and the second model.
10 . The system of claim 9 , wherein the first model and the second model correspond to video analytic models.
11 . The system of claim 9 , wherein each of the first layer and the second layer is based on properties including one or more of:
input size,
output size,
kernel size, or
stride length.
12 . The system of claim 9 , the computer-executable instructions when executed further cause the system to:
receive data drift information associated with a first sub-model of the third model;
generate, based on the data drift information associated with the first sub-model of the third model, a fourth model, wherein the fourth model includes the first sub-model;
update the third model by detaching the first sub-model from the third model; and
transmit the third model and the fourth model.
13 . The system of claim 9 , the computer-executable instructions when executed further cause the system to:
train the third model based on training data associated with the first model and the second model.
14 . The system of claim 13 , wherein training the third model comprises determining that the third model meets an accuracy threshold for performing the data analytics.
15 . The system of claim 9 , wherein the first layer and the second layer are not located in corresponding locations within the first model and the second model, respectively.
16 . A computer-readable recording medium storing computer-executable instructions that when executed by a processor cause a system to:
select, based on memory consumption of a first layer of a first model of a plurality of models in a memory of an edge server, the first layer of the first model as a first sharable layer, wherein the plurality of models comprises the first model and a second model;
determine, based on layer properties of the second model, a second layer of the second model in the memory of the edge server as a second sharable layer, wherein the second layer of the second model indicates one or more same layer properties as the first layer of the first model;
generate a third model as a merged model of the first model and the second model, wherein the third model comprises a third layer as a common layer representing a single instantiation of both the first layer and the second layer; and
transmit the third model to the edge server, thereby causing the edge server to replace a respective instantiation of the first model and the second model in the memory of the edge server with the third model, and further causing the edge server to execute the third model to perform data analytics on the edge server, and wherein the third model consumes less memory on the edge server than a combination of both the first model and the second model.
17 . The computer-readable recording medium of claim 16 , wherein the first model and the second model correspond to video analytic models.
18 . The computer-readable recording medium of claim 16 , wherein each of the first layer and the second layer is based on properties including one or more of:
input size,
output size,
kernel size, or
stride length.
19 . The computer-readable recording medium of claim 16 , the computer-executable instructions when executed further cause the system to:
receive data drift information associated with a first sub-model of the third model;
generate, based on the data drift information associated with the first sub-model of the third model, a fourth model, wherein the fourth model includes the first sub-model;
update the third model by detaching the first sub-model from the third model; and
transmit the third model and the fourth model.
20 . The computer-readable recording medium of claim 16 , the computer-executable instructions when executed further cause the system to:
train the third model based on training data associated with the first model and the second model, wherein training the third model comprises determining that the third model meets an accuracy threshold for performing the data analytics.