Inference execution method for candidate neural networks and switching neural networks
An inference execution method includes: selecting an inference neural network from among a plurality of inference neural network candidates generated from one training neural network that has been trained; sequentially obtaining data; sequentially executing, on the data sequentially obtained, inference using the inference neural network; sequentially outputting results of the inference sequentially executed; and selecting a new inference neural network from among the plurality of inference neural network candidates and switching the inference neural network to be used in the execution of the inference to the new inference neural network during an inference execution period in which the data is sequentially obtained, the inference is sequentially executed, and the results of the inference are sequentially output.
1 . An inference execution method for at least one information processing circuit to execute inference on data using an inference neural network, the inference execution method comprising:
training, by a first computing device, a single neural network to generate a single trained neural network;
generating, based on the single trained neural network and by applying differing sets of parameters, a plurality of inference neural network candidates, wherein the differing sets of parameters correspond to differing environment states, and wherein each of the plurality of inference neural network candidates utilize smaller data bits than the single trained neural network for execution via a second computing device that is lower in performance than the first device;
selecting, among the plurality of inference neural network candidates, a first inference neural network candidate for execution by the second device;
sequentially obtaining image data to be processed by the first inference neural network candidate;
sequentially executing, on the image data sequentially obtained, the inference using the first inference neural network candidate for detection of objects within the image data;
sequentially outputting results of the inference sequentially executed, wherein the outputted results include successful and/or failed detection of the objects within the image data; and
based on an inference accuracy level of the outputted results of the first inference neural network candidate, selecting a second inference neural network candidate from among the plurality of inference neural network candidates, and switching the first inference neural network candidate to be used in the execution of the inference with the second inference neural network candidate; and
adaptably applying, by the second computing device, the second inference neural network candidate during performance of the inference, when the second inference neural network candidate provides a more successful detection of the objects within the image data.
2 . The inference execution method according to claim 1 , wherein
the plurality of inference neural network candidates have different dynamic ranges and different levels of resolution.
3 . The inference execution method according to claim 1 , wherein
the plurality of inference neural network candidates correspond respectively to a plurality of environment state candidates that correspond to the differing environment states in which the image data is obtained by sensing.
4 . The inference execution method according to claim 3 , wherein
the environment state includes an attribute related to a location, an illuminance level, or a time slot.
5 . The inference execution method according to claim 1 , wherein
each of one or more inference neural network candidates among the plurality of inference neural network candidates is evaluated using the image data, and the second inference neural network is selected from among the one or more inference neural network candidates according to an evaluation result.
6 . The inference execution method according to claim 1 , wherein
a neural network having highest resolution among one or more inference neural network candidates is selected from among the plurality of inference neural network candidates as the second inference neural network, the one or more inference neural network candidates each having a dynamic range including a data value used in the inference.
7 . The inference execution method according to claim 3 , further comprising:
obtaining an environment state from a measuring instrument that measures the environment state, wherein
a neural network corresponding to the environment state is selected as the second inference neural network according to the environment state from among the plurality of inference neural network candidates that correspond respectively to the differing environment states.
8 . The inference execution method according to claim 1 , wherein
whether accuracy of the inference is decreasing is determined, and when the accuracy is determined to be decreasing, the second inference neural network is selected, and the first inference neural network is switched to the second inference neural network.
9 . The inference execution method according to claim 8 , wherein
whether the accuracy is decreasing is determined according to at least one of a data value used in the inference or a likelihood related to the results of the inference.
10 . The inference execution method according to claim 9 , wherein
when the data value is outside a first range, the accuracy is determined to be decreasing.
11 . The inference execution method according to claim 9 , further comprising:
determining that the accuracy is decreasing when the data value used in the inference is outside a second range.
12 . The inference execution method according to claim 9 , wherein
when the data value continuously remains within a third range throughout a first segment of the image data, the accuracy is determined to be decreasing.
13 . The inference execution method according to claim 9 , wherein
when the likelihood continuously remains lower than a reference throughout a second segment of the image data, the accuracy is determined to be decreasing.
14 . The inference execution method according to claim 1 , wherein
periodically, the second inference neural network is selected, and the first inference neural network is switched to the second inference neural network.
15 . The inference execution method according to claim 1 , further comprising:
obtaining an environment state in which the image data is obtained by sensing from a measuring instrument that measures the environment state, wherein
when the environment state changes, the second inference neural network is selected, and the first inference neural network is switched to the second inference neural network.
16 . The inference execution method according to claim 1 , further comprising:
obtaining a plurality of datasets, wherein
the plurality of inference neural network candidates that correspond respectively to the plurality of datasets are generated from the single trained neural network according to the plurality of datasets.
17 . A non-transitory computer-readable recording medium having a program recorded thereon for causing at least one information processing circuit to perform an inference execution method for the at least one information processing circuit to execute inference on data using an inference neural network, the inference execution method including:
training, by a first computing device, a single neural network to generate a single trained neural network;
generating, based on the single trained neural network and by applying differing sets of parameters, a plurality of inference neural network candidates, wherein the differing sets of parameters correspond to differing environment states, and wherein each of the plurality of inference neural network candidates utilize smaller data bits than the single trained neural network for execution via a second computing device that is lower in performance than the first device;
selecting, among the plurality of inference neural network candidates, a first inference neural network candidate for execution by the second device;
sequentially obtaining image data to be processed by the first inference neural network candidate;
sequentially executing, on the image data sequentially obtained, the inference using the first inference neural network candidate for detection of objects within the image data;
sequentially outputting results of the inference sequentially executed, wherein the outputted results include successful and/or failed detection of the objects within the image data; and
based on an inference accuracy level of the outputted results of the first inference neural network candidate, selecting a second inference neural network candidate from among the plurality of inference neural network candidates, and switching the first inference neural network candidate to be used in the execution of the inference with the second inference neural network candidate; and
adaptably applying, by the second computing device, the second inference neural network candidate during performance of the inference, when the second inference neural network candidate provides a more successful detection of the objects within the image data.
18 . A non-transitory computer-readable recording medium having recorded thereon an inference execution model including a program for causing at least one information processing circuit to perform an inference execution method for the at least one information processing circuit to execute inference on data using an inference neural network, the inference execution method including:
training, by a first computing device, a single neural network to generate a single trained neural network;
generating, based on the single trained neural network and by applying differing sets of parameters, a plurality of inference neural network candidates, wherein the differing sets of parameters correspond to differing environment states, and wherein each of the plurality of inference neural network candidates utilize smaller data bits than the single trained neural network for execution via a second computing device that is lower in performance than the first device;
selecting, among the plurality of inference neural network candidates, a first inference neural network candidate for execution by the second device;
sequentially obtaining image data to be processed by the first inference neural network candidate;
sequentially executing, on the image data sequentially obtained, the inference using the first inference neural network candidate for detection of objects within the image data;
sequentially outputting results of the inference sequentially executed, wherein the outputted results include successful and/or failed detection of the objects within the image data;
based on an inference accuracy level of the outputted results of the first inference neural network candidate, selecting a second inference neural network candidate from among the plurality of inference neural network candidates, and switching the first inference neural network candidate to be used in the execution of the inference with the second inference neural network candidate, wherein
the inference execution model further includes the single trained neural network that has been trained or the plurality of inference neural network candidates generated from the single trained neural network; and
adaptably applying, by the second computing device, the second inference neural network candidate during performance of the inference, when the second inference neural network candidate provides a more successful detection of the objects within the image data.