Multiple locally stored artificial neural network computations
Systems and methods for performing multiple locally stored artificial neural network (ANN) computations are provided. An example method comprises receiving, by one or more processing units, an ANN dataset associated with at least one ANN of a plurality of ANNs; storing, by processing units, the ANN dataset in a memory coupled to the processing units; associating, by the processing units, a base address with the at least one ANN, wherein the base address is to be used to locate the ANN dataset in the memory; keeping, by the processing units, the ANN dataset in the memory; receiving, by the processing units, an input dataset and the base address; determining, by the processing units and based on the base address, a location of the ANN dataset in the memory; and performing, by the processing units, ANN computation using the ANN dataset and input dataset.
1 . A system comprising:
a memory configured to store artificial neural network (ANN) datasets associated with a plurality of ANNs; and
an electronic circuit configured to process solely one ANN dataset selected from the ANN datasets at a time; wherein the electronic circuit is configured to:
receive, from an application programming interface (API), a first ANN dataset associated with a first ANN of the plurality of ANNs;
store the first ANN dataset in the memory at a first base address;
receive, from the API, an identifier for a second ANN dataset, the identifier being calculated by the API;
determine, based on the identifier, whether the identifier is present in the memory;
in response to the determination that the identifier is not present in the memory:
cause the API to load the second ANN dataset into the memory, the second ANN dataset being associated with a second ANN of the plurality of ANNs; and
store the second ANN dataset in the memory at a second base address; and
while the first ANN dataset and the second ANN dataset are stored in the memory:
receive, from the API, a first input dataset and the first base address;
based on the first base address, locate, in the memory, the first ANN dataset;
perform a first ANN computation based on the first ANN dataset and the first input dataset;
select, from the plurality of ANNs and based on a result of the first ANN computation, the second ANN dataset for performing a second ANN computation;
receive, from the API, a second input dataset and the second base address;
based on the second base address, locate, in the memory, the second ANN dataset; and
perform the second ANN computation based on the second ANN dataset and the second input dataset, the first ANN computation and the second ANN computation being carried out using a same computational block of the electronic circuit.
2 . The system of claim 1 , wherein the storing of the first ANN dataset in the memory is controlled by a first process and the first ANN computation is controlled by a second process, the first process being different from the second process.
3 . The system of claim 2 , wherein after ending the first process, the first ANN dataset is kept in the memory.
4 . The system of claim 1 , wherein the electronic circuit is configured to receive an indication to remove the first ANN dataset from the memory.
5 . The system of claim 1 , wherein the electronic circuit is configured to:
receive, from the API, priorities associated with the first ANN dataset and the second ANN dataset; and
perform the first ANN computation using the first ANN dataset and the second ANN computation using the second ANN dataset in a sequence based on the priorities.
6 . The system of claim 1 , wherein:
the first ANN dataset includes a first layer data associated with a first layer of the first ANN and a second layer data associated with a second layer of the first ANN; and
the electronic circuit is configured to:
keep the first layer data in the memory until receiving a first indication to remove the first layer data from the memory; and
keep the second layer data in the memory until receiving a second indication to remove the second layer data from the memory, wherein the first indication differs from the second indication.
7 . The system of claim 1 , wherein the electronic circuit is further configured to:
store the result of the first ANN computation in the memory; and
receive, from the API, the second base address of the second ANN dataset and an instruction to perform the second ANN computation using the result of the first ANN computation for the first input dataset.
8 . The system of claim 1 , wherein the electronic circuit is configured to:
receive, from the API, a unique identifier of the first ANN dataset instead of the first base address;
associate the unique identifier with the first base address; and
keep the unique identifier in the memory while the first ANN dataset is not removed from the memory.
9 . The system of claim 8 , wherein the unique identifier is determined based on the first ANN dataset, the first ANN dataset including a description of operations included in the first ANN and weights parameters associated with connections of the first ANN.
10 . The system of claim 8 , wherein the electronic circuit is configured to
receive, from the API, the first input dataset and the unique identifier;
determine, based on the unique identifier, the first base address for the first ANN dataset in the memory, the first ANN dataset being associated with the unique identifier; and
perform the first ANN computation using the first ANN dataset and the first input dataset.
11 . The system of claim 8 , wherein the electronic circuit is configured to:
receive, from the API, the unique identifier and an instruction to remove the first ANN dataset; and
remove, from the memory, information of the association between the unique identifier and the first base address and, thereby, remove the first ANN dataset from the memory.
12 . The system of claim 1 , wherein the electronic circuit is further configured to:
associate an input base address with the second input dataset, wherein the input base address is to be used to locate the second input dataset in the memory; and
keep the second input dataset in the memory until receiving an indication to remove the second input dataset.
13 . The system of claim 12 , wherein the second input dataset is to be used for a third ANN computation of one or more other ANNs of the plurality of ANNs.
14 . The system of claim 12 , wherein the electronic circuit is configured to:
receive, from the API, the first base address and the input base address;
determine, based on the first base address, a location of the first ANN dataset in the memory;
determine, based on the input base address, a location of the second input dataset in the memory; and
perform the first ANN computation using the first ANN dataset and the second input dataset.
15 . The system of claim 1 , wherein the electronic circuit and the memory are integrated into an electronic board configured to perform computations of two or more ANNs of the plurality of ANNs.
16 . The system of claim 15 , wherein the electronic circuit includes at least one field programmable gate array.
17 . A method comprising:
receiving, from an application programming interface (API) by an electronic circuit configured to process solely one artificial neural network (ANN) dataset selected from ANN datasets associated with a plurality of ANNs at a time, a first ANN dataset associated with a first ANN of the plurality of ANNs;
determining, by the electronic circuit, a first base address for the first ANN, wherein the first base address is to be used to locate the first ANN dataset in a memory coupled to the electronic circuit;
storing, by the electronic circuit, the first ANN dataset in the memory at the first base address;
receiving, by the electronic circuit, an identifier for a second ANN dataset from the API, the identifier being calculated by the API;
determining, by the electronic circuit and based on the identifier, whether the identifier is present in the memory;
in response to the determination that the identifier is not present in the memory:
causing the API to load the second ANN dataset into the memory, the second ANN dataset being associated with a second ANN of the plurality of ANNs; and
storing, by the electronic circuit, the second ANN dataset in the memory at a second base address; and
wherein while the first ANN dataset and the second ANN dataset are stored in the memory, the electronic circuit is configured to:
receive, from the API, a first input dataset and the first base address;
based on the first base address, locate, in the memory, the first ANN dataset;
perform a first ANN computation based on the first ANN dataset and the first input dataset;
select, from the plurality of ANNs and based on a result of the first ANN computation, the second ANN dataset for performing a second ANN computation;
receive, from the API, a second input dataset and the second base address;
based on the second base address, locate, in the memory, the second ANN dataset; and
perform the second ANN computation based on the second ANN dataset and the second input dataset, the first ANN computation and the second ANN computation being carried out using a same computational block of the electronic circuit.
18 . The method of claim 17 , wherein the storing of the first ANN dataset in the memory is controlled by a first process and the first ANN computation is controlled by a second process, the first process being different from the second process.
19 . A system comprising:
a memory configured to store artificial neural network (ANN) datasets associated with a plurality of ANNs; and
an electronic circuit configured to process solely one ANN dataset selected from the ANN datasets at a time; and wherein the electronic circuit is configured to:
receive, from an application programming interface (API), one or more ANN datasets and one or more input datasets associated with one or more ANNs of the plurality of ANNs, each of the one or more ANN datasets including a description of a structure parameters of at least one ANN of the plurality of ANNs;
store the one or more ANN datasets and the one or more input datasets in the memory;
associate one or more base addresses with the one or more ANN datasets, wherein the one or more base addresses are to be used to locate the one or more ANN datasets in the memory;
associate one or more input base addresses with the one or more input datasets, wherein the one or more input base addresses are to be used to locate the one or more input datasets in the memory;
receive, from the API, a computational model, the computational model including relations between the one or more base addresses and the one or more input base addresses; and
perform, based on the computational model, one or more ANN computations; wherein:
the one or more ANN datasets include a first ANN dataset and a second ANN dataset; and
the first ANN dataset is stored in the memory at a first base address, the first ANN dataset being associated with a first ANN of the plurality of ANNs; and
the electronic circuit is configured to:
receive, from the API, an identifier for the second ANN dataset, the identifier being calculated by the API;
determine, based on the identifier, whether the identifier is present in the memory;
in response to the determination that the identifier is not present in the memory:
cause the API to load the second ANN dataset into the memory, the second ANN dataset being associated with a second ANN of the plurality of ANNs; and
store the second ANN dataset in the memory at a second base address; and
while the first ANN dataset and the second ANN dataset are stored in the memory:
receive, from the API, a first input dataset and the first base address;
based on the first base address, locate, in the memory, the first ANN dataset;
perform a first ANN computation based on the first ANN dataset and the first input dataset;
select, from the plurality of ANNs and based on a result of the first ANN computation, the second ANN dataset for performing a second ANN computation;
receive, from the API, a second input dataset and the second base address;
based on the second base address, locate, in the memory, the second ANN dataset; and
perform the second ANN computation based on the second ANN dataset and the second input dataset, the first ANN computation and the second ANN computation being carried out using a same computational block of the electronic circuit.
20 . The system of claim 19 , wherein the storing of the first ANN dataset in the memory is controlled by a first process and the first ANN computation is controlled by a second process, the first process being different from the second process.