Efficient in-memory representation of computation graph for fast serialization and comparison
Embodiments of the present disclosure introduce an offset-based representation for a computation graph of a neural network model. In this representation, the edges in the graph are implemented as offsets to the base address of the graph when the model is loaded into memory rather than pointers to address locations in memory. By replacing the pointer fields with offsets, the model in memory is no longer dependent on the allocation of memory. This overcomes the limitations of performance and/or power that is critical for comparing two models and copying a model to another memory location in latency sensitive applications and in mobile environments.
1. A method performed by a computing device for providing a data structure associated with a computation graph having a plurality of nodes, the method comprising:
allocating one or more memory blocks associated with the computation graph; and
assigning a plurality of memory units of the allocated one or more memory blocks, wherein each memory unit of the plurality of memory units is associated with a node of the computation graph and includes:
a first section assigned a value corresponding to a current node of the plurality of nodes, and
a second section assigned an offset value referencing another memory unit of the plurality of memory units, wherein the another memory unit is associated with another node coupled to the current node via an edge.
2. The method of claim 1 , further comprising assigning the value corresponding to an operand value of the current node.
3. The method of claim 2 , wherein the another node is an operation node that performs an operation using the operand value.
4. The method of claim 3 , wherein the another memory unit includes a first section configured to have either a pre-set value or a value associated with an output of the operation using the operand value.
5. The method of claim 1 , further comprising assigning the offset value based on a base address of a memory block of the allocated one or more memory blocks.
6. The method of claim 1 , further comprising assigning the offset value based on a base address of a memory sub-block of the allocated one or more memory blocks.
7. The method of claim 1 , wherein each memory unit of the plurality of memory units has a same size.
8. The method of claim 1 , further comprising assigning a root memory unit corresponding to a root node of the computation graph, wherein the root memory unit includes:
a first section assigned a value;
a second section assigned a NULL or negative value.
9. The method of claim 1 , further comprising generating a fast key lookup for configuring the data structure.
10. The method of claim 9 , wherein the fast key lookup is a digital signature.
11. A computing device for providing a data structure associated with a computation graph having a plurality of nodes, the computing device comprising:
a memory configured to store a set of instructions; and
a processor configured to execute the set of instructions to cause the computing device to:
allocate one or more memory blocks associated with the computation graph; and
assign a plurality of memory units of the allocated one or more memory blocks, wherein each memory unit of the plurality of memory units is associated with a node of the computation graph and includes:
a first section assigned a value corresponding to a current node of the plurality of nodes, and
a second section assigned an offset value referencing another memory unit of the plurality of memory units, wherein the another memory unit is associated with another node coupled to the current node via an edge.
12. The computing device of claim 11 , wherein the processor is configured to execute the set of instructions to cause the computing device to further perform:
assign the value corresponding to an operand value of the current node.
13. The computing device of claim 12 , wherein the another node is an operation node that performs an operation using the operand value.
14. The computing device of claim 13 , wherein the another memory unit includes a first section configured to have either a pre-set value or a value associated with an output of the operation using the operand value.
15. The computing device of claim 11 , wherein the processor is configured to execute the set of instructions to cause the computing device to further perform:
assign the offset value based on a base address of a memory block of the allocated one or more memory blocks.
16. The computing device of claim 11 , wherein the processor is configured to execute the set of instructions to cause the computing device to further perform:
assign the offset value based on a base address of a memory sub-block of the allocated one or more memory blocks.
17. The computing device of claim 11 , wherein each memory unit of the plurality of memory units has a same size.
18. The computing device of claim 11 , wherein the processor is configured to execute the set of instructions to cause the computing device to further perform:
assign a root memory unit corresponding to a root node of the computation graph, wherein the root memory unit includes:
a first section assigned a value;
a second section assigned a NULL or negative value.
19. The computing device of claim 11 , wherein the processor is configured to execute the set of instructions to cause the computing device to further perform:
generate a fast key lookup for configuring the data structure.
20. The computing device of claim 19 , wherein the fast key lookup is a digital signature.
21. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computing device to cause the computing device to perform a method for providing a data structure associated with a computation graph having a plurality of nodes, the method comprising:
allocating one or more memory blocks associated with the computation graph; and
assigning a plurality of memory units of the allocated one or more memory blocks, wherein each memory unit of the plurality of memory units is associated with a node of the computation graph and includes:
a first section assigned a value corresponding to a current node of the plurality of nodes, and
a second section assigned an offset value referencing another memory unit of the plurality of memory units, wherein the another memory unit is associated with another node coupled to the current node via an edge.