Maintaining synchronisation between memory writing and reading blocks using an internal buffer and a control channel
A hardware unit for manipulating data stored in a memory comprises an internal buffer, a memory reading block, configured to read the data from the memory and write the data to the internal buffer. a memory writing block, configured to read the data from the internal buffer and write the data to the memory. The hardware unit optionally also comprises a control channel between the memory reading block and the memory writing block, wherein the memory reading block and the memory writing block are configured to communicate via the control channel to maintain synchronisation between them when writing the data to the internal buffer and reading the data from the internal buffer, respectively. The hardware unit may be configured to apply one or more transformations to multidimensional data in the memory. The hardware unit may be configured to traverse the multidimensional array using a plurality of nested loops.
1 . A hardware unit for manipulating data stored in a memory, the hardware unit comprising:
an internal buffer;
a memory reading block, configured to read the data from the memory and write the data to the internal buffer;
a memory writing block, configured to read the data from the internal buffer and write the data to the memory; and
a control channel between the memory reading block and the memory writing block, wherein the memory reading block and the memory writing block are configured to communicate via the control channel to maintain synchronisation between them when writing the data to the internal buffer and reading the data from the internal buffer, respectively;
wherein the data comprises a multidimensional array comprising a plurality of data elements, wherein at least one of the memory reading block and the memory writing block is configured to traverse the multidimensional array using a plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array.
2 . The hardware unit of claim 1 , wherein at least one loop of the plurality of nested loops is configured to iterate a different number of times depending on at least one of:
(a) a loop index of at least one other loop of the plurality of nested loops; and
(b) a software configurable flag.
3 . The hardware unit of claim 1 , wherein each loop of the plurality of loops is configured to perform a variable number of iterations, the variable number being selected at runtime from a group comprising:
a first number of iterations to be performed when one or more outer loops of the plurality of nested loops are not in their end iteration; and
a second number of iterations to be performed when the one or more outer loops of the plurality of nested loops are in their end iteration.
4 . The hardware unit of claim 1 ,
wherein the multidimensional array is stored in the memory in a storage format having storage units of a predetermined third size, wherein one or more dimensions of the multidimensional array are not an integer multiple of the third size.
5 . The hardware unit of claim 4 , wherein at least one of the memory reading block and the memory writing block is configured to traverse the multidimensional array using a plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array,
wherein said at least one block is configured to select, for at least one loop of the plurality of nested loops, a different number of iterations when one or more outer loops are in their end iteration, as compared with the number of iterations of said at least one loop when the one or more outer loops are not in their end iteration.
6 . The hardware unit of claim 1 , wherein the synchronisation between the memory reading block and memory writing block ensures that no data is overwritten in the internal buffer by the memory reading block while it is still needed by the memory writing block, and that data is not read from the internal buffer by the memory writing block until it has been validly written to the internal buffer by the memory reading block.
7 . A hardware unit for manipulating data stored in a memory, the hardware unit comprising:
an internal buffer;
a memory reading block, configured to read the data from the memory and write the data to the internal buffer;
a memory writing block, configured to read the data from the internal buffer and write the data to the memory; and
a control channel between the memory reading block and the memory writing block, wherein the memory reading block and the memory writing block are configured to communicate via the control channel to maintain synchronisation between them when writing the data to the internal buffer and reading the data from the internal buffer, respectively,
wherein each of the memory reading block and the memory writing block has a respective synchronisation counter,
the blocks being configured to communicate their synchronisation counters with each other via the control channel,
wherein the hardware unit is configured to maintain synchronisation between the blocks by comparing the synchronisation counters.
8 . The hardware unit of claim 7 , wherein the data comprises a multidimensional array comprising a plurality of data elements, wherein each of the memory reading block and the memory writing block is configured to traverse the multidimensional array using a respective plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array,
wherein the synchronisation counter of the memory reading block is associated with a first loop of its plurality of loops and the synchronisation counter of the memory writing block is associated with a second loop of its plurality of loops.
9 . The hardware unit of claim 7 , wherein the operation of one of the blocks is paused if the difference between the respective synchronisation counters exceeds a predetermined threshold.
10 . A hardware-implemented method of manipulating data stored in a memory, the data comprising a multidimensional array comprising a plurality of data elements, the method comprising:
(i) reading the data from the memory and writing the data to the internal buffer using a memory reading block; and
(ii) reading the data from the internal buffer and writing the data to the memory using a memory writing block,
wherein at least one of the steps (i) and (ii) is performed using a plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array,
wherein at least one loop of the plurality of nested loops is configured to iterate a different number of times depending on a loop index of at least one other loop of the plurality of nested loops, and
wherein the memory reading block and the memory writing block are configured to communicate via a control channel to maintain synchronisation between step (i) and step (ii);
wherein each loop of the plurality of loops is configured to perform a variable number of iterations, the variable number being selected at runtime from a group comprising:
a first number of iterations to be performed when one or more outer loops of the plurality of nested loops are in their first iteration; and
a second number of iterations to be performed when the one or more outer loops of the plurality of nested loops are in their end iteration.
11 . The method of claim 10 , wherein at least one loop of the plurality of nested loops is further configured to iterate a different number of times depending on a software configurable flag.
12 . The method of claim 10 , further comprising:
maintaining a first synchronisation counter associated with a first loop among the plurality of loops performing step (i), wherein the first synchronisation counter is incremented with each iteration of the first loop;
maintaining a second synchronisation counter associated with a second loop among the plurality of loops performing step (ii), wherein the second synchronisation counter is incremented with each iteration of the second loop;
comparing a current value of the first synchronisation counter with a current value of the second synchronisation counter; and
controlling the progress of step (i) and or step (ii) based on a result of the comparison.
13 . The method of claim 12 , wherein controlling the progress of step (i) and/or step (ii) based on a result of the comparison comprises one or more of:
pausing step (i) if the first synchronisation counter exceeds the second synchronisation counter by more than a predetermined first threshold; and
pausing step (ii) if the second synchronisation counter exceeds the first synchronisation counter by more than a predetermined second threshold.
14 . A method of manufacturing, using an integrated circuit manufacturing system, the hardware unit as set forth in claim 1 , the method comprising:
processing, using a layout processing system, a computer readable dataset description of the hardware unit so as to generate a circuit layout description of an integrated circuit embodying the hardware unit or neural network accelerator system; and
manufacturing, using an integrated circuit generation system, the hardware unit according to the circuit layout description.
15 . A non-transitory computer readable storage medium having stored thereon computer readable code configured to cause the method as set forth in claim 10 to be performed when the code is run on at least one processor.
16 . A non-transitory computer readable storage medium having stored thereon a computer readable data description of a hardware unit as set forth in claim 1 that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the hardware unit or neural network accelerator system.
17 . An integrated circuit manufacturing system comprising:
a non-transitory computer readable storage medium having stored thereon a computer readable dataset description of a hardware unit as set forth in claim 1 ;
a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the hardware unit; and
an integrated circuit generation system configured to manufacture the hardware unit according to the circuit layout description.