Performing consecutive mac operations on a set of data using different kernels in a MAC circuit

1. A circuit arrangement, comprising:

an array of multiply-and-accumulate (MAC) circuits, wherein each MAC circuit includes a cache configured for storage of a plurality of kernels, and the MAC circuits are configured to:

receive a first set of data elements of an input feature map (IFM) at a first rate;

perform first MAC operations on the first set of the data elements and a first one of the kernels (H) associated with a first output feature map (OFM) depth index (d 2 ) during a first MAC cycle, wherein a rate of MAC cycles is faster than the first rate; and

perform second MAC operations on the first set of the data elements and a second one of the kernels (H) associated with a second OFM depth index (d 2 ) during a second MAC cycle that consecutively follows the first MAC cycle.

2. The circuit arrangement of claim 1 , further comprising a control circuit coupled to the array and configured to:

receive a first address of the first kernel in the cache;

generate a second address of the second kernel in the cache based on the first address;

provide the first address to the MAC circuits during the first MAC cycle; and

provide the second address to the MAC circuits during the second MAC cycle.

3. The circuit arrangement of claim 2 , wherein the control circuit is configured to increment the first address by a fixed value to generate the second address in response to the first address being mapped to a start address of a block data transfer.

4. The circuit arrangement of claim 2 , wherein the control circuit is configured to decrement the first address by a fixed value to generate the second address in response to the first address being mapped to an end address of a block data transfer.

5. The circuit arrangement of claim 1 , wherein the MAC circuits of the array are configured to perform third MAC operations on the first set of the data elements and a third one of the kernels (H) associated with a third OFM depth index (d 2 ) during a third MAC cycle that consecutively follows the second MAC cycle.

6. The circuit arrangement of claim 5 , further comprising a first control circuit coupled to the array and configured to:

receive a first address of the first one of the kernels in the cache;

generate a second address of the second one of the kernels in the cache based on the first address;

generate a third address of the third one of the kernels in the cache based on the first address;

provide the first address to the MAC circuits during the first MAC cycle;

provide the second address to the MAC circuits during the second MAC cycle; and

provide the third address to the MAC circuits during the third MAC cycle.

7. The circuit arrangement of claim 6 , wherein the first control circuit is configured to:

increment the first address by a fixed value to generate the second address; and

increment the second address by the fixed value to generate the third address.

8. The circuit arrangement of claim 6 , further comprising a second control circuit coupled to the array and configured to:

receive the first set of the data elements of the IFM and a second set of the data elements of the IFM at an input rate that is equal to or faster than the first rate; and

delay providing the second set of the data elements to the MAC circuits by at least a MAC cycle in response to input control signals.

9. The circuit arrangement of claim 1 , wherein the MAC circuits of the array are configured to:

receive a second set of the data elements of the IFM at a second rate that is slower than the first rate;

perform third MAC operations on the second set of the data elements and a third one of the kernels (H) associated with a third OFM depth index (d 2 ) during a third MAC cycle that consecutively follows the second MAC cycle;

perform fourth MAC operations on the second set of the data elements and a fourth one of the kernels (H) associated with a fourth OFM depth index (d 2 ) during a fourth MAC cycle that consecutively follows the third MAC cycle; and

perform fifth MAC operations on the second set of the data elements and a fifth one of the kernels (H) associated with a fifth OFM depth index (d 2 ) during a fifth MAC cycle that consecutively follows the fourth MAC cycle.

10. The circuit arrangement of claim 1 , wherein the array includes a plurality of context switching accumulators coupled to each column of MAC circuits of the array configured to:

accumulate a stored value with an output of a MAC circuit directly coupled to the accumulators; and

initialize with the output of the MAC circuit directly coupled to the accumulators.

11. A method, comprising:

inputting a first set of data elements of an input feature map (IFM) at a first rate to an array of multiply-and-accumulate (MAC) circuits;

during a first MAC cycle, performing, by the MAC circuits, first MAC operations on the first set of the data elements and a first kernel (H) stored in a cache of the MAC circuits, wherein the first kernel is associated with a first output feature map (OFM) depth index (d 2 ), wherein a rate of MAC cycles is faster than the first rate; and

during a second MAC cycle that consecutively follows the first MAC cycle, performing, by the MAC circuits, second MAC operations on the first set of the data elements and a second kernel (H) stored in the cache of the MAC circuits, wherein the second kernel is associated with a second OFM depth index (d 2 ).

12. The method of claim 11 , further comprising:

inputting a first address of the first kernel in the cache to a control circuit coupled to the array;

generating, by the control circuit, a second address of the second kernel in the cache based on the first address;

inputting the first address to the MAC circuits during the first MAC cycle; and

inputting the second address to the MAC circuits during the second MAC cycle.

13. The method of claim 12 , wherein generating the second address includes incrementing the first address by a fixed value in response to the first address being mapped to a start address of a data block transfer.

14. The method of claim 12 , wherein generating the second address includes decrementing the first address by a fixed value in response to the first address being mapped to an end address of a data block transfer.

15. The method of claim 11 , further comprising during a third MAC cycle that consecutively follows the second MAC cycle, performing, by the MAC circuits, third MAC operations on the first set of the data elements and a third kernel (H) stored in the cache of the MAC circuits, wherein the third kernel is associated with a third OFM depth index (d 2 ).

16. The method of claim 15 , further comprising:

inputting a first address of the first one of the kernels in the cache to a first control circuit coupled to the array;

generating, by the first control circuit, a second address of the second kernel in the cache based on the first address;

generating, by the first control circuit, a third address of the third kernel in the cache based on the first address;

inputting the first address to the MAC circuits during the first MAC cycle;

inputting the second address to the MAC circuits during the second MAC cycle; and

inputting the third address to the MAC circuits during the third MAC cycle.

17. The method of claim 16 , wherein:

generating the second address includes incrementing the first address by a fixed value; and

generating the second address includes incrementing the second address by the fixed value.

18. The method of claim 16 , further comprising:

inputting the first set of the data elements of the IFM and a second set of the data elements of the IFM to a second control circuit coupled to the array at an input rate that is equal to or faster than the first rate; and

delaying, by the second control circuit, inputting the second set of the data elements to the MAC circuits by at least a MAC cycle in response to input control signals to the second control circuit.

19. The method of claim 11 , further comprising:

inputting a second set of the data elements of the IFM to the MAC circuits of the array at a second rate that is slower than the first rate;

during a third MAC cycle that consecutively follows the second MAC cycle, performing, by the MAC circuits, third MAC operations on the second set of the data elements and a third kernel (H) stored in the cache of the MAC circuits, wherein the third kernel is associated with a third OFM depth index (d 2 );

during a fourth MAC cycle that consecutively follows the third MAC cycle, performing, by the MAC circuits, fourth MAC operations on the second set of the data elements and a fourth kernel (H) stored in the cache of the MAC circuits, wherein the fourth kernel is associated with a fourth OFM depth index (d 2 ); and

during a fifth MAC cycle that consecutively follows the fourth MAC cycle, perform, by the MAC circuits, fifth MAC operations on the second set of the data elements and a fifth kernel (H) stored in the cache of the MAC circuits, wherein the fifth kernel is associated with a fifth OFM depth index (d 2 ).

20. The method of claim 11 , further comprising:

accumulating, by a plurality of context switching accumulators coupled to each column of MAC circuits of the array, a stored value with an output of a MAC circuit directly coupled to the accumulators; and

initializing the accumulators with the output of the MAC circuit directly coupled to the accumulators.