Tensor dropout in a neural network

1. A method for selectively dropping out feature elements from a tensor in a neural network, the method comprising:

receiving a first tensor from a first layer of a neural network, the first tensor comprising a first set of N feature elements, where N is a positive integer;

obtaining a compressed mask for the first tensor, the compressed mask comprising a first array of N single-bit mask elements;

receiving the first array of N single bit mask elements at each of N lanes of a reconfigurable computing unit;

respectively receiving a feature element of the first set of N feature elements of the first tensor at the N lanes of the reconfigurable computing unit;

selectively dropping out feature elements from the first set of N feature elements of the first tensor to generate a third set of N feature elements to use as at least part of a second tensor by selecting either a zero value or a feature element of the first set of N feature elements received at a lane of the N lanes of the reconfigurable computing unit for a corresponding feature element of the third set of N feature elements, based on a single-bit mask element of the first array of N single-bit mask elements selected based on the lane; and

propagating the second tensor to a second layer of the neural network.

2. The method of claim 1 , wherein the first tensor also includes a second set of N feature elements, and the compressed mask also includes a second array of N single-bit mask elements, the method further comprising:

receiving the second array of N single bit mask elements at each of N lanes of the reconfigurable computing unit;

respectively receiving a feature element of the second set of N feature elements of the first tensor at the N lanes of the reconfigurable computing unit;

selectively dropping out feature elements from the second set of N feature elements of the first tensor to generate a fourth set of N feature elements to use as a part of the second tensor by selecting either the zero value or a feature element of the second set of N feature elements received at the lane of the N lanes of the reconfigurable computing unit for a corresponding feature element of the fourth set of N feature elements, based on a single-bit mask element of the second array of N single-bit mask elements selected based on the lane.

3. The method of claim 2 , further comprising:

selectively dropping out feature elements from the first set of N feature elements of the first tensor to generate the third set of N feature elements to use as a part of the second tensor by, at each of the N lanes of the reconfigurable computing unit, selecting either the zero value or a respective feature element of the first set of N feature elements received by a respective lane of the N lanes of the reconfigurable computing unit for a corresponding feature element of the third set of N feature elements, based on a single-bit mask element of the first array of N single-bit mask elements selected based on the respective lane; and

selectively dropping out feature elements from the second set of N feature elements of the first tensor to generate the fourth set of N feature elements to use as a part of the second tensor by, at each of the N lanes of the reconfigurable computing unit, selecting either the zero value or a respective feature element of the second set of N feature elements received by the respective lane of the N lanes of the reconfigurable computing unit for a corresponding feature element of the fourth set of N feature elements, based on a single-bit mask element of the second array of N single-bit mask elements selected based on the respective lane.

4. The method of claim 2 , wherein:

the first set of N feature elements are adjacent feature elements in the first tensor and the second set of N feature elements are adjacent feature elements in the first tensor separate from the first set of N feature elements; and

said receiving of the second set of N feature elements occurs during a second vector bus cycle that starts after a start of a first vector bus cycle where said receiving of the first set of N feature elements occurs.

5. The method of claim 4 , wherein the first array of N single-bit mask elements is received over a scalar bus during the first vector bus cycle and the second array of N single-bit mask elements is received over the scalar bus during the second vector bus cycle.

6. The method of claim 4 , further comprising at each of the N lanes of the reconfigurable computing unit:

during a first dropout cycle that includes the first vector bus cycle, right-shifting the first array of N single-bit mask elements by a number of bits corresponding to a lane number of a respective lane of the reconfigurable computing unit to generate a right-shifted first array of single-bit mask elements, and selecting either the zero value or the feature element of the first set of N feature elements received by the lane for the corresponding feature element of the third set of N feature elements, based on a Least Significant Bit (LSB) of the right-shifted first array of single-bit mask elements; and

during a second dropout cycle that includes the second vector bus cycle, right-shifting the second array of N single-bit mask elements by a number of bits corresponding to the lane number to generate a right-shifted second array of single-bit mask elements, and selecting either the zero value or the feature element of the second set of N feature elements received by the lane for the corresponding feature element of the fourth set of N feature elements, based on a LSB of the right-shifted second array of single-bit mask elements;

wherein the N lanes of the reconfigurable computing unit are numbered from 0 to N-1.

7. The method of claim 4 , further comprising:

during a first dropout cycle that includes the first vector bus cycle in the lane of the N lanes of the reconfigurable computing unit, right-shifting the first array of N single-bit mask elements by a number of bits corresponding to a lane number of the lane to generate a right-shifted first array of single-bit mask elements, and selecting either the zero value or the feature element of the first set of N feature elements received by the lane for the corresponding feature element of the third set of N feature elements based on a Least Significant Bit (LSB) of the right-shifted first array of single-bit mask elements; and

during a second dropout cycle that includes the second vector bus cycle in the lane of the N lanes of the reconfigurable computing unit, right-shifting the second array of N single-bit mask elements by the number of bits corresponding to the lane number of the lane to generate a right-shifted second array of single-bit mask elements, and selecting either the zero value or the feature element of the second set of N feature elements received by the lane for the corresponding feature element of the fourth set of N feature elements, based on a LSB of the right-shifted second array of single-bit mask elements;

wherein the N lanes of the reconfigurable computing unit are numbered from 0 to N-1.

8. The method of claim 7 , wherein the first dropout cycle and the second dropout cycle overlap in time.

9. The method of claim 2 , wherein the first array of N single-bit mask elements and the second array of N single-bit mask elements are received by each of the N lanes of the reconfigurable computing unit in a single scalar bus cycle.

10. The method of claim 2 , wherein:

the first set of N feature elements and the second set of N feature elements are interleaved in the first tensor so that the first set of N feature elements has even feature elements of the first tensor and the second set of N feature elements has odd feature elements of the first tensor;

the first array of N single bit mask elements consists of consecutively arranged even single-bit mask elements and the second array of N single bit mask elements consists of consecutively arranged odd single-bit mask elements; and

said receiving of the first set of N feature elements and said receiving of the second set of N feature elements occur during a single vector bus cycle, wherein each of the N lanes of the reconfigurable computing unit receive an even feature element and an odd feature element of the first tensor.

11. The method of claim 10 , further comprising, in the lane of the N lanes of the reconfigurable computing unit:

right-shifting both the first array of N single bit mask elements and the second array of N single bit mask elements by a number of bits based on a lane number of the lane of the N lanes of the reconfigurable computing unit to respectively generate a right-shifted even array of single-bit mask elements and a right-shifted odd array of single-bit mask elements;

selecting, for the corresponding feature element of the third set of feature N elements, either the zero value or the received even feature element of the first set of N feature elements based on a Least Significant Bit (LSB) of the right-shifted even array of single-bit mask elements;

selecting, for the corresponding feature element of the fourth set of N feature elements, either the zero value or the received odd feature element of the second set of N feature elements based on a LSB of the right-shifted odd array of single-bit mask elements.

12. The method of claim 10 , further comprising at each of the N lanes of the reconfigurable computing unit:

right-shifting both the first array of N single bit mask elements and the second array of N single bit mask elements by a number of bits based on a lane number of a respective lane of the N lanes of the reconfigurable computing unit to generate a right-shifted even array of single-bit mask elements and a right-shifted odd array single-bit mask elements;

selecting, for a corresponding feature element of the third set, either the zero value or the received even feature element of the first set of N feature elements based on a Least Significant Bit (LSB) of the right-shifted even array of single-bit mask elements;

selecting, for a corresponding feature element of the fourth set of N feature elements, either the zero value or the received odd feature element of the second set of N feature elements based on a LSB of the right-shifted odd array of single-bit mask elements.

13. The method of claim 1 , further comprising:

generating a mask comprising at least N mask elements, wherein each mask element of the at least N mask elements has either a first value or a second value that are each represented using a plurality of bits;

respectively compressing N mask elements of the at least N mask elements to generate a corresponding single-bit mask element of the first array of N single-bit mask elements of the compressed mask; and

storing the compressed mask in a memory.

14. A non-transitory computer readable storage medium impressed with computer program instructions, the instructions, when executed by a reconfigurable computing unit having N lanes, implement a method comprising:

receiving a first array of N single bit mask elements of a compressed mask at each of N lanes of a reconfigurable computing unit;

respectively receiving a feature element of a first set of N feature elements of a first tensor at the N lanes of the reconfigurable computing unit;

selectively dropping out feature elements from the first set of N feature elements of the first tensor to generate a third set of N feature elements to use as at least a part of a second tensor by, at each of the N lanes of the reconfigurable computing unit, selecting either a zero value or a respective feature element of the first set of N feature elements received by respective lanes of the N lanes of the reconfigurable computing unit for a corresponding feature element of the third set of N feature elements, based on a single-bit mask element of the first array of N single-bit mask elements selected based on the respective lane.

15. The non-transitory computer readable storage medium of claim 14 , wherein the first tensor also includes a second set of N feature elements, and the compressed mask also includes a second array of N single-bit mask elements, the method further comprising:

receiving the second array of N single bit mask elements at each of N lanes of the reconfigurable computing unit;

respectively receiving a feature element of the second set of N feature elements of the first tensor at the N lanes of the reconfigurable computing unit;

selectively dropping out feature elements from the second set of N feature elements of the first tensor to generate a fourth set of N feature elements to use as a part of the second tensor by, at each of the N lanes of the reconfigurable computing unit, selecting either the zero value or a respective feature element of the second set of N feature elements received by respective lanes of the N lanes of the reconfigurable computing unit for a corresponding feature element of the fourth set of N feature elements, based on a single-bit mask element of the second array of N single-bit mask elements selected based on the respective lane.

16. The non-transitory computer readable storage medium of claim 15 , wherein:

the first set of N feature elements are adjacent feature elements in the first tensor and the second set of N feature elements are adjacent feature elements in the first tensor separate from the first set of N feature elements; and

said receiving of the second set of N feature elements occurs during a second vector bus cycle that starts after a start of a first vector bus cycle wherein said receiving of the first set of N feature elements occurs;

the method, at each of the N lanes of the reconfigurable computing unit, further comprising:

during a first dropout cycle that includes the first vector bus cycle, right-shifting the first array of N single-bit mask elements by a number of bits corresponding to a lane number of s respective lane of the reconfigurable computing unit to generate a right-shifted first array of single-bit mask elements, and selecting either the zero value or the feature element of the first set of N feature elements received by the lane for the corresponding feature element of the third set of N feature elements, based on a Least Significant Bit (LSB) of the right-shifted first array of single-bit mask elements; and

during a second dropout cycle that includes the second vector bus cycle, right-shifting the second array of N single-bit mask elements by a number of bits corresponding to the lane number of the respective lane of the reconfigurable computing unit to generate a right-shifted second array of single-bit mask elements, and selecting either the zero value or the feature element of the second set of N feature elements received by the respective lane for the corresponding feature element of the fourth set of N feature elements, based on a LSB of the right-shifted second array of single-bit mask elements;

wherein the N lanes of the reconfigurable computing unit are numbered from 0 to N-1.

17. The non-transitory computer readable storage medium of claim 15 , wherein:

the first set of N feature elements and the second set of N feature elements are interleaved in the first tensor so that the first set of N feature elements has even feature elements of the first tensor and the second set of N feature elements has odd feature elements of the first tensor;

the first array of N single bit mask elements consists of consecutively arranged even single-bit mask elements and the second array of N single bit mask elements consists of consecutively arranged odd single-bit mask elements; and

said receiving of the first set of N feature elements and said receiving of the second set of N feature elements occur during a single vector bus cycle, wherein each of the N lanes of the reconfigurable computing unit receive an even feature element and an odd feature element of the first tensor;

the method, at each of the N lanes of the reconfigurable computing unit, further comprising:

right-shifting both the first array of N single bit mask elements and the second array of N single bit mask elements by a number of bits based on a lane number of a respective lane of the N lanes of the reconfigurable computing unit to generate a right-shifted even array of single-bit mask elements and a right-shifted odd array of single-bit mask elements;

selecting, for a corresponding feature element of the third set of N feature elements, either the zero value or the received even feature element of the first set of N feature elements based on a Least Significant Bit (LSB) of the right-shifted even array of single-bit mask elements;

selecting, for a corresponding feature element of the fourth set of N feature elements, either the zero value or the received odd feature element of the second set of N feature elements based on a LSB of the right-shifted odd array of single-bit mask elements.

18. The non-transitory computer readable storage medium of claim 14 , the method further comprising:

obtaining the first tensor from a first layer of a neural network; and

propagating the second tensor to a second layer of the neural network.

19. A data processing system, comprising a reconfigurable computing unit having N lanes, coupled to one or more buses, and configured to:

receive a first array of N mask bits of a compressed mask (where N is a positive integer) over the one or more buses at each of the N lanes of the reconfigurable computing unit;

respectively receive a feature element of a first set of N feature elements of a first tensor at the N lanes of the reconfigurable computing unit over the one or more buses; and

selectively dropout feature elements from the first set of N feature elements of the first tensor to generate a third set of N feature elements to use as at least part of a second tensor by selecting either a zero value or a feature element of the first set of N feature elements received at a lane of the N lanes of the reconfigurable computing unit for a corresponding feature element of the third set of N feature elements, based on a single mask bit of the first array of N mask bits selected based on the lane.

20. The data processing system of claim 19 , further comprising:

a first layer of a neural network; and

a second layer of the neural network;

the reconfigurable computing unit further configured to:

receive the first tensor from the first layer of the neural network; and

propagate the second tensor to the second layer of the neural network.

21. The data processing system of claim 19 , wherein the first tensor also includes a second set of N feature elements, the compressed mask also includes a second array of N mask bits, and the reconfigurable computing unit is further configured to:

receive the second array of N mask bits over the one or more buses at each of the N lanes of the reconfigurable computing unit;

respectively receive a feature element of the second set of N feature elements of the first tensor at the N lanes of the reconfigurable computing unit;

selectively dropout feature elements from the second set of N feature elements of the first tensor to generate a fourth set of N feature elements to use as a part of the second tensor by selecting either the zero value or a feature element of the second set of N feature elements received at the lane of the N lanes of the reconfigurable computing unit for a corresponding feature element of the fourth set of N feature elements, based on a single mask bit of the second array of N mask bits selected based on the lane.

22. The data processing system of claim 21 , wherein the first set of N feature elements are adjacent feature elements in the first tensor and the second set of N feature elements are adjacent feature elements in the first tensor separate from the first set of N feature elements, and the lane of the N lanes of the reconfigurable computing unit is further configured to:

during a first dropout cycle that includes the receiving of the first set of N feature elements, right-shift the first array of N mask bits by a number of bits corresponding to a lane number of the lane to generate a right-shifted first array of mask bits, and selecting either the zero value or the feature element of the first set of N feature elements received by the lane for the corresponding feature element of the third set N of feature elements based on a Least Significant Bit (LSB) of the right-shifted first array of mask bits; and

during a second dropout cycle that includes the receiving of the second set of N feature elements, right-shift the second array of N mask bits by the number of bits corresponding to the lane number of the lane to generate a right-shifted second array of mask bits, and selecting either the zero value or the feature element of the second set of N feature elements received by the lane for the corresponding feature element of the fourth set of N feature elements, based on a LSB of the right-shifted second array of mask bits;

wherein the N lanes of the reconfigurable computing unit are numbered from 0 to N-1.

23. The data processing system of claim 21 , wherein:

the first set of N feature elements and the second set of N feature elements are interleaved in the first tensor so that the first set of N feature elements has even feature elements of the first tensor and the second set of N feature elements has odd feature elements of the first tensor;

the first array of N mask bits consists of consecutively arranged even mask bits and the second array of N mask bits consists of consecutively arranged odd mask bits; and

the lane of the N lanes of the reconfigurable computing unit is further configured to:

right-shift both the first array of N mask bits and the second array of N mask bits by a number of bits based on a lane number of the lane of the N lanes of the reconfigurable computing unit to generate a right-shifted even array of mask bits and a right-shifted odd array of mask bits;

select, for the corresponding feature element of the third set of N feature elements, either the zero value or the received even feature element of the first set of N feature elements based on a Least Significant Bit (LSB) of the right-shifted even array of mask bits; and

select, for the corresponding feature element of the fourth set of feature elements, either the zero value or the received odd feature element of the second set of N feature elements based on a LSB of the right-shifted odd array of mask bits.

24. The data processing system of claim 19 , wherein the reconfigurable computing unit comprises a compute unit of a Coarse Grain Reconfigurable Architecture (CGRA) processor.

25. The data processing system of claim 19 , further comprising:

a reconfigurable memory, coupled to the one or more buses, to store the compressed mask; and

a host processor coupled to the reconfigurable computing unit and the reconfigurable memory, the host processor configured to:

generate a mask comprising at least N mask elements, wherein each mask element of the at least N mask elements has either a first value or a second value that are each represented using a plurality of bits;

respectively compress N mask elements of the at least N mask elements to generate a corresponding single-bit mask element of the first array of N single-bit mask elements of the compressed mask; and

store the compressed mask in the reconfigurable memory unit.