Efficient direct convolution using SIMD instructions

1. A system, comprising:

a processor configured to perform a convolution on a vector comprising a plurality of data lanes of multiple data bits, wherein to perform the convolution on the vector the processor is configured to execute a plurality of vector instructions, wherein each of the plurality of vector instructions is configured to perform an operation on a plurality of data lanes of multiple data bits, and wherein the plurality of vector instructions are configured to:

load into respective different vector registers of the processor:

three or more source vectors comprising:

a center vector comprising a contiguous plurality of data lanes of the vector starting with a first data lane of the vector;

a left vector comprising a contiguous plurality of data lanes of the vector starting one data lane prior to the first data lane of the vector; and

a right vector comprising a contiguous plurality of data lanes of the vector starting one data lane after the first data lane of the vector; and

a kernel vector comprising a plurality of weighting values including a center weighting value, a left weighting value and a right weighting value; and

subsequent to the loading, generate one or more output vectors respectively comprising weighted sums of the three or more source vectors loaded into the respective different vector registers, wherein to generate a particular output vector of the one or more output vectors a particular vector instruction of the plurality of vector instructions is configured to:

load a scalar weighting value from a lane of the kernel vector, the lane specified to the particular vector instruction according to the particular output vector and a particular source vector of the three or more source vectors, wherein the particular source vector is specified as an operand of the particular vector instruction;

scale respective ones of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector;

add the scaled vector to an accumulator vector stored in a vector accumulator to generate an output vector; and

store the output vector in the vector accumulator.

2. The system of claim 1 , wherein to load the left and right vectors the plurality of vector instructions are configured to:

load a previous vector immediately prior to the vector;

load a next vector immediately subsequent to the vector;

execute a vector extraction instruction to generate the left vector using the previous vector and the vector; and

execute another vector extraction instruction to generate the right vector using the vector and the next vector.

3. The system of claim 2 , wherein the vector extraction instruction is configured to:

concatenate a first vector register with a second vector register to produce an intermediate result;

shift the intermediate result left a number of lanes; and

generate an output vector comprising an upper portion of the intermediate result.

4. The system of claim 1 :

wherein to generate the particular output vector, the plurality of vector instructions comprise a vector scaling instruction for each one of the three or more source vectors, wherein each one of the three or more source vectors is configured to:

load a weighting value for the convolution from a specified lane of a register containing the kernel vector;

multiply each of a plurality of data lanes of a vector register specified as an operand of the vector scaling instruction by the weighting value to generate a scaled vector; and

add the scaled vector to the particular one of the one or more the output vectors.

5. The system of claim 1 , wherein the convolution on the vector implements a stride value, wherein a value of data lanes to omit in the convolution for each data lane to include is based on the stride value, and wherein to load the three or more source vectors the plurality of vector instructions are configured to:

load a previous vector immediately prior to the vector;

load a next vector immediately subsequent to the vector;

execute vector extraction instructions to generate first and second intermediate vectors using the previous vector, the vector and the next vector;

execute a vector shuffle instruction to load the left vector using the first and second intermediate vectors; and

execute vector shuffle instructions to load the center vector and right vector using the vector and next vector.

6. The system of claim 1 , wherein the convolution is a multi-dimensional convolution, and wherein the one or more output vectors comprises a plurality of output vectors.

7. The system of claim 1 , wherein the processor is configured to perform the convolution as part of a convolutional neural network.

8. A method, comprising:

performing, by a computer comprising a processor implementing vector instructions, a convolution on a vector comprising a plurality of data lanes of multiple data bits, comprising:

loading into different vector respective registers of the processor:

three or more source vectors comprising:

a center vector comprising a contiguous plurality of data lanes of the vector starting with a first data lane of the vector;

a left vector comprising a contiguous plurality of data lanes of the vector starting one data lane prior to the first data lane of the vector; and

a right vector comprising a contiguous plurality of data lanes of the vector starting one data lane after the first data lane of the vector; and

a kernel vector comprising a plurality of weighting values including a center weighting value, a left weighting value and a right weighting value; and

subsequent to the loading, generating one or more output vectors respectively comprising weighted sums of the three or more source vectors loaded into the respective different vector registers, wherein generating a particular output vector of the one or more output vectors comprises executing a particular vector instruction of the plurality of vector instructions to:

load a scalar weighting value from a lane of the kernel vector, the lane specified to the particular vector instruction according to the particular output vector and a particular source vector of the three or more source vectors, wherein the particular source vector is specified as an operand of the particular vector instruction;

scale respective ones of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector;

add the scaled vector to an accumulator vector stored in a vector accumulator to generate an output vector; and

store the output vector in the vector accumulator;

wherein the vector instructions operate on one or more data registers comprising a plurality of data lanes each comprising multiple data bits.

9. The method of claim 8 , wherein the loading of the left and right vectors comprises:

loading a previous vector immediately prior to the vector;

loading a next vector immediately subsequent to the vector;

executing a vector extraction instruction to generate the left vector using the previous vector and the vector; and

executing the vector extraction instruction to generate the right vector using the vector and the next vector.

10. The method of claim 9 , wherein executing the vector extraction instruction comprises:

concatenating a first vector register a second vector register to produce an intermediate result;

shifting the intermediate result left a number of lanes; and

generating an output vector comprising an upper portion of the intermediate result.

11. The method of claim 8 , wherein generating the particular output vector comprises executing for each one of the three or more source vectors a vector scaling instruction, wherein executing the vector scaling instruction comprises:

loading a weighting value from a specified lane of a register containing the kernel vector;

generating a scaled vector comprising respective data lanes of a vector register specified as an operand of the vector scaling instruction scaled by the weighting value; and

adding the scaled vector to the particular one of the one or more output vectors.

12. The method of claim 8 , wherein the convolution is a multi-dimensional convolution, and wherein the one or more output vectors comprises a plurality of output vectors.

13. The method of claim 8 , wherein the convolution is performed as part of a convolutional neural network.

14. A non-transitory computer-readable storage medium storing a plurality of vector instructions that when executed on a processor cause the processor to perform a convolution on a vector comprising:

loading into different vector respective registers of the processor:

three or more source vectors comprising:

a center vector comprising a contiguous plurality of data lanes of the vector starting with a first data lane of the vector;

a left vector comprising a contiguous plurality of data lanes of the vector starting one data lane prior to the first data lane of the vector; and

a right vector comprising a contiguous plurality of data lanes of the vector starting one data lane after the first data lane of the vector; and

kernel vector comprising a plurality of weighting values including a center weighting value, a left weighting value and a right weighting value; and

subsequent to the loading, generating one or more output vectors respectively comprising weighted sums of the three or more source vectors loaded into the respective different vector registers, wherein generating a particular output vector of the one or more output vectors comprises executing a particular vector instruction of the plurality of vector instructions to:

load a scalar weighting value from a lane of the kernel vector, the lane specified to the particular vector instruction according to the particular output vector and a particular source vector of the three or more source vectors, wherein the particular source vector is specified as an operand of the particular vector instruction;

scale respective ones of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector;

add the scaled vector to an accumulator vector stored in a vector accumulator to generate an output vector; and

store the output vector in the vector accumulator.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the loading of the left and right vectors comprises:

loading a previous vector immediately prior to the vector;

loading a next vector immediately subsequent to the vector;

executing a vector extraction instruction to generate the left vector using the previous vector and the vector; and

executing the vector extraction instruction to generate the right vector using the vector and the next vector.

16. The non-transitory computer-readable storage medium of claim 15 , wherein executing the vector extraction instruction comprises:

concatenating a first vector register a second vector register to produce an intermediate result;

shifting the intermediate result left a number of lanes; and

generating an output vector comprising an upper portion of the intermediate result.

17. The non-transitory computer-readable storage medium of claim 14 , wherein generating the particular output vector comprises executing for each one of the three or more source vectors a vector scaling instruction to perform:

loading a weighting value from a specified lane of a register containing the kernel vector;

generating a scaled vector comprising respective data lanes of a vector register specified as an operand of the vector scaling instruction scaled by the weighting value; and

adding the scaled vector to the particular one of the one or more output vectors.

18. The non-transitory computer-readable storage medium of claim 14 , wherein the convolution on the vector implements a stride value, wherein a value of data lanes to omit in the convolution for each data lane to include is based on the stride value, and wherein the loading of the three or more source vectors comprises:

loading a previous vector immediately prior to the vector;

loading a next vector immediately subsequent to the vector;

executing vector extraction instructions to generate first and second intermediate vectors using the previous vector, the vector and the next vector;

executing a vector shuffle instruction to load the left vector using the first and the second intermediate vectors; and

executing vector shuffle instructions to load the center vector and right vector using the vector and next vector.

19. The non-transitory computer-readable storage medium of claim 14 , wherein the convolution is a multi-dimensional convolution, and wherein the one or more output vectors comprises a plurality of output vectors.

20. The non-transitory computer-readable storage medium of claim 14 , wherein the convolution is part of a convolutional neural network.