Method, accelerator, and electronic device with tensor processing

1 . A processor-implemented tensor processing method, comprising:

receiving a request to process a neural network including a normalization layer by an accelerator; and

generating an instruction executable by the accelerator in response to the request,

wherein, by executing the instruction, the accelerator is configured to determine an intermediate tensor corresponding to a result of a portion of operations of the normalization layer, by performing, in a channel axis direction, a convolution based on an input tensor and a kernel such that an element in the intermediate tensor is determined based on elements of a plurality of channels of the input tensor,

wherein the input tensor is of the normalization layer, a number of input channels of the kernel is determined based on the input tensor, and at least a portion of scaling values of elements of the kernel are determined based on an inverse of a number of elements included in a same channel of the input tensor,

wherein diagonal elements of the kernel have a first scaling value, and remaining elements of the kernel have a second scaling value different from the first scaling value,

wherein the convolution is performed between the kernel and a transformed input tensor transformed such that elements included in a same channel of the input tensor are arranged in the channel axis direction,

wherein the intermediate tensor is determined by transforming elements determined as a result of the convolution to a same form as the input tensor, and

wherein the transforming of the determined elements comprises transforming the determined elements such that a number of channels of the intermediate tensor is equal to the number of channels of the input tensor.

2 . The method of claim 1 , wherein

the intermediate tensor is determined by subtracting an average value of one or more elements of the input tensor from a value of each of the one or more elements through the convolution, and

an output tensor corresponding to an output of the normalization layer is determined based on the intermediate tensor.

3 . The method of claim 1 , wherein

the number of input channels and a number of output channels of the kernel are equal to the number of channels of the input tensor.

4 . The method of claim 1 , wherein

the number of input channels and a number of output channels of the kernel are equal to the number of elements included in the same channel of the input tensor.

5 . The method of claim 1 , wherein

the number of input channels of the kernel is equal to the number of channels of the input tensor, and

the scaling values of the elements of the kernel correspond to the inverse of the number of channels of the input tensor.

6 . The method of claim 1 , wherein the intermediate tensor is determined based on an average value of elements included in the input tensor.

7 . The method of claim 6 , wherein elements of the intermediate tensor are respectively determined by averaging elements included in either one of:

a same column in each channel of the input tensor; and

the same channel of the input tensor.

8 . The method of claim 1 , wherein

the number of input channels of the kernel is equal to the number of elements included in the same channel of the input tensor, and

scaling values of the elements of the kernel correspond to the inverse of the number of elements included in the same channel.

9 . The method of claim 1 , wherein the convolution is performed in the accelerator such that the input tensor is not transmitted externally from the accelerator for performing an operation according to the normalization layer.

10 . The method of claim 1 , wherein the accelerator is included in either one or both of a user terminal configured to receive data to be inferred using the neural network, and a server configured to receive the data to be inferred from the user terminal.

11 . The method of claim 1 , wherein the receiving of the request comprises receiving the request by a host processor, and the generating of the instruction comprises generating the instruction by the host processor.

12 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 .

13 . An accelerator, comprising:

one or more processors configured to:

obtain an input tensor of a normalization layer included in a neural network,

obtain a kernel having a number of input channels determined based on the input tensor and including at least a portion of elements of scaling values are determined based on an inverse of a number of elements included in a same channel of the input tensor, and

determine an intermediate tensor corresponding to a result of a portion of operations of the normalization layer, by performing, in a channel axis direction, a convolution which is based on the input tensor and the kernel such that an element in the intermediate tensor is determined based on elements of a plurality of channels of the input tensor,

wherein diagonal elements of the kernel have a first scaling value, and remaining elements of the kernel have a second scaling value different from the first scaling value,

wherein the convolution is performed between the kernel and a transformed input tensor transformed such that elements included in a same channel of the input tensor are arranged in the channel axis direction,

wherein the intermediate tensor is determined by transforming elements determined as a result of the convolution to a same form as the input tensor, and

wherein the transforming of the determined elements comprises transforming the determined elements such that a number of channels of the intermediate tensor is equal to the number of channels of the input tensor.

14 . The accelerator of claim 13 , wherein

the one or more processors are configured to determine the intermediate tensor by subtracting an average value of one or more elements of the input tensor from a value of each of the one or more elements through the convolution, and

an output tensor corresponding to an output of the normalization layer is determined based on the intermediate tensor.

15 . The accelerator of claim 13 , wherein

the number of input channels and a number of output channels of the kernel are equal to the number of channels of the input tensor.

16 . The accelerator of claim 13 , wherein

the number of input channels and a number of output channels of the kernel are equal to the number of elements included in the same channel of the input tensor.

17 . The accelerator of claim 13 , wherein

the number of input channels of the kernel is equal to the number of channels of the input tensor, and

the scaling values of the elements of the kernel correspond to an inverse of the number of channels of the input tensor.

18 . The accelerator of claim 13 , wherein

the number of input channels of the kernel is equal to the number of elements included in the same channel of the input tensor, and

scaling values of the elements of the kernel correspond to an inverse of the number of elements included in the same channel.

19 . The accelerator of claim 13 , wherein the convolution is performed inside the accelerator such that the input tensor is not transmitted externally from the accelerator for performing an operation according to the normalization layer.

20 . An electronic device comprising:

a host processor configured to generate an instruction; and

the accelerator of claim 19 , wherein the accelerator is configured to, by executing the instruction, determine the intermediate tensor.

21 . An electronic device, comprising:

a host processor configured to generate an instruction executable by an accelerator in response to a request to process a neural network including a normalization layer by the accelerator; and

the accelerator configured to, by executing the instruction, determine an intermediate tensor corresponding to a result of a portion of operations of the normalization layer, by performing, in a channel axis direction, a convolution which is based on an input tensor and a kernel such that an element in the intermediate tensor is determined based on elements of a plurality of channels of the input tensor,

wherein the input tensor is of the normalization, a number of input channels of the kernel is determined based on the input tensor, and at least a portion of scaling values of elements of the kernel are determined based on an inverse of a number of elements included in a same channel of the input tensor,

wherein diagonal elements of the kernel have a first scaling value, and remaining elements of the kernel have a second scaling value different from the first scaling value,

wherein the convolution is performed between the kernel and a transformed input tensor transformed such that elements included in a same channel of the input tensor are arranged in the channel axis direction,

wherein the intermediate tensor is determined by transforming elements determined as a result of the convolution to a same form as the input tensor, and

wherein the transforming of the determined elements comprises transforming the determined elements such that a number of channels of the intermediate tensor is equal to the number of channels of the input tensor.

22 . The electronic device of claim 21 , wherein

the accelerator is configured to determine the intermediate tensor by subtracting an average value of one or more elements of the input tensor from a value of each of the one or more elements through the convolution, and

an output tensor corresponding to an output of the normalization layer is determined based on the intermediate tensor.