Computer-implemented data processing method, micro-controller system and computer program product for applying pooling with respect to an output buffer in a neural network
A computer-implemented method applies a pooling operator to an input array of data, the pooling operator having an absorbing element value and a set of pooling parameters. A size of an output buffer is computer as a function of the set of pooling parameters. The elements of the output buffer are initialized to the value of the absorbing element of the pooling operator. The output array of data is generated by, for a plurality of iterations associated with respective pooling windows: associating, as a function of the pooling parameters, elements of the input array of a pooling window with output elements of the output buffer; and combining, for each output element of the output buffer, the respective input elements associated with the output element. The combining may include determining a combination of respective elements of the output buffer with the input elements associated with the output elements.
1. A computer-implemented method for neural network processing, comprising:
applying a pooling operator to an input array of data to the neural network, the pooling operator having an absorbing element value and a set of pooling parameters, the applying the pooling operator to the input array of data comprising:
computing a size of an output buffer as a function of the set of pooling parameters, wherein the output buffer is a memory buffer;
initializing elements of the output buffer to the value of the absorbing element of the pooling operator; and
generating an output array of data stored in the output buffer, the generating the output array of data including, for a plurality of iterations associated with respective pooling windows:
associating, as a function of the pooling parameters, elements of the input array of a pooling window with output elements of the output buffer; and
combining, for each output element of the output buffer, the respective input elements associated with the output element; and
generating an output of the neural network based on a result of the combining of the respective input elements associated with the output element, wherein,
the input array has an array size and is indexed;
the associating elements of the input array comprises computing range limit values comprising a starting index and a termination index of the pooling window; and
the starting index x start and the terminating index x end are determined according to:
x
start
=
max
(
⌊
x
i
+
p
+
s
-
c
s
⌋
,
0
)
,
x
e
n
d
=
min
(
⌊
x
i
+
p
s
⌋
,
size
-
1
)
wherein x i is the i-th index of the input array, p is a pooling pad size, c is a pooling window size, and s is a stride size.
2. The computer-implemented method of claim 1 , wherein the combining includes determining a combination of respective elements of the output buffer with the input elements associated with the output elements.
3. The computer-implemented method of claim 1 , wherein the combining includes applying a weight to an input element.
4. The method of claim 1 , wherein the pooling operator is selected from a set of pooling operators, and the set of pooling operators comprises at least one of:
a max pooling operator, having an absorbing element tending towards a minimum bottom end of a numeric representation interval; and
an average pooling operator, having an absorbing element of zero and a normalization factor equal to an input array size.
5. The method of claim 1 , wherein the set of pooling parameters comprise at least one of a pooling window size, a pooling stride size and a pooling pad size.
6. The method of claim 1 , comprising applying a normalization operator to the output array of data.
7. The method of claim 1 , wherein computing the size of the output buffer as a function of said set of pooling parameters comprises selecting:
a first output buffer size d of according to:
d of =max(┌( d i −c+p left 30 p right )/ s ┐, 1 )
a second output buffer size d oc according to:
d oc =┌( d i −c+p left +p right )/ s┐+ 1; or
a third output buffer size d op when padding size is zero, according to:
d op =┌( d i +s −1)/ s┐+ 1
where c is a pooling window size, s is a stride size, p left is a pad size on a left edge, p right is a pad size on a right edge, d i is an input array length-size.
8. The method of claim 1 , comprising:
applying artificial neural network processing to said input array, the pooling operator being applied in a pooling data processing layer, or a convolution data processing layer.
9. A micro-controller for neural network processing, comprising:
memory; and
processing circuitry coupled to the memory, wherein the processing circuitry, in operation, applies a pooling operator to an input array of data of the neural network, the pooling operator having an absorbing element value and a set of pooling parameters, the applying the pooling operator to the input array of data comprising:
allocating a portion of the memory to an output buffer, the output buffer being a memory buffer and having a size that is a function of the set of pooling parameters;
initializing elements of the output buffer to the value of the absorbing element of the pooling operator; and
generating an output array of data stored in the output buffer, the generating the output array of data including, for a plurality of iterations associated with respective pooling windows:
associating, as a function of the pooling parameters, elements of the input array of a pooling window with output elements of the output buffer; and
combining, for each output element of the output buffer, the respective input elements associated with the output element; and
generates an output of the neural network based on a result of the combining of the respective input elements associated with the output element, wherein
the input array has an array size and is indexed;
the associating elements of the input array comprises computing range limit values comprising a starting index and a termination index of the pooling window; and
the starting index x start and the terminating index x end are determined according to:
x
start
=
max
(
⌊
x
i
+
p
+
s
-
c
s
⌋
,
0
)
,
x
e
n
d
=
min
(
⌊
x
i
+
p
s
⌋
,
size
-
1
)
wherein x i is an i-th index of the input array, p is a pooling pad size, c is a pooling window size, and s is a stride size.
10. The micro-controller of claim 9 , wherein the combining includes determining a combination of respective elements of the output buffer with the input elements associated with the output elements.
11. The micro-controller of claim 9 , comprising an interface configured to couple the micro-controller to other processing units or actuating devices.
12. The micro-controller of claim 9 , comprising a bus system coupling the processing circuitry and the memory to exchange data therebetween.
13. The micro-controller of claim 9 , wherein the pooling operator is selected from a set of pooling operators, and the set of pooling operators comprises at least one of:
a max pooling operator, having an absorbing element tending towards a minimum bottom end of a numeric representation interval; and
an average pooling operator, having an absorbing element of zero and a normalization factor equal to an input array size.
14. The micro-controller of claim 9 , wherein the set of pooling parameters comprise at least one of a pooling window size, a pooling stride size and a pooling pad size.
15. The micro-controller of claim 9 , wherein the processing circuitry, in operation, applies a normalization operator to the output array of data.
16. The micro-controller of claim 9 , wherein computing the size of the output buffer as a function of said set of pooling parameters comprises selecting:
a first output buffer size d of according to:
d of =max(┌( d i −c+p left +p right )/ s┐, 1)
a second output buffer size d oc according to:
d oc =┌( d i −c+p left +p right )/ s┐+ 1; or
a third output buffer size d op when padding size is zero, according to:
d op =┌( d i +s− 1)/ s┐+ 1
where c is a pooling window size, s is a stride size, p left is a pad size on a left edge, p right is a pad size on a right edge, d i is an input array length-size.
17. A non-transitory computer-readable storage medium whose stored contents configure a computing system to implement a method for neural network processing, the method comprising:
applying a pooling operator to an input array of data of a neural network, the pooling operator having an absorbing element value and a set of pooling parameters, the applying the pooling operator to the input array of data comprising:
computing a size of an output buffer as a function of the set of pooling parameters, wherein the output buffer is a memory buffer;
initializing elements of the output buffer to the value of the absorbing element of the pooling operator; and
generating an output array of data stored in the output buffer, the generating the output array of data including, for a plurality of iterations associated with respective pooling windows:
associating, as a function of the pooling parameters, elements of the input array of a pooling window with output elements of the output buffer; and
combining, for each output element of the output buffer, the respective input elements associated with the output element; and
generating an output of the neural network based on a result of the combining of the respective input elements associated with the output element, wherein,
the input array has an array size and is indexed;
the associating elements of the input array comprises computing range limit values comprising a starting index and a termination index of the pooling window; and
the starting index x start and the terminating index x end are determined according to:
x
start
=
max
(
⌊
x
i
+
p
+
s
-
c
s
⌋
,
0
)
,
x
end
=
min
(
⌊
x
i
+
p
s
⌋
,
size
-
1
)
wherein x i is an i-th index of the input array, p is a pooling pad size, c is a pooling window size, and s is a stride size.
18. The non-transitory computer-readable storage medium of claim 17 ,
wherein the combining includes determining a combination of respective elements of the output buffer with the input elements associated with the output elements.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the pooling operator is selected from a set of pooling operators, and the set of pooling operators comprises at least one of:
a max pooling operator; and
an average pooling operator, having an absorbing element and a normalization factor equal to input array size.
20. The non-transitory computer-readable storage medium of claim 17 , wherein the set of pooling parameters comprise at least one of a pooling window size, a pooling stride size and a pooling pad size.
21. The non-transitory computer-readable medium of claim 17 , wherein the
contents comprise instructions, which when executed by the computing system, cause the computing system to perform the method.