Lossless tiling in convolution networks—padding and re-tilling at section boundaries
Disclosed is a method that includes generating by an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor. The plurality of output tiles of the output tensor is written in a memory, where the writing includes zero-padding the plurality of output tiles of the output tensor in the memory. The zero-padded plurality of output tiles of the output tensor are tiled, to generate a plurality of input tiles of an input tensor. The plurality of input tiles of the input tensor is processed in a second section of the processing graph.
1 . A non-transitory computer readable storage medium impressed with computer program instructions that, when executed on a processor, implement a method comprising:
generating by an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor, individual output tiles in the plurality of output tiles having a first size;
writing the plurality of output tiles of the output tensor in a memory, wherein the writing comprises zero-padding the plurality of output tiles of the output tensor in the memory to create a zero-padded plurality of output tiles;
tiling the zero-padded plurality of output tiles of the output tensor to generate a plurality of input tiles of an input tensor, individual input files in the plurality of input tiles having a second size that is larger than the first size; and
processing the plurality of input tiles of the input tensor in a second section of the processing graph.
2 . The non-transitory computer readable storage medium of claim 1 , further comprising:
initializing a plurality of memory locations to zero, the plurality of memory locations including (i) a first subset of memory locations, and (ii) a second subset of memory locations surrounding the first subset of memory locations,
wherein writing the plurality of output tiles comprises writing the plurality of output tiles of the output tensor in the first subset of memory locations in the memory, and
wherein the plurality of output tiles in the first subset of memory locations is surrounded by zeros in the second subset of memory locations.
3 . The non-transitory computer readable storage medium of claim 2 , wherein the zero-padded plurality of output tiles of the output tensor comprises an aggregate of (i) the plurality of output tiles in the first subset of memory locations and (ii) the zeros in the second subset of memory locations.
4 . The non-transitory computer readable storage medium of claim 2 , wherein writing the plurality of output tiles further comprises:
sequentially writing the plurality of output tiles of the output tensor in the first subset of memory locations in the memory, such that a first output tile of the plurality of output tiles of the output tensor is written to a first section of the first subset of memory locations, followed by writing of a second output tile of the plurality of output tiles of the output tensor to a second section of the first subset of memory locations.
5 . The non-transitory computer readable storage medium of claim 2 , wherein writing the plurality of output tiles further comprises:
at least in part parallelly writing the plurality of output tiles of the output tensor in the first subset of memory locations in the memory, such that a first output tile of the plurality of output tiles of the output tensor is written to a first section of the first subset of memory locations at least in part simultaneously with writing of a second output tile of the plurality of output tiles of the output tensor to a second section of the first subset of memory locations.
6 . The non-transitory computer readable storage medium of claim 2 , wherein tiling the zero-padded plurality of output tiles of the output tensor comprises:
tiling a combination of (i) the plurality of output tiles of the output tensor in the first subset of memory locations and (ii) the zeros in the second subset of memory locations surrounding the plurality of output tiles of the output tensor.
7 . The non-transitory computer readable storage medium of claim 1 , wherein:
one or more first input tiles of the plurality of input tiles of the input tensor have zero padding along one or more edges, and one or more second input tiles of the plurality of input tiles of the input tensor do not have zero padding along any edge.
8 . The non-transitory computer readable storage medium of claim 7 , wherein:
the one or more first input tiles of the plurality of input tiles of the input tensor have zero padding along those edges that coincide with edges of the input tensor.
9 . The non-transitory computer readable storage medium of claim 1 , wherein:
a first input tile of the plurality of input tiles of the input tensor has zero padding only on a top edge and a left edge;
a second input tile of the plurality of input tiles of the input tensor has zero padding only on a top edge and a right edge;
a third input tile of the plurality of input tiles of the input tensor has zero padding only on a bottom edge and a left edge; and
a fourth input tile of the plurality of input tiles of the input tensor has zero padding only on a bottom edge and a right edge.
10 . The non-transitory computer readable storage medium of claim 9 , wherein:
a fifth input tile of the plurality of input tiles of the input tensor has zero padding only on a top edge;
a sixth input tile of the plurality of input tiles of the input tensor has zero padding only on a right edge;
a seventh input tile of the plurality of input tiles of the input tensor has zero padding only on a bottom edge;
an eighth input tile of the plurality of input tiles of the input tensor has zero padding only on a left edge; and
a ninth input tile of the plurality of input tiles of the input tensor does not have any zero padding of any of its edges.
11 . The non-transitory computer readable storage medium of claim 1 , wherein:
the plurality of output tiles of the output tensor is non-overlapping tiles; and
the plurality of input tiles of the input tensor is overlapping tiles.
12 . A computer implemented method, comprising:
generating by an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor, individual output tiles in the plurality of output tiles having a first size;
writing the plurality of output tiles of the output tensor in a memory, wherein the writing comprises zero-padding the plurality of output tiles of the output tensor in the memory to create a zero-padded plurality of output tiles;
tiling the zero-padded plurality of output tiles of the output tensor to generate a plurality of input tiles of an input tensor, individual input files in the plurality of input tiles having a second size that is larger than the first size; and
processing the plurality of input tiles of the input tensor in a second section of the processing graph.
13 . The method of claim 12 , further comprising:
initializing a plurality of memory locations to zero, the plurality of memory locations including (i) a first subset of memory locations, and (ii) a second subset of memory locations surrounding the first subset of memory locations,
wherein writing the plurality of output tiles comprises writing the plurality of output tiles of the output tensor in the first subset of memory locations in the memory, and
wherein the plurality of output tiles in the first subset of memory locations is surrounded by zeros in the second subset of memory locations.
14 . The method of claim 12 , wherein:
one or more first input tiles of the plurality of input tiles of the input tensor have zero padding along one or more edges that coincide with edges of the input tensor, and one or more second input tiles of the plurality of input tiles of the input tensor do not have zero padding along any edge.
15 . The method of claim 12 , wherein:
a first input tile of the plurality of input tiles of the input tensor has zero padding only on a top edge and a left edge;
a second input tile of the plurality of input tiles of the input tensor has zero padding only on a top edge and a right edge;
a third input tile of the plurality of input tiles of the input tensor has zero padding only on a bottom edge and a left edge;
a fourth input tile of the plurality of input tiles of the input tensor has zero padding only on a bottom edge and a right edge;
a fifth input tile of the plurality of input tiles of the input tensor has zero padding only on a top edge;
a sixth input tile of the plurality of input tiles of the input tensor has zero padding only on a right edge;
a seventh input tile of the plurality of input tiles of the input tensor has zero padding only on a bottom edge;
an eighth input tile of the plurality of input tiles of the input tensor has zero padding only on a left edge; and
a ninth input tile of the plurality of input tiles of the input tensor does not have any zero padding of any of its edges.
16 . The method of claim 12 , wherein:
the plurality of output tiles of the output tensor is non-overlapping tiles; and
the plurality of input tiles of the input tensor is overlapping tiles.
17 . A data processing system, comprising:
runtime logic configured to:
generate, at an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor, individual output tiles in the plurality of output tiles having a first size,
write the plurality of output tiles of the output tensor in a memory, wherein the writing comprises zero-padding the plurality of output tiles of the output tensor in the memory to create a zero-padded plurality of output tiles,
tile the zero-padded plurality of output tiles of the output tensor to generate a plurality of input tiles of an input tensor, individual input files in the plurality of input tiles having a second size that is larger than the first size, and
process the plurality of input tiles of the input tensor in a second section of the processing graph.
18 . The data processing system of claim 17 , wherein the runtime logic is further configured to:
initialize a plurality of memory locations to zero, the plurality of memory locations including (i) a first subset of memory locations, and (ii) a second subset of memory locations surrounding the first subset of memory locations; and
write the plurality of output tiles of the output tensor in the first subset of memory locations in the memory,
wherein the plurality of output tiles in the first subset of memory locations is surrounded by zeros in the second subset of memory locations.
19 . The data processing system of claim 18 , wherein:
the first section and the second section of the processing graph are executed in one or more processing units that are in an Integrated Circuit (IC) chip; and
the memory is external to the IC chip.
20 . The data processing system of claim 17 , wherein:
the plurality of output tiles of the output tensor is non-overlapping tiles; and
the plurality of input tiles of the input tensor is overlapping tiles.