Method and apparatus for tensor processing
A data processing method and apparatus are described. The data processing apparatus obtains an input tensor corresponding to input data. The data processing apparatus determines M 1 first-type tensor blocks and M 2 second-type tensor blocks. P processing units in the data processing apparatus process the M tensor blocks concurrently. In a first time period, all of the tensor blocks that are processed concurrently by the P processing units are first-type tensor blocks. In a second time period, all of the tensor blocks that are processed concurrently by the P processing units are second-type tensor blocks.
1 . A data processing method, comprising:
obtaining an input tensor corresponding to input data, wherein the input tensor is an N-dimensional tensor, N is a positive integer greater than or equal to 1, the input tensor consists of M tensor blocks, and M is a positive integer greater than or equal to 2; and
processing, by P processing units, M tensor blocks, comprising:
processing, in a first time period by P processing units in parallel, a part of tensor blocks in the M tensor blocks, each tensor block of the part of tensor blocks comprises a first type of tensor block,
processing, in a second time period by the P processing units in parallel, another part of tensor blocks in the M tensor blocks tensor blocks, each tensor block of the another part of tensor blocks comprises a second type of tensor block, and the first type and second type are different, and
the first time period does not overlap the second time period.
2 . The method according to claim 1 , wherein the part of tensor blocks in the M tensor blocks is M 1 first tensor block indexes, and the another part of tensor blocks in the M tensor blocks tensor blocks is M 2 second tensor block indexes, and M 1 and M 2 are positive integers greater than or equal to 1.
3 . The method according to claim 2 , wherein the M 1 first tensor block indexes are stored in a first storage space, and the M 2 second tensor block indexes are stored in a second storage space.
4 . The method according to claim 2 , wherein before the processing, the M tensor blocks, the method further comprises:
determining the M 1 first tensor block indexes and the M 2 second tensor block indexes, wherein the M 1 first tensor block indexes respectively indicate M 1 first-type tensor blocks, the M 2 second tensor block indexes respectively indicate M 2 second-type tensor blocks;
saving the M 1 first tensor block indexes in a first storage space, and saving the M 2 second tensor block indexes in a second storage space; and
obtaining the M 1 first tensor block indexes from the first storage space, and obtaining the M 2 second tensor block indexes from the second storage space; and the processing, the M tensor blocks concurrently further comprises:
processing, by the P processing units, the M tensor blocks based on the M 1 first tensor block indexes and the M 2 second tensor block indexes, wherein the M 1 first-type tensor blocks that are processed concurrently by the P processing units in the first time period are determined from the M tensor blocks based on the M 1 first tensor block indexes, and the M 2 second-type tensor blocks that are processed concurrently by the P processing units in the second time period are determined from the M tensor blocks based on the M 2 second tensor block indexes.
5 . The method according to claim 3 , wherein a sequence of each of the M 1 first tensor block indexes in the first storage space is the same as a sequence of first-type tensor blocks indicated by a first tensor block index in the M tensor blocks; and
a sequence of each of the M 2 second tensor block indexes in the second storage space is the same as a sequence of second-type tensor blocks indicated by a second tensor block index in the M tensor blocks.
6 . The method according to claim 1 , wherein before the processing, by the P processing units, the M tensor blocks, the method further comprises:
determining a first correspondence between the P processing units and the first-type tensor blocks;
determining a second correspondence between the P processing units and the second-type tensor blocks; and
the processing, the M tensor blocks comprises:
processing, by the P processing units, the M tensor blocks concurrently based on the first correspondence and the second correspondence.
7 . The method according to claim 2 , wherein the method further comprises:
determining tensor block attribute indication information comprising M pieces of indication information that respectively indicate tensor block attributes of the M tensor blocks; and
determining M tensor block indexes that respectively indicate the M tensor blocks; and
the determining M 1 first tensor block indexes and M 2 second tensor block indexes comprises:
determining the M 1 first tensor block indexes and the M 2 second tensor block indexes from the M tensor block indexes based on the M pieces of indication information.
8 . The method according to claim 7 , wherein the determining the M 1 first tensor block indexes and the M 2 second tensor block indexes from the M tensor block indexes based on the M pieces of indication information comprises:
determining first location information based on the M pieces of indication information, wherein the first location information indicates a location of each piece of first indication information in the M pieces of indication information in M 1 pieces of first indication information, and a tensor block attribute indicated by the M 1 pieces of first indication information is a first attribute;
determining second location information based on the M pieces of indication information, wherein the second location information indicates a location of each piece of second indication information in the M pieces of indication information in M 2 pieces of second indication information, and a tensor block attribute indicated by the M 2 pieces of second indication information is a second attribute;
determining the M 1 first tensor block indexes based on the first location information; and
determining the M 2 second tensor block indexes based on the second location information.
9 . A system, comprising:
P processing units comprised in one or more processors, wherein P is a positive integer greater than or equal to 2; and
a memory, coupled to the one or more processors and configured to store a plurality of instructions that, when executed by the processor, causes the processor to:
obtaining, an input tensor corresponding to input data, wherein the input tensor is an N-dimensional tensor, the input tensor consists of M tensor blocks, N is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to 2,
processing, in a first time period by the P processing units in parallel, a part of tensor blocks in the M tensor blocks, each tensor block of the part of tensor blocks comprises a first type of tensor block,
processing, in a second time period by the P processing units in parallel, another part of tensor blocks in the M tensor blocks tensor blocks, each tensor block of the another part of tensor blocks comprises a second type of tensor block, and the first type and second type are different, and
the first time period does not overlap the second time period.
10 . The system according to claim 9 , wherein the part of tensor blocks in the M tensor blocks is M 1 first tensor block indexes, and the another part of tensor blocks in the M tensor blocks tensor blocks is M 2 second tensor block indexes, and M 1 and M 2 are positive integers greater than or equal to 1, the memory is further configured to save the M 1 first tensor block indexes and the M 2 second tensor block indexes, wherein the M 1 tensor block indexes are saved in a first storage space of the memory, and the M 2 second tensor block indexes are saved in a second storage space of the memory.
11 . The system according to claim 10 , the P processing units are further configured to: obtain the M 1 first tensor block indexes from the first storage space, obtain the M 2 second tensor block indexes from the second storage space, and process the M tensor blocks concurrently based on the M 1 first tensor block indexes and the M 2 second tensor block indexes, wherein M 1 first-type tensor blocks that are processed concurrently by the P processing units in the first time period are determined from the M tensor blocks based on the M 1 first tensor block indexes, and M 2 second-type tensor blocks that are processed concurrently by the P processing units in the second time period are determined from the M tensor blocks based on the M 2 second tensor block indexes.
12 . The system according to claim 10 , wherein a sequence of each of the M 1 first tensor block indexes in the first storage space is the same as a sequence of first-type tensor blocks indicated by a first tensor block index in the M tensor blocks; and
a sequence of each of the M 2 second tensor block indexes in the second storage space is the same as a sequence of second-type tensor blocks indicated by a second tensor block index in the M tensor blocks.
13 . The system according to claim 10 , wherein
the one or more processors are further configured to determine a first correspondence and a second correspondence, wherein the first correspondence is a correspondence between the P processing units and first-type tensor blocks, and the second correspondence is a correspondence between the P processing units and second-type tensor blocks; and
the P processing units are further configured to process the M tensor blocks concurrently based on the first correspondence and the second correspondence.
14 . The system according to claim 10 , wherein the processor is further configured to:
determine tensor block attribute indication information comprising M pieces of indication information that respectively indicate tensor block attributes of the M tensor blocks;
determine M tensor block indexes that respectively indicate the M tensor blocks; and
determine the M 1 first tensor block indexes and the M 2 second tensor block indexes from the M tensor block indexes based on the M pieces of indication information.
15 . The system according to claim 14 , wherein the processor is further configured to:
determine first location information based on the M pieces of indication information, wherein the first location information indicates a location of each piece of first indication information in the M pieces of indication information in M 1 pieces of first indication information, and a tensor block attribute indicated by the M 1 pieces of first indication information is a first attribute;
determine second location information based on the M pieces of indication information, wherein the second location information indicates a location of each piece of second indication information in the M pieces of indication information in M 2 pieces of second indication information, and a tensor block attribute indicated by the M 2 pieces of second indication information is a second attribute;
determine the M 1 first tensor block indexes based on the first location information; and
determine the M 2 second tensor block indexes based on the second location information.
16 . A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer-executable instructions when executed by P processing units of a system, cause the system to:
obtain an input tensor corresponding to input data, wherein the input tensor is an N-dimensional tensor, N is a positive integer greater than or equal to 1, the input tensor consists of M tensor blocks, and M is a positive integer greater than or equal to 2; and
process, by the P processing units, M tensor blocks, comprising the system to:
process, in a first time period by the P processing units in parallel, a part of tensor blocks in the M tensor blocks, each tensor block of the part of tensor blocks comprises a first type of tensor block,
process, in a second time period by the P processing units in parallel, another part of tensor blocks in the M tensor blocks tensor blocks, each tensor block of the another part of tensor blocks comprises a second type of tensor block, and the first type and second type are different, and
the first time period does not overlap the second time period.
17 . The non-transitory computer-readable storage medium according to claim 16 , wherein the part of tensor blocks in the M tensor blocks is M 1 first tensor block indexes, and the another part of tensor blocks in the M tensor blocks tensor blocks is M 2 second tensor block indexes, and M 1 and M 2 are positive integers greater than or equal to 1, wherein the M 1 first tensor block indexes are stored in a first storage space, and the M 2 second tensor block indexes are stored in a second storage space.
18 . The non-transitory computer-readable storage medium according to claim 17 , wherein the computer-executable instructions further cause the system to:
determine M 1 first tensor block indexes and M 2 second tensor block indexes, wherein the M 1 first tensor block indexes respectively indicate M 1 first-type tensor blocks, the M 2 second tensor block indexes respectively indicate M 2 second-type tensor blocks;
save the M 1 first tensor block indexes in a first storage space, and saving the M 2 second tensor block indexes in a second storage space; and
obtain the M 1 first tensor block indexes from the first storage space, and obtaining the M 2 second tensor block indexes from the second storage space; and the processing, the M tensor blocks concurrently further comprises:
process, the M tensor blocks based on the M 1 first tensor block indexes and the M 2 second tensor block indexes, wherein the M 1 first-type tensor blocks that are processed concurrently by the P processing units in the first time period are determined from the M tensor blocks based on the M 1 first tensor block indexes, and the M 2 second-type tensor blocks that are processed concurrently by the P processing units in the second time period are determined from the M tensor blocks based on the M 2 second tensor block indexes.
19 . The computer readable storage medium according to claim 17 , wherein the M 1 first tensor block indexes are stored in the first storage space in a consecutive manner, and the M 2 second tensor block indexes are stored in the second storage space in a consecutive manner.
20 . The computer readable storage medium according to claim 16 , the computer-executable instructions further cause the system to:
determine a first correspondence between the P processing units and the first-type tensor blocks;
determine a second correspondence between the P processing units and the second-type tensor blocks; and
the processing, the M tensor blocks comprises:
process, by the P processing units, the M tensor blocks concurrently based on the first correspondence and the second correspondence.