IP Library › Granted Patent US 11,983,128
Granted Patent B1
US 11,983,128 · App. 18/067,109 · Granted May 14, 2024

Multidimensional and multiblock tensorized direct memory access descriptors

Inventors: Kun Xu (Austin, TX); Ron Diamant (Santa Clara, CA); Ilya Minkin (Los Altos, CA); Mohammad El-Shabani (Redwood City, CA); Raymond S. Whiteside (Austin, TX); Uday Shilton Udayaselvam (Austin, TX)
Assignee: Amazon Technologies, Inc.
G06F13/30G06F13/1621G06F13/1642
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,983,128
App. No.
18/067,109
Granted
May 14, 2024
Kind
B1
Abstract

Techniques to reduce overhead in a direct memory access (DMA) engine can include processing descriptors from a descriptor queue to obtain a striding configuration to generate tensorized memory descriptors. The striding configuration can include, for each striding dimension, a stride and a repetition number indicating a number of times to repeat striding in the corresponding striding dimension. One or more sets of tensorized memory descriptors can be generated based on the striding configuration. Data transfers are then performed based on the generated tensorized memory descriptors.

Claims (57)

1. A method for operating a direct memory access engine, the method comprising:

retrieving a first descriptor from a descriptor queue;

determining that the first descriptor is a first tensorized header descriptor;

setting a striding configuration to generate addresses at a first stride in a first striding dimension for a first repetition number of times, the first stride and the first repetition number being provided in the first tensorized header descriptor;

retrieving a second descriptor from a descriptor queue;

determining that the second descriptor is a second tensorized header descriptor;

setting the striding configuration to generate addresses at the first stride in the first striding dimension for a first repetition number of times, and at a second stride in a second striding dimension for a second repetition number of times, the second stride and the second repetition number being provided in the second tensorized header descriptor;

retrieving a third descriptor from the descriptor queue;

determining that the third descriptor is a first tensorized template descriptor;

generating a first set of tensorized memory descriptors using a first starting address and a first data transfer size provided in the first tensorized template descriptor, the first set of tensorized memory descriptors generated based on the striding configuration including the first stride in the first striding dimension and the second stride in the second striding dimension; and

providing the first set of tensorized memory descriptors to a DMA interface to perform data transfers based on the first set of tensorized memory descriptors.

2. The method of claim 1 , further comprising:

retrieving a fourth descriptor from the descriptor queue;

determining that the fourth descriptor is a second tensorized template descriptor;

generating a second set of tensorized memory descriptors using a second starting address and a second data transfer size provided in the second tensorized template descriptor, the second set of tensorized memory descriptors generated based on the striding configuration including the first stride in the first striding dimension and the second stride in the second striding dimension; and

providing the second set of tensorized memory descriptors to the DMA interface to perform data transfers based on the second set of tensorized memory descriptors.

3. The method of claim 1 , further comprising:

retrieving a fourth descriptor from the descriptor queue;

determining that the fourth descriptor is a memory descriptor; and

providing the memory descriptor to the DMA interface to perform data transfers based on the first set of tensorized memory descriptors.

4. The method of claim 3 , further comprising:

in response to determining that the fourth descriptor is a memory descriptor, resetting the striding configuration.

5. A method comprising:

obtaining, by direct memory access (DMA) circuitry, a set of attributes from a descriptor queue, the set of attributes including a data transfer size, a starting address, and a striding configuration for multiple striding dimensions, wherein the striding configuration contains striding information for each striding dimension, and wherein the striding information includes a stride and a repetition number indicating a number of times to repeat striding in the corresponding striding dimension;

generating, by the DMA circuitry, a set of tensorized memory descriptors that include addresses having multi-dimensional strides based on the set of attributes; and

performing, by the DMA circuitry, data transfers according to the set of tensorized memory descriptors.

6. The method of claim 5 , wherein the striding configuration is obtained from a set of tensorized header descriptors.

7. The method of claim 5 , wherein the data transfer size and the starting address are obtained from a tensorized template descriptor in the descriptor queue.

8. The method of claim 6 , wherein the set of tensorized header descriptors includes a tensorized header descriptor for each striding dimension of the multiple striding dimensions, and each tensorized header descriptor contains the striding information for one of the multiple striding dimensions.

9. The method of claim 8 , wherein the set of tensorized header descriptors are obtained sequentially from the descriptor queue, and each successive tensorized header descriptor provides the striding information for a higher striding dimension.

10. The method of claim 7 , wherein the tensorized template descriptor follows a tensorized header descriptor in the descriptor queue.

11. The method of claim 7 , wherein the tensorized template descriptor includes a set of common fields that are included in each tensorized memory descriptor being generated from the tensorized template descriptor.

12. An integrated circuit device comprising:

a data buffer;

a descriptor queue operable to store descriptors;

a direct memory access (DMA) interface operable to transfer data between the data buffer and a component of a computing system; and

descriptor processing circuitry operable to:

process descriptors from the descriptor queue to obtain a striding configuration to generate tensorized memory descriptors, wherein the striding configuration includes, for each striding dimension, a stride and a repetition number indicating a number of times to repeat striding in the corresponding striding dimension;

generate multiple sets of tensorized memory descriptors based on the striding configuration; and

provide the DMA interface with the sets of tensorized memory descriptors to perform data transfers.

13. The integrated circuit device of claim 12 , wherein the descriptor processing circuitry is operable to process a tensorized header descriptor from the descriptor queue by adding a striding dimension to the striding configuration used for generating the tensorized memory descriptors, wherein the tensorized header descriptor contains the stride and the repetition number for the corresponding striding dimension.

14. The integrated circuit device of claim 12 , wherein the descriptor processing circuitry is operable to process a tensorized template descriptor from the descriptor queue by generating a set of tensorized memory descriptors based on the striding configuration using a starting address provided in the tensorized template descriptor.

15. The integrated circuit device of claim 12 , wherein the descriptor queue is operable to store a descriptor packet that includes, sequentially, a tensorized header descriptor, a first tensorized template descriptor, and a second tensorized template descriptor, and

wherein the descriptor processing circuitry is operable to process the descriptor packet by:

setting the striding configuration to generate addresses at a stride provided in the tensorized header descriptor;

generating a first set of tensorized memory descriptors based on the striding configuration using a first starting address provided in the first tensorized template descriptor; and

generating a second set of tensorized memory descriptors based on the striding configuration using a second starting address provided in the second tensorized template descriptor.

16. The integrated circuit device of claim 12 , wherein the descriptor queue is operable to store a descriptor packet that includes, sequentially, a first tensorized header descriptor, a second tensorized header descriptor, and a first tensorized template descriptor, and

wherein the descriptor processing circuitry is operable to process the descriptor packet by:

setting the striding configuration to generate addresses at a first stride in a first dimension and a second stride in a second dimension higher than the first dimension, the first stride being provided in the first tensorized header descriptor and the second stride being provided in the second tensorized header descriptor; and

generating a first set of tensorized memory descriptors based on the striding configuration using a first starting address provided in the first tensorized template descriptor.

17. The integrated circuit device of claim 13 , wherein the descriptors from the descriptor queue includes sequential tensorized header descriptors, and each sequential tensorized header descriptor adds a higher striding dimension to the striding configuration.

18. The integrated circuit device of claim 14 , wherein the descriptor processing circuitry is operable to process sequential tensorized template descriptors having different starting addresses by generating a set of tensorized memory descriptors for each sequential tensorized template descriptor based on the same striding configuration, wherein each set of tensorized memory descriptors being generated uses a different starting address.

19. The integrated circuit device of claim 14 , wherein the tensorized template descriptor further includes a data transfer size.

20. The integrated circuit device of claim 16 , wherein the descriptor packet further includes a second tensorized template descriptor following the first tensorized template descriptor, and

wherein the descriptor processing circuitry is operable to process the second tensorized template descriptor by generating a second set of tensorized memory descriptors using a second starting address provided in the second tensorized template descriptor, the second set of tensorized memory descriptors generated based on the striding configuration including the first stride in the first dimension and the second stride in the second dimension.

21. The integrated circuit device of claim 19 , wherein the descriptor processing circuitry is operable to process sequential tensorized template descriptors having different starting addresses and different data transfer sizes by generating a set of tensorized memory descriptors for each sequential tensorized template descriptor based on the same striding configuration, wherein each set of tensorized memory descriptors being generated uses a different starting address and a different data transfer size.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2022
From: XU, KUN; DIAMANT, RON; MINKIN, ILYA; EL-SHABANI, MOHAMMAD; WHITESIDE, RAYMOND S.; UDAYASELVAM, UDAY SHILTON
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 062131/0130 →
Cited By (4)
US 12,259,833 US 12,332,801 US 12,411,780 US 12,411,785