Granted Patent B2

US 12,282,526 · App. 18/620,228 · Granted Apr 22, 2025

Inventors: Piotr Majcher (Sunnyvale, CA); Mostafa Hagog (Folsom, CA); Philippe Vandermersch (San Jose, CA)

Assignee: NVIDIA Corporation

G06F17/16G06F9/3001G06F9/30145G06N3/08G06N5/046

Abstract

Claims (67)

1. An acceleration processor unit (APU) comprising:

one or more core complexes, wherein the one or more core complexes include one or more central processing unit (CPU) cores;

one or more graphics complexes, wherein the one or more graphics complexes include one or more compute units and include at least one L2 cache;

one or more fabric interconnects;

one or more memory controllers;

one or more input/output (I/O) bus interfaces, at least one comprising a peripheral component interconnect express (PCIe) interface;

wherein the APU is utilized to implement an application programming interface (API) call to select one or more general matrix-to-matrix multiply (GEMM) implementations from among a plurality of GEMM implementations;

wherein the API call is to use:

input matrices;

a matrix multiply operation descriptor;

a matrix layout descriptor;

a search preferences parameter;

an algorithm count parameter to specify a number of algorithms desired;

a results array parameter;

a results count parameter to store a number of algorithms returned; and

an operation status to indicate whether the API call is successful or if other errors have occurred.

2. The APU of claim 1 , wherein the matrix multiply operation descriptor is a pointer.

3. The APU of claim 1 , wherein the search preferences parameter is to indicate a search method to utilize to search for the possible algorithms.

4. The APU of claim 1 , wherein the search preferences parameter is to indicate a preferred resource utilization.

5. The APU of claim 1 , wherein the algorithm count parameter is to include an integer value of the number of algorithms desired.

6. The APU of claim 1 , wherein the input matrices comprise one or more pointers to one or more values of the input matrices.

7. The APU of claim 1 , wherein the matrix layout descriptor comprises one or more pointers.

8. The APU of claim 1 , wherein the results array parameter comprises a pointer to a location in which a result of the GEMM is to be stored.

9. A method comprising:

implementing an application programming interface (API) call to select one or more general matrix-to-matrix multiply (GEMM) implementations from among a plurality of GEMM implementations, wherein the API call is to use:

input matrices;

a matrix multiply operation descriptor;

a matrix layout descriptor;

a search preferences parameter;

an algorithm count parameter to specify a number of algorithms desired;

a results array parameter;

a results count parameter to store a number of algorithms returned; and

an operation status to indicate whether the API call is successful or if other errors have occurred; and

the API call is to be implemented by a processor comprising:

one or more core complexes, wherein the one or more core complexes include one or more central processing unit (CPU) cores;

one or more graphics complexes, wherein the one or more graphics complexes include one or more compute units and include at least one L2 cache;

one or more fabric interconnects;

one or more memory controllers;

one or more input/output (I/O) bus interfaces, at least one comprising a peripheral component interconnect express (PCIe) interface.

10. The method of claim 9 , wherein the matrix multiply operation description is a pointer.

11. The method of claim 9 , wherein the search preferences parameter is to indicate a search method.

12. The method of claim 9 , wherein the search preferences parameter is to indicate a preferred resource utilization.

13. The method of claim 9 , wherein the algorithm count parameter is to include an integer value of the number of algorithms desired.

14. The method of claim 9 , further comprising performing the selected one or more GEMM implementations.

15. A system comprising:

memory; and

an acceleration processor unit (APU) comprising:

one or more core complexes, wherein the one or more core complexes include one or more central processing unit (CPU) cores;

one or more graphics complexes, wherein the one or more graphics complexes include one or more compute units and include at least one L2 cache;

one or more fabric interconnects;

one or more memory controllers;

one or more input/output (I/O) bus interfaces, at least one comprising a peripheral component interconnect express (PCIe) interface;

wherein the APU is utilized to implement an application programming interface (API) call to select one or more general matrix-to-matrix multiply (GEMM) implementations from among a plurality of GEMM implementations;

wherein the API call is to use:

input matrices;

a matrix multiply operation descriptor;

a matrix layout descriptor;

a search preferences parameter;

an algorithm count parameter to specify a number of algorithms desired;

a results array parameter;

a results count parameter to store a number of algorithms returned; and

an operation status to indicate whether the API call is successful or if other errors have occurred.

16. The system of claim 15 , wherein the matrix multiply operation descriptor is a pointer.

17. The system of claim 15 , wherein the search preferences parameter is to indicate a search method to utilize to search for the possible algorithms.

18. The system of claim 15 , wherein the search preferences parameter is to indicate a preferred resource utilization.

19. The system of claim 15 , wherein the algorithm count parameter is to include an integer value of the number of algorithms desired.

20. The system of claim 15 , wherein the input matrices comprise one or more pointers to one or more values of the input matrices.

Assignments (1)

Continuity (2)

References Cited (67)