Data transfer technique

Inventors: Rachit Garg (San Jose, CA); Sukru Burc Eryilmaz (San Francisco, CA); Michael Andersch (Potsdam, DE); Apoorv Parle (Santa Clara, CA); Herbert Hum (Portland, OR); Kefeng Duan (Shanghai, CN); Jiang Jiang (Shanghai, CN)

Assignee: NVIDIA Corporation

G06F12/0862 G06F12/1072

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12675409
App. No.: 18/634,643
Granted: Jul 7, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Apparatuses, systems, and techniques are to transfer data based, at least in part, on a computational graph. In at least one embodiment, a processor causes a compiler to generate instructions to prefetch one or more data values from dynamic random access memory (DRAM) into an in-processor cache based, at least in part, on a computational graph.

Claims (31)

1 . A system, comprising:

one or more processors to cause a compiler to identify one or more data dependencies, indicating that one or more data values used by a first software instruction are to be used by a second software instruction, and generate one or more prefetch instructions based on the one or more data dependencies;

store the one or more prefetch instructions to be accessible by the one or more processors; and

cause the one or more data values to be transferred from a first storage location to a second storage location prior to executing the second software instruction, based at least in part on the one or more prefetch instructions.

2 . The system of claim 1 , wherein the one or more prefetch instructions are to cause the one or more processors to transfer the one or more data values from the first storage location to the second storage location prior to the second software instruction using the one or more data values.

3 . The system of claim 1 , wherein the one or more processors are to cause the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches based, at least in part, on a computational graph comprising one or more software instructions as nodes and the one or more data dependencies between the one or more software instructions as edges.

4 . The system of claim 1 , wherein the one or more processors are to cause the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches based, at least in part, on a computational graph comprising one or more indications of the one or more data values to be used by the first software instruction and by the second software instruction.

5 . The system of claim 1 , wherein the one or more processors are to cause the compiler to identify the first software instruction as compute-bound and the second software instruction to be performed after the first software instruction as memory-bound.

6 . The system of claim 1 , wherein the one or more prefetch instructions are to indicate when the one or more processors are to transfer the one or more data values from a storage location to a different storage location.

7 . The system of claim 1 , wherein the one or more processors are to cause the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches as part of a neural network training process.

8 . One or more processors, comprising:

circuitry to:

cause a compiler to identify one or more data dependencies, indicating that one or more data values used by a first software instruction are to be used by a second software instruction, and generate one or more prefetch instructions based on the one or more data dependencies;

store the one or more prefetch instructions to be accessible by the one or more processors; and

9 . The one or more processors of claim 8 , wherein the circuitry is to transfer the one or more data values from dynamic random access memory (DRAM) to an in-processor cache prior to the second software instruction using the one or more data values based, at least in part, on the one or more prefetch instructions.

10 . The one or more processors of claim 8 , wherein the circuitry is to cause the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches based, at least in part, on a graph comprising one or more indications of dependencies between the first software instruction and the second software instruction.

11 . The one or more processors of claim 8 , wherein the circuitry is to cause the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches based, at least in part, on a graph comprising one or more indications that the one or more data values are to be used by the first software instruction or by the second software instruction.

12 . The one or more processors of claim 8 , wherein the circuitry is to cause the compiler to identify the first software instruction as compute-bound and the second software instruction as memory-bound.

13 . The one or more processors of claim 8 , wherein the one or more prefetch instructions are to indicate when the circuitry is to transfer the one or more data values from dynamic random access memory (DRAM) to an in-processor cache.

14 . The one or more processors of claim 8 , wherein the circuitry to cause the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches as part of a neural network forward and backward propagation process.

15 . A method, comprising:

causing a compiler to identify one or more data dependencies, using a graph indicating memory operations of a first software instruction and a second software instruction, indicating that one or more data values used by the first software instruction are to be used by the second software instruction;

generating one or more prefetch instructions based on the one or more data dependencies;

storing the one or more prefetch instructions in memory accessible by one or more processors; and

causing the one or more data values to be transferred from a slower storage location to a faster storage location prior to executing the second software instruction, based at least in part on the one or more prefetch instructions.

16 . The method of claim 15 , further comprising transferring the one or more data values from a storage location to a different storage location to allow a second software instruction faster access to the one or more data values based, at least in part, on the one or more prefetch instructions.

17 . The method of claim 15 , wherein causing the compiler to cause the one or more prefetch instructions to be stored in one or more instruction caches is based, at least in part, on identifying in a computational graph one or more nodes representing kernels that are connected by one or more edges representing the one or more data values.

18 . The method of claim 15 , further comprising identifying the one or more data values to be removed from a cache and later prefetched into the cache based, at least in part, on a computational graph.

19 . The method of claim 15 , further comprising identifying the first software instruction as compute-bound, and identifying the second software instruction as memory-bound.

20 . The method of claim 15 , wherein a graphics processing application comprises the first software instruction.