Data path for GPU machine learning training with key value SSD
A system and method for machine learning. The system includes a GPU with a GPU memory, and a key value storage device connected to the GPU memory. The method includes, writing, by the GPU, a key value request to a key value request queue in a input-output region of the GPU memory, the key value request including a key. The method further includes reading, by the key value storage device, the key value request from the key value request queue, and writing, by the key value storage device, in response to the key value request, a value to the input-output region of the GPU memory, the value corresponding to the key of the key value request.
1. A method comprising:
receiving, by a key value storage device, a first key value request from a queue that is memory mapped to an input-output region of memory of another device based on a configuration by a host device; and
writing, by the key value storage device, to the input-output region of the memory of the other device, a first value based on the first key value request.
2. The method of claim 1 , further comprising, performing, by the key value storage device, a key lookup in the key value storage device to identify the first value.
3. The method of claim 1 , wherein the first key value request includes a return-value region, the return-value region specifying a region in the input-output region of the memory of the other device that is allocated for the first value.
4. The method of claim 3 , wherein the writing of the first value to the input-output region of the memory of the other device comprises writing the first value to the return-value region included in the first key value request.
5. The method of claim 1 , further comprising:
receiving, by the other device, the configuration from the host device; and
based on the configuration from the host device:
transmitting, by the other device, key value requests to the queue that is memory mapped to the input-output region of the memory of the other device; and
reading, by the other device, values from the input-output region of the memory of the other device.
6. The method of claim 1 , wherein the other device comprises a graphics processing unit configured to:
write the first key value request to the queue that is memory mapped to the input-output region of the memory of the other device; and
read the first value from the input-output region of the memory of the other device.
7. The method of claim 1 , wherein the first key value request is read, by the key value storage device, from the queue via peer-to-peer direct memory access.
8. The method of claim 1 , wherein the first value is written, by the key value storage device, to the input-output region of the memory via peer-to-peer direct memory access.
9. The method of claim 1 , wherein the key value storage device is connected to the other device by a peripheral component interconnect connection.
10. The method of claim 1 , wherein the first key value request is written, by the other device, to the queue that is memory mapped to the input-output region of the other device; and
wherein the method further comprises writing, by the other device, a second key value request to the queue before the writing, by the key value storage device, of the first value to the input-output region of the memory of the other device.
11. A system comprising:
a first device comprising memory; and
a key value storage device comprising a queue that is memory mapped to an input-output region of the memory of the first device based on a configuration by a host device,
wherein the key value storage device is configured to:
receive a first key value request, issued by the first device, from the queue; and
write, to the input-output region of the memory of the first device, a first value based on the first key value request.
12. The system of claim 11 , wherein the key value storage device is further configured to perform a key lookup in the key value storage device to identify the first value.
13. The system of claim 11 , wherein the first key value request includes a return-value region, the return-value region specifying a region in the input-output region of the memory of the first device that is allocated for the first value.
14. The system of claim 11 , wherein the first device comprises a graphics processing unit configured to:
write the first key value request to the queue of the key value storage device that is memory mapped to the input-output region of the memory of the first device; and
read the first value from the input-output region of the memory of the first device.
15. The system of claim 11 , wherein the key value storage device is configured to read the first key value request from the queue via peer-to-peer direct memory access.
16. The system of claim 11 , wherein the key value storage device is configured to write the first value to the input-output region of the memory via peer-to-peer direct memory access.
17. A system comprising:
a graphics processing unit comprising memory; and
a key value storage device comprising a queue that is memory mapped to an input-output region of the memory of the graphics processing unit based on a configuration by a host device,
wherein the key value storage device is configured to:
receive a first key value request, issued by the graphics processing unit, from the queue; and
write, to the input-output region of the memory of the graphics processing unit, a first value based on the first key value request.
18. The system of claim 17 , wherein the graphics processing unit is configured to:
write the first key value request to the queue of the key value storage device; and
read the first value from the input-output region of the memory of the graphics processing unit.
19. The system of claim 18 , wherein the key value storage device is configured to read the first key value request from the queue via peer-to-peer direct memory access, and
wherein the key value storage device is configured to write the first value to the input-output region of the memory via peer-to-peer direct memory access.
20. The system of claim 17 , wherein the first key value request comprises an address exposing the input-output region of the memory of the graphics processing unit based on the configuration by the host device.