Storing and retrieving training data for models in a data center
Technology for a memory controller is described. The memory controller can receive a request to store training data. The request can include a model identifier (ID) that identifies a model that is associated with the training data. The memory controller can send a write request to store the training data associated with the model ID in a memory region in a pooled memory that is allocated for the model ID. The training data that is stored in the memory region in the pooled memory can be addressable based on the model ID.
1. A memory controller, comprising logic to:
receive a request to register a model associated with training data, the request to register the model indicates a model identifier (ID) for the model, a desired quality of service (QoS) for the model as defined in a service level agreement (SLA) and a memory capacity to store training data for the model;
register the model via allocation of at least one memory region in a pooled memory to the model ID, the allocation of the at least one memory region to the model ID to correspond to the memory capacity included in the request;
receive a request to store training data that includes the model ID; and
send a write request to cause the training data to be stored in the at least one memory region, wherein the training data that is stored in the at least one memory region is addressable based on the model ID.
2. The memory controller of claim 1 , wherein the training data that is stored in the at least one memory region is further addressable based on a model type of the model that is associated with the training data.
3. The memory controller of claim 1 , wherein the logic is further configured to:
receive, from a hardware platform that runs the model, a request to read the training data from the at least one memory region;
send a read request to read the training data from the at least one memory region, wherein the training data is read from the at least one memory region based on the model ID or a model type of the model; and
provide the training data to the hardware platform to enable the hardware platform to train the model using the training data.
4. The memory controller of claim 1 , wherein the logic is further configured to provide discovery information to one or more of a processor or a hardware platform to enable discovery of available training data per model ID in the pooled memory.
5. The memory controller of claim 1 , wherein the logic is further configured to forward the request received from a processor as a peer-to-peer (P2P) request to a second memory controller in a data center, wherein the P2P request is sent to the second memory controller when the request received from the processor is unable to be processed at the memory controller, wherein the memory controller includes information that identifies a presence of the second memory controller.
6. The memory controller of claim 1 , wherein the at least one memory region includes multiple memory regions, the logic is further configured to:
send a write request to store cause the training data associated with the model ID to be stored across the multiple memory regions, wherein the training data is distributed across the multiple memory regions to satisfy the SLA; and
send a read request to read the training data from a selection of one or more of the multiple memory regions, the training data is read from the selected one or more memory regions to satisfy the SLA.
7. The memory controller of claim 1 , wherein the logic is further configured to cause the training data to be stored in the at least one memory region and read the training data from the at least one memory region, respectively, via a secondary memory controller that interfaces with one of a processor or a hardware platform that runs the model.
8. The memory controller of claim 1 , wherein the logic is further configured to cause the training data to be stored in the at least one memory region and read the training data from the at least one memory region, respectively, on a per model ID basis.
9. The memory controller of claim 1 , wherein the pooled memory is configured to dynamically increase or decrease an amount of data associated to a particular model ID.
10. The memory controller of claim 1 , wherein the model is an artificial intelligence (AI) model.
11. The memory controller of claim 1 , wherein the model that is trained using the training data is an inferencing artificial intelligence (AI) model.
12. The memory controller of claim 1 , wherein the memory controller is included in a storage rack in a data center.
13. A system for storing training data in pooled memory, the system comprising:
a processor operable to provide training data;
a hardware platform that runs a model;
a pooled memory that includes a memory region; and
a memory controller to include logic, the logic to:
receive a request from the processor to register a model associated with training data, the request to register the model indicates a model identifier (ID) for the model, a desired quality of service (QoS) for the model as defined in a service level agreement (SLA) and a memory capacity to store training data for the model;
register the model via allocation of at least one memory region in a pooled memory to the model ID, the allocation of the at least one memory region to the model ID to correspond to the memory capacity included in the request;
receive a request from the processor to store the training data, the request to include the model ID; and
send a write request to cause the training data to be stored in the at least one memory region, wherein the training data that is stored in the at least one memory region is addressable based on the model ID.
14. The system of claim 13 , wherein the training data that is stored in the at least one memory region is further addressable based on a model type of the model that is associated with the training data.
15. The system of claim 13 , wherein the model is an artificial intelligence (AI) model.
16. The system of claim 13 , wherein the memory controller further comprises logic to provide discovery information to one or more of the processor or a hardware platform to enable discovery of available training data per model ID in the pooled memory.
17. The system of claim 13 , wherein the memory controller further comprises logic to forward the request received from the processor as a peer-to-peer (P2P) request to a second memory controller, wherein the P2P request is sent to the second memory controller when the request received from the processor is unable to be processed at the memory controller, wherein the memory controller includes information that identifies a presence of the second memory controller.
18. The system of claim 13 , wherein the at least one memory region includes multiple memory regions, the memory controller further comprises logic to:
send a write request to cause the training data associated with the model ID to be stored across the multiple memory regions, wherein the training data is distributed across the multiple memory regions to satisfy the SLA; and
send a read request to read the training data from a selection of one or more of the multiple memory regions, the training data is read from the selected one or more memory regions to satisfy the SLA.
19. A method for initiating operations at a memory controller, the method comprising:
receiving a request to register a model associated with training data, the request to register the model indicates a model identifier (ID) for the model, a desired quality of service (QoS) for the model as defined in a service level agreement (SLA) and a memory capacity to store training data for the model;
registering the model via allocation of at least one memory region in a pooled memory to the model ID, the allocation of the at least one memory region to the model ID to correspond to the memory capacity included in the request;
receiving a request to store training data that includes the model ID; and
sending a write request to cause the training data to be stored in the at least one memory region, wherein the training data that is stored in the at least one memory region is addressable based on the model ID.
20. The method of claim 19 , further comprising:
receiving, from a hardware platform that runs the model, a request to read the training data from the at least one memory region;
sending a read request to read the training data from the at least one memory region, wherein the training data is read from the at least one memory region based on the model ID or a model type of the model; and
providing the training data to the hardware platform to enable the hardware platform to train the model using the training data.
21. The method of claim 19 , further comprising:
the at least one memory region including multiple memory regions;
sending a write request to cause the training data associated with the model ID to be stored across the multiple memory regions, wherein the training data is distributed across the multiple memory regions to satisfy the SLA; and
sending a read request to read the training data from a selection of one or more of the multiple memory regions, the training data is read from the selected one or more memory regions to satisfy the SLA.
22. The method of claim 19 , further comprising storing the training data in the at least one memory region pooled memory and read the training data from the at least one memory region, respectively, on a per model ID basis.