Video memory management method, apparatus, device and system
A video memory management method is provided. The method includes: determining priorities of a plurality of machine learning tasks executed by a graphics processing unit; if video memory resources are to be allocated for a higher-priority task, and an amount of allocatable video memory resources is smaller than an amount of video memory resources required by the higher-priority task, releasing at least a part of video memory resources occupied by a lower-priority task; and allocating video memory resources to the higher-priority task, wherein the higher-priority task is executed at least according to tensor data in a video memory space.
1. A video memory management method, comprising:
determining priorities of a plurality of machine learning tasks executed by a graphics processing unit;
if video memory resources are to be allocated for a higher-priority task, and an amount of allocatable video memory resources is smaller than an amount of video memory resources required by the higher-priority task, releasing at least a part of video memory resources occupied by a lower-priority task; and
allocating video memory resources to the higher-priority task, wherein the higher-priority task is executed at least according to tensor data in a video memory space; wherein the lower-priority task comprises an iterative learning task and releasing at least a part of the video memory resources occupied by the lower-priority task comprises:
if an amount of idle video memory resources of the lower-priority task is smaller than the amount of video memory resources required by the higher-priority task,
allocating internal memory resources to the lower-priority task for the at least a part of tensor data;
after the lower-priority task completes current iterative learning, releasing at least a part of the video memory resources occupied by the lower-priority task, wherein the at least a part of tensor data is transferred to an internal memory space and the lower-priority task is executed at least according to the tensor data in the internal memory space.
2. The method according to claim 1 , further comprising:
releasing idle video memory resources occupied by the plurality of machine learning tasks.
3. The method according to claim 2 , wherein releasing the idle video memory resources occupied by the plurality of machine learning tasks comprises:
determining usage status information of video memory resources of the plurality of machine learning tasks; and
if the usage status information satisfies a video memory resource release condition, releasing the idle video memory resources.
4. The method according to claim 3 , wherein the usage status information comprises an upper limit of video memory resources used by one of the plurality of machine learning tasks; and the video memory resource release condition comprises a duration during which an amount of video memory resource allocated to the one of the plurality of machine learning tasks is larger than the upper limit reaches a duration threshold.
5. The method according to claim 1 , further comprising:
releasing idle video memory resources of other higher-priority tasks.
6. The method according to claim 1 , wherein releasing at least a part of the video memory resources occupied by the lower-priority task comprises:
if an amount of idle video memory resources of the lower-priority task is larger than or equal to the amount of video memory resources required by the higher-priority task, releasing the idle video memory resources occupied by the lower-priority task.
7. The method according to claim 1 , wherein the machine learning tasks comprise distributed deep learning tasks.
8. The method according to claim 1 , further comprising:
if the amount of the allocatable video memory resources increases to an amount of the video memory resources required by the lower-priority task, allocating video memory resources to the lower-priority task, wherein the lower-priority task is executed according to tensor data in the video memory space.
9. The method according to claim 1 , wherein before releasing at least a part of the video memory resources occupied by the lower-priority task, the method further comprises:
allocating internal memory resources to the higher-priority task, wherein the higher-priority task is executed according to tensor data in the internal memory space and the tensor data in the video memory space.
10. The method according to claim 9 , wherein after allocating memory resources to the higher-priority task, the method further comprises:
releasing the internal memory resources of the higher-priority task.
11. An electronic device for performing a video memory management method, the electronic device comprises:
a memory configured to store instructions; and
one or more processors configured to execute the instructions to cause the electronic device to perform:
determining priorities of a plurality of machine learning tasks executed by a graphics processing unit;
if video memory resources are to be allocated for a higher-priority task, and an amount of allocatable video memory resources is smaller than an amount of video memory resources required by the higher-priority task, releasing at least a part of video memory resources occupied by a lower-priority task; and
allocating video memory resources to the higher-priority task, wherein the higher-priority task is executed at least according to tensor data in a video memory space; wherein the lower-priority task comprises an iterative learning task and releasing at least a part of the video memory resources occupied by the lower-priority task comprises:
if an amount of idle video memory resources of the lower-priority task is smaller than the amount of video memory resources required by the higher-priority task,
allocating internal memory resources to the lower-priority task for the at least a part of tensor data;
after the lower-priority task completes current iterative learning, releasing at least a part of the video memory resources occupied by the lower-priority task, wherein the at least a part of tensor data is transferred to an internal memory space and the lower-priority task is executed at least according to the tensor data in the internal memory space.
12. The electronic device according to claim 11 , wherein the one or more processors are further configured to execute the instructions to cause the electronic device to perform:
releasing idle video memory resources occupied by the plurality of machine learning tasks.
13. The electronic device according to claim 11 , wherein the one or more processors are further configured to execute the instructions to cause the electronic device to perform:
releasing idle video memory resources of other higher-priority tasks.
14. The electronic device according to claim 11 , wherein in releasing at least a part of the video memory resources occupied by the lower-priority task, the one or more processors are further configured to execute the instructions to cause the electronic device to perform:
if an amount of idle video memory resources of the lower-priority task is larger than or equal to the amount of video memory resources required by the higher-priority task, releasing the idle video memory resources occupied by the lower-priority task.
15. A video memory management system, comprising a storage resource coordinator and a plurality of storage resource allocators, wherein
the storage resource coordinator is configured to:
determine priorities of a plurality of machine learning tasks that are executed by a graphics processing unit; and
if video memory resources are to be allocated for a higher-priority task, and an amount of allocatable video memory resources is smaller than an amount of video memory resources required by the higher-priority task,
send a video memory resource release instruction to a first of the plurality of storage resource allocators; and
send a video memory resource allocation instruction to a second of the plurality of storage resource allocators;
the first of the plurality of storage resource allocators is configured to release at least a part of video memory resources occupied by a lower-priority task according to the video memory release instruction; and
the second of the plurality of storage resource allocators is configured to allocate the video memory resources to a higher-priority task according to the video memory allocation instruction, wherein the higher-priority task is executed at least according to tensor data in a video memory space; wherein the lower-priority task comprises an iterative learning task and the storage resource coordinator is further configured to:
if an amount of idle video memory resources of the lower-priority task is smaller than the amount of video memory resources required by the higher-priority task,
send the video memory resource allocation instruction to the second of the plurality of storage resource allocators; and
send the video memory resource release instruction to the first of the plurality of storage resource allocators;
the second of the plurality of storage resource allocators is further configured to allocate internal memory resources to the lower-priority task for the at least a part of tensor data;
after the lower-priority task completes current iterative learning, the first of the plurality of storage resource allocators is configured to release at least a part of the video memory resources occupied by the lower-priority task, wherein the at least a part of tensor data is transferred to an internal memory space and the lower-priority task is executed at least according to the tensor data in the internal memory space.
16. The system according to claim 15 , wherein the plurality of storage resource allocators are further configured to send usage status information of the video memory resources of the plurality of machine learning tasks; and
the storage resource coordinator is further configured to send the video memory resource release instruction to the plurality of storage resource allocators if the usage status information satisfies a video memory resource release condition.
17. The system according to claim 16 , wherein the plurality of storage resource allocators are further configured to send the usage status information to the storage resource coordinator according to a preset period.