Dynamic power management for artificial intelligence hardware accelerators
A computer-implemented method for dynamically managing the power usage and/or performance of an artificial intelligence (AI) hardware accelerator may include (1) receiving an instruction stream that includes one or more instructions for performing at least one AI-specific computing task, (2) identifying a plurality of special-purpose, hardware-based functional units configured to perform AI-specific computing tasks, (3) predicting, based on an analysis of at least a portion of the instruction stream, a power-usage requirement for at least one of the functional units when executing the instruction stream, and then (4) modifying, based on the power-usage requirement, the power supplied to at least one of the functional units. Various other methods and systems are also disclosed.
1. A computing device comprising:
a plurality of special-purpose, hardware accelerator functional units configured to perform artificial-intelligence-specific computing tasks;
a memory device; and
an instruction stream analysis unit coupled to the memory device and the plurality of functional units, the instruction stream analysis unit configured to:
receive an instruction stream that comprises one or more instructions for performing at least one artificial-intelligence-specific computing task with at least a first hardware functional unit of the plurality of functional units and a second hardware accelerator functional unit of the plurality of functional units, wherein the first functional unit is distinct from the second functional unit;
predict, based on an analysis of at least a portion of the instruction stream, a power-usage requirement for the first and second hardware accelerator functional units when executing the instruction stream; and
limit, based on the power-usage requirement, an operating capacity of the first hardware accelerator functional unit and an operating capacity of the second hardware accelerator functional unit by modifying power supplied to the first and second hardware accelerator functional units.
2. The computing device of claim 1 , wherein the instruction stream analysis unit predicts the power-usage requirement for at least one of the functional units by:
causing at least one of the functional units to execute a portion of the instruction stream;
observing utilization of the functional units when executing the portion of the instruction stream; and
forecasting, based at least in part on the observed utilization of the functional units when executing the portion of the instruction stream, a power-usage requirement for at least one of the functional units when executing a remaining portion of the instruction stream.
3. The computing device of claim 1 , wherein the memory device is configured to store a power-utilization profile for at least one artificial intelligence program, and wherein the power-utilization profile identifies the power-usage requirement for at least one of the functional units when executing the artificial intelligence program.
4. The computing device of claim 3 , wherein the instruction stream analysis unit predicts the power-usage requirement for at least one of the functional units by:
determining that the instruction stream corresponds to the artificial intelligence program;
retrieving the power-utilization profile for the artificial intelligence program from the memory device; and
determining, based on the power-utilization profile, the power-usage requirement for at least one of the functional units when executing the artificial intelligence program.
5. The computing device of claim 1 , wherein the instruction stream analysis unit predicts the power-usage requirement by:
identifying, based on the analysis of the instruction stream, at least one element of sparsity within the artificial-intelligence-specific computing task; and
predicting the power-usage requirement based at least in part on the identified element of sparsity.
6. The computing device of claim 1 , wherein at least one of the functional units is power gated to draw power only when in use.
7. The computing device of claim 1 , wherein the instruction stream analysis unit modifies the power supplied to at least one of the functional units by scaling a frequency of at least one of the functional units.
8. The computing device of claim 1 , wherein the instruction stream analysis unit modifies the power supplied to at least one of the functional units by scaling a voltage supplied to at least one of the functional units.
9. The computing device of claim 1 , wherein the instruction stream analysis unit comprises at least one of:
a general-purpose processing unit;
a special-purpose processing unit; or
a logical operation unit.
10. The computing device of claim 1 , wherein the instruction stream analysis unit is at least one of:
integrated within a hardware accelerator that comprises the functional units; or
external to the hardware accelerator.
11. The computing device of claim 1 , wherein the functional units comprise at least one of:
a multiply-accumulate unit;
a direct memory access unit; or
a second memory device.
12. A hardware accelerator comprising:
a plurality of special-purpose, hardware-based functional units configured to perform artificial-intelligence-specific computing tasks; and
an instruction stream analysis unit coupled to the plurality of functional units and a cache, the instruction stream analysis unit configured to:
receive an instruction stream that comprises one or more instructions for performing at least one artificial-intelligence-specific computing task with at least a first functional unit of the plurality of functional units and a second functional unit of the plurality of functional units, wherein the first functional unit is distinct from the second functional unit;
predict, based on an analysis of at least a portion of the instruction stream, a power-usage requirement for the at least one of the functional units when executing the instruction stream; and
limit, based on the power-usage requirement, an operating capacity of the first and second functional units by modifying power supplied to the first and second functional units.
13. A computer-implemented method comprising:
receiving an instruction stream that comprises one or more instructions for performing at least one artificial-intelligence-specific computing task;
identifying a plurality of special-purpose, hardware accelerator functional units configured to perform artificial-intelligence-specific computing tasks;
predicting, based on an analysis of at least a portion of the instruction stream, a power-usage requirement for at least a first hardware accelerator functional unit of the functional units and a second hardware accelerator functional unit of the functional units when executing the instruction stream, wherein the first hardware accelerator function unit is distinct from the second hardware accelerator functional unit; and
limiting, based on the power-usage requirement, an operating capacity of the first and second hardware accelerator functional units by modifying power supplied to the first and second hardware accelerator functional units.
14. The computer-implemented method of claim 13 , wherein predicting the power-usage requirement for at least one of the functional units comprises:
executing a portion of the instruction stream;
observing utilization of the functional units when executing the portion of the instruction stream; and
forecasting, based at least in part on the observed utilization of the functional units when executing the portion of the instruction stream, a power-usage requirement for at least one of the functional units when executing a remaining portion of the instruction stream.
15. The computer-implemented method of claim 13 , further comprising, prior to receiving the instruction stream, storing a power-utilization profile for at least one artificial intelligence program, wherein the power-utilization profile identifies the power-usage requirement for at least one of the functional units when executing the artificial intelligence program.
16. The computer-implemented method of claim 15 , wherein predicting the power-usage requirement for at least one of the functional units comprises:
determining that the instruction stream corresponds to the artificial intelligence program;
retrieving the power-utilization profile for the artificial intelligence program from a memory device; and
determining, based on the power-utilization profile, the power-usage requirement for at least one of the functional units when executing the artificial intelligence program.
17. The computer-implemented method of claim 13 , wherein predicting the power-usage requirement further comprises:
identifying, based on the analysis of the instruction stream, at least one element of sparsity within the artificial-intelligence-specific computing task; and
predicting the power-usage requirement based at least in part on the identified element of sparsity.
18. The computer-implemented method of claim 13 , wherein at least one of the functional units is power gated to draw power only when in use.
19. The computer-implemented method of claim 13 , wherein modifying the power supplied to at least one of the functional units comprises scaling a frequency of at least one of the functional units.
20. The computer-implemented method of claim 13 , wherein modifying the power supplied to at least one of the functional units comprises scaling a voltage supplied to at least one of the functional units.