IP Library › Granted Patent US 12,260,323
Granted Patent B2
US 12,260,323 · App. 18/491,246 · Granted Mar 25, 2025

Methods, systems, articles of manufacture and apparatus to map workloads

Inventors: Estelle Aflalo (Haifa, IL); Amit Bleiweiss (Yad Binyamin, IL); Mattias Marder (Haifa, IL); Eliran Zimmerman (Maalot, IL)
Assignee: Intel Corporation
G06N3/063G06F9/5011G06F9/5044G06F18/217G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,323
App. No.
18/491,246
Granted
Mar 25, 2025
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture are disclosed to map workloads. An example apparatus includes a constraint definer to define performance characteristic targets of the neural network, an action determiner to apply a first resource configuration to candidate resources corresponding to the neural network, a reward determiner to calculate a results metric based on (a) resource performance metrics and (b) the performance characteristic targets, and a layer map generator to generate a resource mapping file, the mapping file including respective resource assignments for respective corresponding layers of the neural network, the resource assignments selected based on the results metric.

Claims (43)

1. A non-transitory machine readable storage medium comprising instructions to cause first processor circuitry to at least:

cause execution of a neural network (NN) model with a first configuration of hardware circuitry to determine first performance metrics;

cause execution of the NN model with a second configuration of the hardware circuitry to determine second performance metrics;

compare the first performance metrics and the second performance metrics; and

determine one of the first configuration of the hardware circuitry or the second configuration of the hardware circuitry for execution of the NN model based on the comparison of the first performance metrics and the second performance metrics.

2. The non-transitory machine readable storage medium as defined in claim 1 , further including the instructions to cause the first processor circuitry to generate a report corresponding to the determined one of the first configuration of the hardware circuitry or the second configuration of the hardware circuitry.

3. The non-transitory machine readable storage medium as defined in claim 1 , wherein the instructions cause the first processor circuitry to:

cause execution of the NN model during a first iteration to determine first latency metrics; and

cause execution of the NN model during a second iteration to determine second latency metrics, the first and second latency metrics corresponding to the first and second performance metrics, respectively.

4. The non-transitory machine readable storage medium as defined in claim 1 , wherein the instructions cause the processor circuitry to generate configuration instructions corresponding to the first configuration of the hardware circuitry or the second configuration of the hardware circuitry.

5. The non-transitory machine readable storage medium as defined in claim 1 , wherein the NN model includes a first layer and a second layer, the instructions to cause the processor circuitry to:

determine first layer performance metrics corresponding to execution of the first configuration of the hardware circuitry with the first layer; and

determine second layer performance metrics corresponding to execution of the first configuration of the hardware circuitry with the second layer.

6. The non-transitory machine readable storage medium as defined in claim 5 , wherein the instructions cause the processor circuitry to compare the first layer performance metrics with the second layer performance metrics.

7. The non-transitory machine readable storage medium as defined in claim 6 , wherein the instructions cause the processor circuitry to assign one of the first layer or the second layer of the NN model to execute on the first configuration of the hardware circuitry based on the comparison.

8. The non-transitory machine readable storage medium as defined in claim 6 , wherein the instructions cause the processor circuitry to instantiate a simulator to determine the first layer performance metrics and the second layer performance metrics.

9. The non-transitory machine readable storage medium as defined in claim 6 , wherein the instructions cause the processor circuitry to generate a relative score between the first layer performance metrics and the second layer performance metrics.

10. An apparatus comprising:

action determiner circuitry to:

cause execution of a neural network (NN) model with a first configuration of hardware circuitry to determine first performance metrics; and

cause execution of the NN model with a second configuration of the hardware circuitry to determine second performance metrics;

reward determiner circuitry to compare the first performance metrics and the second performance metrics; and

map generator circuitry to determine one of the first configuration of the hardware circuitry or the second configuration of the hardware circuitry to execute the NN model based on the comparison of the first performance metrics and the second performance metrics.

11. The apparatus as defined in claim 10 , further including state definer circuitry to generate a report corresponding to the determined one of the first configuration of the hardware circuitry or the second configuration of the hardware circuitry.

12. The apparatus as defined in claim 10 , wherein the action determiner circuitry is to:

cause execution of the NN model during a first iteration to determine first latency metrics; and

cause execution of the NN model during a second iteration to determine second latency metrics, the first and second latency metrics corresponding to the first and second performance metrics, respectively.

13. The apparatus as defined in claim 10 , wherein the reward determiner circuitry is to generate configuration instructions corresponding to the first configuration of the hardware circuitry or the second configuration of the hardware circuitry.

14. The apparatus as defined in claim 10 , wherein the action determination circuitry is to:

determine first layer performance metrics corresponding to a first layer of the NN model, the first layer performance metrics corresponding to execution of the first configuration of the hardware circuitry with the first layer of the NN model; and

determine second layer performance metrics corresponding to a second layer of the NN model, the second layer performance metrics corresponding to execution of the first configuration of the hardware circuitry with the second layer of the NN model.

15. The apparatus as defined in claim 14 , wherein the reward determiner circuitry is to compare the first layer performance metrics with the second layer performance metrics.

16. The apparatus as defined in claim 15 , wherein the map generator circuitry is to assign one of the first layer or the second layer of the NN model to execute on the first configuration of the hardware circuitry based on the comparison.

17. A method comprising:

determining, by executing an instruction with programmable circuitry, first performance metrics corresponding to execution of a neural network (NN) model with a first configuration of hardware circuitry;

determining, by executing an instruction with the programmable circuitry, second performance metrics corresponding to execution of the NN model with a second configuration of the hardware circuitry;

comparing, by executing an instruction with the programmable circuitry, the first performance metrics and the second performance metrics; and

determining, by executing an instruction with the programmable circuitry, one of the first configuration of the hardware circuitry or the second configuration of the hardware circuitry for execution of the NN model based on the comparison of the first performance metrics and the second performance metrics.

18. The method as defined in claim 17 , further including generating a report corresponding to the determined one of the first configuration of the hardware circuitry or the second configuration of the hardware circuitry.

19. The method as defined in claim 17 , further including:

determining first layer performance metrics corresponding to execution of a first layer of the NN model with the first configuration of the hardware circuitry; and

determining second layer performance metrics corresponding to execution of a second layer of the NN model with the first configuration of the hardware circuitry.

20. The method as defined in claim 19 , further including assigning one of the first layer or the second layer of the NN model to execute on the first configuration of the hardware circuitry based on a comparison of the first layer performance metrics and the second layer performance metrics.

Continuity (3)
Continuation 17977774 · Oct 31, 2022
Continuation 16541878 · Aug 15, 2019
Related Publication 20240119271A1 · Apr 11, 2024
References Cited (28)
US 11301718B2 · Hamedi et al. · 2022 [cited by applicant]
US 11526736B2 · Aflalo et al. · 2022 [cited by applicant]
US 20190080019A1 · Young et al. · 2019 [cited by applicant]
US 20190095796A1 · Chen · 2019 [cited by applicant]
US 20190325296A1 · Fowers · 2019 [cited by examiner]
US 20190354850A1 · Watson · 2019 [cited by applicant]
US 20200210840A1 · Rouhani et al. · 2020 [cited by applicant]
US 20210248268A1 · Ardhanari · 2021 [cited by examiner]
US 20230289583A1 · Driscoll · 2023 [cited by examiner]
Silver, “Lecture 1: Introduction to Reinforcement Learning,” David Silver, 2015, 46 pages. [cited by applicant]
Levine, “Actor-Critic Algorithms,” CS 294-112: Deep Reinforcement Learning, University of California-Berkeley, 28 pages. [cited by applicant]
Simonini, “An Intro to Advantage Actor Critic Methods: Let's Play Sonic the Hedgehog!” FreeCodeCamp, Jul. 26, 2018, retrieved from https://www.freecodecamp.org/news/an-intro-to-advantage-actor-critic-methods-lets-play-s… [cited by applicant]
Yoon, “Understanding Actor Critic Methods and A2C: Important Concepts in Deep Reinforcement Learning,” Medium, Feb. 5, 2019, 13 pages. [cited by applicant]
Paliwal et al., “Regal: Transfer Learning for Fast Optimization of Computation Graphs,” ARXIV.org, Cornell University Library, May 7, 2019, 12 pages. [cited by applicant]
EdgeCortix Inc., “Dynamic Neural Accelerator ™: Artificial Intelligence Processor for Edge Devices,” EdgeCortix Inc. pamphlet, 2020, 2 pages. [cited by applicant]
European Patent Office, “Extended European Search Report,” issued in connection with European Patent Application No. 20179887.3, issued on Dec. 8, 2020, 9 pages. [cited by applicant]
Nez et al., “Dynamic Neural Accelerator for Reconfigurable & Energy-efficient Neural Network Inference,” EdgeCortix Inc. Presentation, 2021, 21 pages. [cited by applicant]
Demler, “Software is Critical to Edge-AI Deployment,” The Linley Group, Sep. 2021, 9 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 16/541,878, mailed on Aug. 9, 2022, 8 pages. [cited by applicant]
Nunes et al., “Compiling and Optimizing a Model with TVMC,” TVM Tutorial, retrieved from https://tvm.apache.org/docs/tutorial/tvmc_command_line_driver.html#:-:text=Compiling%20and%200ptimizing%20a%20Model% 20with%20TVMC… [cited by applicant]
European Patent Office, “Communication pursuant to Article 94(3) EPC,” issued in connection with European Patent Application No. 20179887.3, dated Feb. 15, 2023, 5 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 17/977,774, mailed on Mar. 14, 2023, 9 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 17/977,774, mailed on Jul. 11, 2023, 7 pages. [cited by applicant]
European Patent Office, “Extended European Search Report,” issued in connection with European Patent Application No. 23178001.6, dated Aug. 30, 2023, 10 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action” issued in U.S. Appl. No. 16/863,157, on Nov. 14, 2023 (64 pages). [cited by applicant]
European Patent Office, “Communication pursuant to Article 94(3) EPC,” issued in connection with European Patent Application No. 20 179 887.3, dated Sep. 26, 2024, 6 pages. [cited by applicant]
Zhang et al., “DNNBuilder: an automated tool for building high-performance DNN hardware accelerators for FPGAs,” International Conference on Computer-Aided Design, Nov. 5, 2018, 9 pages. [cited by applicant]
European Patent Office, “Extended European Search Report,” issued in connection with European Patent Application No. 24197216.5, dated Nov. 15, 2024, 6 pages. [cited by applicant]