IP Library Granted Patent US 10,877,811
Granted Patent B1
US 10,877,811 · App. 16/565,793 · Granted Dec 29, 2020

Scheduler for vector processing operator allocation

Inventor: Peter Verplaetse (Redwood City, CA)
Assignee: Ambarella International LP
G06F9/5044G06F9/4881G06F9/5038G06F16/9024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,877,811
App. No.
16/565,793
Granted
Dec 29, 2020
Kind
B1
Abstract

An apparatus includes a scheduler circuit and an operator allocation circuit. The scheduler circuit may be configured to (i) parse a directed acyclic graph into one or more operators, (ii) track a plurality of first status signals indicating a readiness state of a plurality of unscheduled operators that are to be allocated to a plurality of hardware engines, (iii) track a plurality of second status signals indicating a readiness state of the hardware engines, and (iv) for each operator, track a resource type parameter. The operator allocation circuit may be configured to (a) select a resource type from a list of resource types in use, (b) determine available hardware engines corresponding to the selected resource type based on the second status signals, (c) generate scores for the unscheduled operators based on (i) the selected resource type, (ii) the first status signals, and (iii) the resource type parameters, and (d) allocate at least one of the unscheduled operators to at least one of the available hardware engines based on the scores.

Claims (33)

1. An apparatus comprising:

a scheduler circuit configured to (i) parse a directed acyclic graph into one or more operators, (ii) track a plurality of first status signals indicating a readiness state of a plurality of unscheduled operators that are to be allocated to a plurality of hardware engines, (iii) track a plurality of second status signals indicating a readiness state of said hardware engines, and (iv) track a resource type parameter for each operator; and

an operator allocation circuit configured to (a) select a resource type from a list of resource types in use, (b) determine available hardware engines corresponding to the selected resource type based on the second status signals, (c) generate scores for the unscheduled operators based on (i) the selected resource type, (ii) the first status signals, and (iii) the resource type parameters, and (d) allocate at least one of said unscheduled operators to at least one of said available hardware engines based on said scores.

2. The apparatus according to claim 1 , wherein said directed acyclic graph defines a computer vision process.

3. The apparatus according to claim 1 , wherein (i) said scheduler circuit is further configured to generate said list of resource types in use by tracking a plurality of resource types corresponding to said operators and (ii) said resource types identify said hardware engines suitable to process said operators.

4. The apparatus according to claim 1 , wherein said scores for said unscheduled operators are generated further based on priority parameters and offset parameters.

5. The apparatus according to claim 1 , wherein said scheduler circuit is further configured to track an offset parameter for each operator and the offset parameters are updated in response to allocation of said unscheduled operators to said available hardware engines.

6. The apparatus according to claim 1 , wherein said scheduler circuit is further configured to track a priority parameter for each operator and said unscheduled operators having similar priority are allocated to said hardware engines according to an arbitration scheme.

7. The apparatus according to claim 1 , wherein said scheduler circuit is further configured to track a plurality of preemption thresholds corresponding to said operators.

8. The apparatus according to claim 7 , wherein said scheduler circuit is further configured to preempt a scheduled operator being used in favor of a ready one of said unscheduled operators based on said preemption thresholds.

9. The apparatus according to claim 7 , wherein said scheduler circuit is further configured to stall allocation of a ready one of said unscheduled operators while all of said hardware engines are busy processing based on said preemption thresholds.

10. The apparatus according to claim 1 , wherein said hardware engines and said scheduler circuit are part of a computer vision system.

11. The apparatus according to claim 1 , wherein said hardware engines and said scheduler circuit are part of a digital camera.

12. A method for allocating operators parsed from a directed acyclic graph, comprising the steps of:

tracking a plurality of first status signals indicating a readiness state of a plurality of unscheduled operators that are to be allocated to a plurality of hardware engines;

tracking a plurality of second status signals indicating a readiness state of said hardware engines;

tracking a resource type parameter for each operator;

utilizing an operator allocation circuit to (a) select a resource type from a list of resource types in use, (b) determine available hardware engines corresponding to the selected resource type based on the plurality of second status signals, (c) generate scores for the unscheduled operators based on (i) the selected resource type, (ii) the plurality of first status signals, and (iii) the resource type parameters, and (d) allocate at least one of said unscheduled operators to at least one of said available hardware engines based on said scores; and

processing a plurality of vectors in said hardware engines using said operators.

13. The method according to claim 12 , further comprising:

generating said list of resource types in use by tracking a plurality of resource types corresponding to said operators, wherein said resource types identify said hardware engines suitable to process said operators.

14. The method according to claim 12 , wherein said scores for said unscheduled operators are generated further based upon priority parameters and offset parameters corresponding to said operators.

15. The method according to claim 12 , further comprising:

tracking an offset parameter for each operator; and

updating said offset parameters corresponding to said operators in response to allocation of said unscheduled operators to said available hardware engines.

16. The method according to claim 12 , further comprising:

tracking a priority parameter for each operator, wherein said unscheduled operators having similar priority are allocated to said hardware engines according to an arbitration scheme.

17. The method according to claim 12 , further comprising:

tracking a plurality of preemption threshold parameters corresponding to said operators.

18. The method according to claim 17 , further comprising:

preempting a scheduled operator being used in favor of a ready one of said unscheduled operators based on comparison of said preemption threshold parameters with a preemption priority value of said hardware engines.

19. The method according to claim 18 , wherein said preemption threshold parameters comprise statically set values.

20. The method according to claim 18 , wherein said unscheduled operators are allocated based on said preemption threshold parameters having a value greater than said preemption priority value.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2020
From: AMBARELLA, INC.
To: AMBARELLA INTERNATIONAL LP
Reel/Frame 051692/0711 →
Continuity (2)
Continuation 15591352 · May 10, 2017
Provisional Application 62492398 · May 1, 2017