Granted Patent B2

US 12,436,801 · App. 17/985,120 · Granted Oct 7, 2025

Inventors: Amar Phanishayee (Seattle, WA); Saurabh Agarwal (Madison, WI)

Assignee: Microsoft Technology Licensing, LLC

G06F9/4881G06F9/5077G06F2209/501G06F2209/505

Abstract

Claims (27)

1. A system, comprising:

storage configured to store computer-readable instructions; and

a processor configured to execute the computer-readable instructions to generate a deep learning cluster scheduler modular toolkit that includes multiple deep learning (DL) scheduler abstraction modules and interactions between the multiple DL scheduler abstraction modules and allows user composition of the multiple DL scheduler abstraction modules to realize a DL scheduler and is further configured to allow the user to change composition of an individual DL scheduler abstraction module to change the DL scheduler without having to change other DL scheduler abstraction modules.

2. The system of claim 1 , wherein the deep learning cluster scheduler modular toolkit further includes interaction pathways between pairs of the multiple DL scheduler abstraction modules upon which the interactions occur.

3. The system of claim 1 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning admission policy module.

4. The system of claim 3 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning job wait queue that is configured to receive DL jobs from the deep learning admission policy module.

5. The system of claim 4 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning scheduling policy module configured to receive DL jobs from the deep learning job wait queue.

6. The system of claim 5 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning job placement policy module configured to receive DL job scheduling instructions from the deep learning scheduling policy module.

7. The system of claim 6 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning job preemption launch and restart policy module configured to receive DL job placement instructions from the deep learning job placement policy module.

8. The system of claim 7 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning and machine learning metrics collection module configured to track application and system-level metrics information relating to a cluster that is training the DL jobs.

9. The system of claim 8 , wherein the deep learning cluster scheduler modular toolkit comprises a deep learning cluster management module configured to track resources of the cluster.

10. A device-implemented method, comprising:

providing a deep learning cluster scheduler modular toolkit that includes multiple modular deep learning scheduler abstractions and interaction paths between the multiple modular deep learning scheduler abstractions;

receiving user input for individual modular deep learning scheduler abstractions; and,

composing multiple modular deep learning scheduler abstraction modules to realize a deep learning scheduler from the multiple modular deep learning scheduler abstractions and the user input that follows the interaction paths and is configured to receive user adjustments to an individual modular deep learning scheduler abstraction module without adjusting other individual modular deep learning scheduler abstraction modules.

11. The method of claim 10 , wherein the receiving comprises receiving through an application program interface provided by the deep learning cluster scheduler modular toolkit.

12. The method of claim 11 , wherein the receiving comprises receiving user input that defines values for an individual modular deep learning scheduler abstraction.

13. A system, comprising:

a deep learning central scheduler configured to utilize deep learning scheduler abstractions to make deep learning job scheduling decisions for job submissions;

a deep learning worker manager configured to manage cluster resources for the deep learning job scheduling decisions based upon the deep learning scheduler abstractions;

and,

a deep learning cluster scheduler modular toolkit client library configured to collect application related metrics and provide integration between applications and the deep learning central scheduler and further configured to allow changes to a composition of an individual deep learning scheduler abstraction to change the deep learning central scheduler without having to change other deep learning scheduler abstractions.

14. The system of claim 13 , wherein the deep learning scheduler abstractions of the deep learning central scheduler include a deep learning job admission deep learning scheduler abstraction, a deep learning cluster management deep learning scheduler abstraction, a deep learning and machine learning metrics collection deep learning scheduler abstraction, a deep learning job scheduling deep learning scheduler abstraction, a deep learning job preemption launch and restart deep learning scheduler abstraction, and a deep learning job placement deep learning scheduler abstraction.

15. The system of claim 14 , wherein an instance of the deep learning worker manager launches for each deep learning job running on a cluster.

16. The system of claim 15 , wherein the deep learning cluster scheduler modular toolkit client library comprises a deep learning cluster scheduler modular toolkit (DLCSMT) data loader sub-component.

17. The system of claim 16 , wherein the deep learning cluster scheduler modular toolkit client library comprises a push metrics sub-component.

18. The system of claim 17 , further comprising a DLCSMT throughput predictor model that is configured to utilize hardware features of the cluster to predict total throughput of submitted jobs that are co-located on the cluster.

Assignments (1)

Continuity (1)

References Cited (71)