Scheduling in a dataset management system
Systems and methods for maintaining a project schedule in a dataset management system are provided. For example, a progress update comprising an indication of a status of training of a first machine learning algorithm may be received using at least one communication device from an external device involved in the training the first machine learning algorithm. A project schedule record may be accessed. An expected running time of the training of the first machine learning algorithm may be updated in the project schedule record based on the status of the training of the first machine learning algorithm. Information related to training of a second machine learning algorithm may be updated in the project schedule record based on the status of the training of the first machine learning algorithm.
1. A system for maintaining a project schedule in a dataset management system, the system comprising:
at least one communication device;
at least one storage device configured to store a project schedule record; and
at least one processor configured to:
use the at least one communication device to receive from an external device involved in training a first machine learning algorithm a progress update comprising an indication of a status of the training of the first machine learning algorithm;
access the project schedule record;
update an expected running time of the training of the first machine learning algorithm in the project schedule record based on the status of the training of the first machine learning algorithm;
update information related to training of a second machine learning algorithm in the project schedule record based on the status of the training of the first machine learning algorithm;
analyze the project schedule record to predict a delay due to the expected running time of the training of the first machine learning algorithm;
determine a probability that the predicted delay is longer than a first selected threshold; and
when the probability that the predicted delay is longer than a first selected threshold is above a second selected threshold, provide an indication in response to the predicted delay.
2. The system of claim 1 , wherein the provided indication comprises a suggestion to avoid the predicted delay by replacing the first machine learning algorithm with a third machine learning algorithm that is predicted to take shorter processing time.
3. The system of claim 1 , wherein the training of the first machine learning algorithm uses a first hyper-parameter, and the provided indication comprises a suggestion to avoid the predicted delay by replacing the first hyper-parameter with a second hyper-parameter that is predicted to take shorter processing time.
4. The system of claim 1 , wherein the training of the first machine learning algorithm uses a first dataset, and the provided indication comprises a suggestion to avoid the predicted delay by using a substitute smaller dataset that is predicted to shorten processing time instead of the first dataset.
5. The system of claim 1 , wherein the progress update comprises an indication of a failure due to annotations that are incompatible with the training of the first machine learning algorithm, and the at least one processor is further configured to suggest alternative annotations.
6. A non-transitory computer readable medium storing data and computer implementable instructions for carrying out a method for maintaining a project schedule in a dataset management system, the method comprising:
using at least one communication device to receive from an external device involved in training a first machine learning algorithm a progress update comprising an indication of a status of the training of the first machine learning algorithm;
accessing a project schedule record;
updating an expected running time of the training of the first machine learning algorithm in the project schedule record based on the status of the training of the first machine learning algorithm;
updating information related to training of a second machine learning algorithm in the project schedule record based on the status of the training of the first machine learning algorithm;
analyzing the project schedule record to predict a delay due to the expected running time of the training of the first machine learning algorithm;
determining a probability that the predicted delay is longer than a first selected threshold; and
when the probability that the predicted delay is longer than a first selected threshold is above a second selected threshold, providing an indication in response to the predicted delay.
7. The non-transitory computer readable medium of claim 6 , wherein the progress update comprises an indication of a failure due to a failure to converge of the first machine learning algorithm, and the method further comprising modifying the project schedule record to use different machine learning algorithm in response to the failure.
8. The non-transitory computer readable medium of claim 6 , wherein the provided indication comprises a suggestion to avoid the predicted delay by replacing the first machine learning algorithm with a third machine learning algorithm that is predicted to take shorter processing time.
9. The non-transitory computer readable medium of claim 6 , wherein the training of the first machine learning algorithm uses a first hyper-parameter, and the provided indication comprises a suggestion to avoid the predicted delay by replacing the first hyper-parameter with a second hyper-parameter that is predicted to take shorter processing time.
10. The non-transitory computer readable medium of claim 6 , wherein the training of the first machine learning algorithm uses a first dataset, and the provided indication comprises a suggestion to avoid the predicted delay by using a substitute smaller dataset that is predicted to shorten processing time instead of the first dataset.
11. The non-transitory computer readable medium of claim 6 , wherein the training of the first machine learning algorithm uses a first device, and the provided indication comprises a suggestion to avoid the predicted delay by replacing the first device with a more powerful device that is predicted to finish the training of the first machine learning algorithm faster.
12. The non-transitory computer readable medium of claim 6 , wherein the method further comprises:
determining that the predicted delay is shorter than a selected threshold; and
based on said determination, withholding providing the indication.
13. The non-transitory computer readable medium of claim 6 , wherein the progress update comprises an indication of a failure due to a missing GPU, and the method further comprising:
selecting an alternative device; and
modifying the project schedule record to use the selected alternative device.
14. The non-transitory computer readable medium of claim 6 , wherein the progress update comprises an indication of a failure due to a dataset with too few examples for the first machine learning algorithm, and the method further comprising modifying the project schedule record to use an alternative dataset.
15. The non-transitory computer readable medium of claim 6 , wherein the progress update comprises an indication of a failure due to annotations that are incompatible with the training of the first machine learning algorithm, and the method further comprising suggesting alternative annotations.
16. The non-transitory computer readable medium of claim 6 , wherein the method further comprises:
determining that an amount associated with the progress update is below a selected threshold; and
withholding updating the project schedule record based on said determination.
17. A method for maintaining a project schedule in a dataset management system, the method comprising:
using at least one communication device to receive from an external device involved in training a first machine learning algorithm a progress update comprising an indication of a status of the training of the first machine learning algorithm;
accessing a project schedule record;
updating an expected running time of the training of the first machine learning algorithm in
the project schedule record based on the status of the training of the first machine learning algorithm;
updating information related to training of a second machine learning algorithm in the project schedule record based on the status of the training of the first machine learning algorithm;
analyzing the project schedule record to predict a delay due to the expected running time of the training of the first machine learning algorithm;
determining a probability that the predicted delay is longer than a first selected threshold; and
when the probability that the predicted delay is longer than a first selected threshold is above a second selected threshold, providing an indication in response to the predicted delay.