Quota management in a dataset management system

1. A system for controlling access through a computer network to a dataset management system using quotas, the system comprising:

at least one communication device;

at least one storage device configured to store a plurality of quota records associated with entities; and

at least one processor configured to:

receive from an external entity, using the at least one communication device, a request to train a machine learning algorithm using information from a dataset management system, the information including a record of annotations;

based on an identity of the external entity and the machine learning algorithm, select a quota record associated with the external entity and the machine learning algorithm of the plurality of quota records;

based on the selected quota record associated with the external entity and the machine learning algorithm, determine whether the external entity has sufficient quota to train the machine learning algorithm using the information from the dataset management system;

based on the selected quota record, determining that the external entity has insufficient quota to train the machine learning algorithm using the information from the dataset management system;

select, from a plurality of alternative records of annotations, at least one substitute record of annotations, based on a similarity between the record of annotations and the at least one substitute record of annotations; and

based on determining that the external entity has an insufficient quota, provide an indication to the external entity, the indication including a suggestion of the at least one substitute record of annotations.

2. A method controlling access through a computer network to a dataset management system using quotas, the method comprising:

receiving from an external entity, using at least one communication device, a request to train a machine learning algorithm using information from a dataset management system, the information including a record of annotations;

based on an identity of the external entity and the machine learning algorithm, selecting a quota record associated with the external entity and the machine learning algorithm of a plurality of quota records;

based on the selected quota record associated with the external entity and the machine learning algorithm, determining whether the external entity has sufficient quota to train the machine learning algorithm using the information from the dataset management system;

based on the selected quota record, determining that the external entity has insufficient quota to train the machine learning algorithm using the information from the dataset management system;

selecting, from a plurality of alternative records of annotations, at least one substitute record of annotations, based on a similarity between the record of annotations and the at least one substitute record of annotations; and

based on determining that the external entity has an insufficient quota, providing an indication to the external entity, the indication including a suggestion of the at least one substitute record of annotations.

3. The method of claim 2 , further comprising:

estimating at least one resource requirement associated with training the machine learning algorithm using the information from the dataset management system; and

further basing the determination that the external entity has sufficient quota to train the machine learning algorithm using the information from the dataset management system on the estimated at least one resource requirement.

4. The method of claim 3 , further comprising:

estimating a distribution of resource requirements associated with training the machine learning algorithm using the information from the dataset management system;

using the estimated distribution of resource requirements and the selected quota record associated with the external entity and the machine learning algorithm to estimate a probability that the external entity has sufficient quota to train the machine learning algorithm using the information; and

using the estimated probability that the external entity has sufficient quota to train the machine learning algorithm using the information from the dataset management system to determine that the external entity has sufficient quota to train the machine learning algorithm using the information from the dataset management system.

5. The method of claim 4 , further comprising:

receiving at least one characteristic of one or more processing devices; and

using the at least one characteristic of the one or more processing devices to estimate the distribution of resource requirements associated with training the machine learning algorithm using the information from the dataset management system.

6. The method of claim 5 , further comprising:

using the distribution of resource requirements associated with training the machine learning algorithm using the information from the dataset management system to select at least one processing device of the one or more processing devices; and

causing the selected at least one processing device to train the machine learning algorithm using the information from the dataset management system.

7. The method of claim 3 , further comprising:

receiving at least one characteristic of one or more processing devices; and

using the at least one characteristic of the one or more processing devices to estimate the at least one resource requirement associated with training the machine learning algorithm using the information from the dataset management system.

8. The method of claim 7 , further comprising:

using the estimated at least one resource requirement associated with training the machine learning algorithm using the information from the dataset management system to select at least one processing device of the one or more processing devices; and

causing the selected at least one processing device to train the machine learning algorithm using the information from the dataset management system.

9. The method of claim 7 , further comprising:

receiving at least one characteristic of an additional processing device;

using the at least one characteristic of one or more processing devices and the at least one characteristic of an additional processing device to construct a function that converts estimated resource requirements associated with the one or more processing devices to estimated resource requirements associated with the additional processing device;

estimating at least one resource requirement associated with the one or more processing devices and with training the machine learning algorithm using the information from the dataset management system; and

using the constructed function to estimate at least one resource requirement associated with the additional processing device and with training the machine learning algorithm using the information from the dataset management system from the estimated at least one resource requirement associated with the one or more processing devices and with training the machine learning algorithm using the information from the dataset management system.

10. The method of claim 2 , further comprising, in response to the determination that the external entity has sufficient quota, training the machine learning algorithm using the information from the dataset management system.

11. The method of claim 2 , further comprising, in response to the determination that the external entity has sufficient quota, updating the quota record.

12. The method of claim 2 , further comprising:

based on the identity of the external entity, selecting a permission record associated with the external entity of a plurality of permission records; and

based on the selected permission record, determining that the external entity has permission to train the machine learning algorithm using the information from the dataset management system.

13. The method of claim 2 , wherein the indication comprises a suggestion of at least one alternative action.

14. The method of claim 2 , wherein the indication comprises a suggestion to increase the quota.

15. The method of claim 2 , wherein training the machine learning algorithm using the information from the dataset management system comprises usage of a dataset, the method further comprises selecting at least one substitute dataset of a plurality of alternative datasets based on a statistical distance between a distribution of elements in the dataset and a distribution of elements in the at least one substitute dataset, and wherein the provided indication comprises a suggestion of the at least one substitute dataset.

16. The method of claim 2 , wherein training the machine learning algorithm using the information from the dataset management system comprises usage of a dataset, the method further comprises selecting at least one substitute dataset of a plurality of alternative datasets based on a Tsallis divergence between a distribution of elements in the dataset and a distribution of elements in the at least one substitute dataset, and wherein the provided indication comprises a suggestion of the at least one substitute dataset.

17. The method of claim 2 , wherein the similarity between the record of annotations and the at least one substitute record of annotations is based on names of labels in the record of annotations.

18. A non-transitory computer readable medium storing data and computer implementable instructions for carrying out a method controlling access through a computer network to a dataset management system using quotas, the method comprising:

receiving from an external entity, using at least one communication device, a request to train a machine learning algorithm using information from a dataset management system, the information including a record of annotations;

based on an identity of the external entity and the machine learning algorithm, selecting a quota record associated with the external entity and the machine learning algorithm of a plurality of quota records;

based on the selected quota record associated with the external entity and the machine learning algorithm, determining whether the external entity has sufficient quota to train the machine learning algorithm using the information from the dataset management system;

based on the selected quota record, determining that the external entity has insufficient quota to train the machine learning algorithm using the information from the dataset management system;

selecting, from a plurality of alternative records of annotations, at least one substitute record of annotations, based on a similarity between the record of annotations and the at least one substitute record of annotations; and

based on determining that the external entity has an insufficient quota, providing an indication to the external entity, the indication including a suggestion of the at least one substitute record of annotations.