Efficient random sampling from dynamically changing sample pool
A method for cost-efficient repeated random sampling from a dynamically-changing sampling pool includes defining an array of elements to be selectively masked and unmasked throughout repeated random sampling operations from a first sampling pool. The first sampling pool includes unmasked elements of the array and excludes masked elements of the array. The method further includes identifying an exclusion element within the first sampling pool that is to be excluded from a sampling operation and removing the exclusion element from the first sampling pool. Removing the exclusion element is achieved by moving the exclusion element to a new position by swapping an array index of the exclusion element with an array index that was, during an immediately prior sampling operation, included within and bounding the first sampling pool and by masking the array index corresponding to a new position of the exclusion element. Following the swapping and masking operations, the first sampling pool is randomly sampled.
1. A processor-implemented method for cost-efficiently performing repeated random sampling operations from a first sampling pool, the method comprising:
defining a first array of elements to be selectively masked and unmasked throughout the repeated random sampling operations from the first sampling pool, the first sampling pool including unmasked elements of the first array and excluding masked elements of the first array;
defining a dummy array for tracking locations of elements that are moved between different positions within the first array, the dummy array storing identifiers for the elements in the first array in association with pointers to current locations of the elements within the first array of elements;
identifying an element within the first sampling pool that is to be excluded from a sampling operation;
removing the element from the first sampling pool by:
moving the element to a new position by swapping a first array index of the element with a second array index that was, during an immediately prior sampling operation, included within and bounding the first sampling pool; and
masking the second array index corresponding to the new position of the element;
updating a pointer associated with the element the dummy array to point to the second array index instead of the first array index; and
implementing the sampling operation by randomly sampling from the first sampling pool;
subsequent to the sampling operation, selectively adding the element back into the first sampling pool by:
swapping the second array index of the element with a third array index that was, during an immediately prior sampling operation, external to and bounding the first sampling pool;
updating the pointer associated with the element in the dummy array to point to the third array index instead of the second array index; and
unmasking the third array index.
2. The processor-implemented method of claim 1 , wherein masking the second array index corresponding to the new position of the element further comprises decrementing a size of the first sampling pool.
3. The processor-implemented method of claim 1 , wherein moving the element to the new position further comprises:
swapping the first array index of the element with an array index of a highest-indexed unmasked element in the first array; and wherein masking the second array index corresponding to the new position further comprises:
masking the array index of the highest-indexed unmasked element.
4. The processor-implemented method of claim 1 , wherein unmasking the third array index further comprises incrementing a size of the first sampling pool.
5. The processor-implemented method of claim 4 , wherein adding the element back into the first sampling pool further comprises:
unmasking an array index of a lowest-indexed masked element in the first array and swapping the second array index of the element with the unmasked array index.
6. The processor-implemented method of claim 1 , wherein the first array is a source array that includes data sources available to provide data for migration in a data allocation system, and wherein randomly sampling from the first sampling pool further comprises:
randomly sampling a data source from the source array;
identifying a collection of data chunks at the sampled data source available for migration; and
sampling a data chunk from the collection of data chunks according to a predefined probability distribution.
7. The processor-implemented method of claim 6 , further comprising:
randomly sampling a data destination from a destination array that includes data destinations available to receive migrated data in the data allocation system; and
migrating data from the sampled data source to the sampled data destination.
8. The processor-implemented method of claim 1 , wherein identifying the element further comprises determining that the element violates a defined constraint.
9. The processor-implemented method of claim 1 , wherein selectively adding the element back into the first sampling pool is performed in response to determining that the element is no longer in violation of a defined constraint.
10. A system for cost-efficiently performing repeated random sampling from a first sampling pool, the system comprising:
a first array defining elements eligible to be selectively masked and unmasked throughout repeated random sampling from the first sampling pool, the first sampling pool including unmasked elements of the first array and excluding masked elements of the first array;
a dummy array that identifies current locations of masked and unmasked elements in the first array, the dummy array storing an identifier for each element in the first array in association with a pointer that identifies a current location of the element within the first array;
a random sampling management tool stored in memory and executable to:
identify an element within the first sampling pool that is to be excluded from a sampling operation;
remove the element from the first sampling pool by:
moving the element to a new position by swapping a first array index of the element with a second array index that was, during an immediately prior sampling operation, included within and bounding the first sampling pool; and
masking the second array index corresponding to the new position of the element;
updating a pointer associated with the element in the dummy array to point to the second array index instead of the first array index; and
implementing the sampling operation by randomly sampling from the first sampling pool;
subsequent to the sampling operation, selectively add the element back into the first sampling pool by:
swapping the second array index of the element with a third array index that was, during an immediately prior sampling operation, external to and bounding the first sampling pool;
updating the pointer associated with the element the dummy array to point to the third array index instead of the second array index; and
unmasking the third array index.
11. The system of claim 10 , wherein masking the second array index corresponding to the new position further comprises decrementing a size of the first sampling pool.
12. The system of claim 10 , wherein the random sampling management tool removes the element from the first sampling pool by performing operations including:
swapping the first array index of the element with an array index of a highest-indexed unmasked element in the first array and wherein masking the array index corresponding to the new position further comprises masking the element at the array index of the highest-indexed unmasked element.
13. The system of claim 10 , wherein unmasking the third array index further comprises incrementing a size of the first sampling pool.
14. The system of claim 10 , wherein adding the element back into the first sampling pool further includes:
unmasking an array index of a lowest-indexed masked element in the first array and swapping the second array index of the element with the unmasked array index.
15. The system of claim 10 , wherein in the random sampling management tool identifies the element by determining that the element violates a defined constraint.
16. The system of claim 10 , wherein the random sampling management tool selectively adds the element back into the first sampling pool in response to determining that the element is no longer in violation of a defined constraint.
17. The system of claim 10 , wherein the first array is a source array that includes data sources available to provide data for migration in a data allocation system, and wherein implementing the sampling operation further comprises:
randomly sampling a data source from the source array;
randomly sampling a data destination from a destination array that includes data destinations available to receive migrated data in the data allocation system; and
migrating data from the sampled data source to the sampled data destination.
18. One or more computer-readable storage media encoding computer-executable instructions for executing a computer process, the computer process comprising:
defining a first array of elements to be selectively masked and unmasked throughout repeated random sampling operations from a first sampling pool, the first sampling pool including unmasked elements of the first array and excluding masked elements of the first array;
defining a dummy array for tracking locations of elements that are moved between different positions within the first array, the dummy array storing identifiers for the elements in the first array in association with pointers to current locations of the elements within the first array of elements;
identifying an element within the first sampling pool that is to be excluded from a sampling operation;
removing the element from the first sampling pool by:
moving the element to a new position by swapping first array index of the element with a second array index that was, during an immediately prior sampling operation, included within and bounding the first sampling pool;
masking the second array index corresponding to the new position of the element;
updating a pointer associated with the element the dummy array to point to the second array index instead of the first array index; and
implementing the sampling operation by randomly sampling from the first sampling pool; and
subsequent to the sampling operation, selectively adding the element back into the first sampling pool by operations that include:
swapping the second array index of the element with a third array index that was, during an immediately prior sampling operation, external to and bounding the first sampling pool;
updating the pointer associated with the element the dummy array to point to the third array index instead of the second array index; and
unmasking the third array index.