Multi-threaded dynamic queries on an unpartitioned database
Systems, devices, and techniques are disclosed for multi-threaded dynamic queries on an unpartitioned database. An instruction to perform a database operation on rows of a database may be received. Processing threads may be generated by generating three random numbers. A starting point of a processing thread may be set to a first random number, an offset may be set to a second random number, and an order may be based on a third random number. The processing threads may each generate a dynamic query. A dynamic query generated by a processing thread may include an instruction to perform the database operation on rows of the database based on the offset, starting point, and order for the processing thread. Each dynamic query may be run against the database to perform the database operation.
1. A computer-implemented method comprising:
receiving an instruction to perform a database operation on rows of a database;
generating two or more processing threads by, for each processing thread of the two or more processing threads:
generating three random numbers,
setting a starting point of the processing thread to a first of the three random numbers,
setting an offset of the processing thread to a second of the three random numbers, and
setting an order of the processing thread based on a third of the three random
numbers wherein the order indicates a direction of either ascending from the starting point or descending from the starting point;
generating, with each of the two or more processing threads, a dynamic query, wherein a dynamic query generated by a processing thread of the two or more processing threads comprises an instruction to perform the database operation on rows of the database that are part of a number of rows of the database based on the offset of the processing thread and located in the direction indicated by the order of the processing thread starting at a row of the database indicated by the starting point of the processing thread;
running each dynamic query against the database to perform the database operation; and
terminating or resetting a processing thread of the two or more processing threads after the processing thread finishes running its dynamic query against the database by performing the database operation on the rows of the database that are part of the number of rows of the database based on the offset of the processing thread and located in the direction indicated by the order of the processing thread starting at the row of the database indicated by the starting point of the processing thread.
2. The computer-implemented method of claim 1 , wherein the database operation is idempotent.
3. The computer-implemented method of claim 1 , further comprising:
generating one or more additional processing threads by, for each processing thread of the one or more additional processing threads:
generating three random numbers,
setting a starting point of the processing thread to a first of the three random numbers,
setting an offset of the processing thread to a second of the three random numbers, and
setting an order of the processing thread based a third of the three random numbers.
4. The computer-implemented method of claim 1 wherein the second of the three random numbers is generated from a range of 1000 to 10000.
5. The computer-implemented method of claim 1 , further comprising:
resetting a processing thread of the two or more processing threads when the database operation has been performed on all of the rows that are part of a number of rows of the database equal to the offset of the processing thread and in a direction indicated by the order of the processing thread starting at a row of the database indicated by the starting point of the processing thread, wherein resetting the processing thread of the two or more processing threads comprises:
generating three additional random numbers;
setting a new starting point of the processing thread to a first of the three additional random numbers,
setting a new offset of the processing thread to a second of the three additional random numbers, and
setting a new order of the processing thread to a third of the three additional random numbers.
6. The computer-implemented method of claim 1 , wherein there are conditions on the database operation, and further comprising:
checking, by each of the two or more processing threads, rows of the database to determine if the rows of the database meet the conditions; and
performing the database operation on rows that meet the conditions on the database operation.
7. The computer-implemented method of claim 1 , wherein the database is not partitioned when the dynamic queries are run against the database.
8. A computer-implemented system comprising:
one or more storage devices comprising a database comprising rows; and
a processor that receives an instruction to perform a database operation on the rows of the database,
generates two or more processing threads by, for each processing thread of the two or more processing threads:
generating three random numbers,
setting a starting point of the processing thread to a first of the three random numbers,
setting an offset of the processing thread to a second of the three random numbers, and
setting an order of the processing thread based on a third of the three random
numbers, wherein the order indicates a direction of either ascending from the starting point or descending from the starting point,
generates, with each of the two or more processing threads, a dynamic query, wherein a dynamic query generated by a processing thread of the two or more processing threads comprises an instruction to perform the database operation on rows of the database that are part of a number of rows of the database based on the offset of the processing thread and located in the direction indicated by the order of the processing thread starting at a row of the database indicated by the starting point of the processing thread, and
runs each dynamic query against the database to perform the database operation; and
terminates or resets a processing thread of the two or more processing threads after the processing thread finishes running its dynamic query against the database by performing the database operation on the rows of the database that are part of the number of rows of the database based on the offset of the processing thread and located in the direction indicated by the order of the processing thread starting at the row of the database indicated by the starting point of the processing thread.
9. The computer-implemented system of claim 8 , wherein the database operation is idempotent.
10. The computer-implemented system of claim 8 , wherein the processor further generates one or more additional processing threads by, for each processing thread of the one or more additional processing threads:
generating three random numbers,
setting a starting point of the processing thread to a first of the three random numbers,
setting an offset of the processing thread to a second of the three random numbers, and
setting an order of the processing thread based a third of the three random numbers.
11. The computer-implemented system of claim 8 , wherein the second of the three random numbers is generated from a range of 1000 to 10000.
12. The computer-implemented system of claim 8 , wherein the processer further resets a processing thread of the two or more processing threads when the database operation has been performed on all of the rows that are part of a number of rows of the database equal to the offset of the processing thread and in a direction indicated by the order of the processing thread starting at a row of the database indicated by the starting point of the processing thread, by:
generating three additional random numbers;
setting a new starting point of the processing thread to a first of the three additional random numbers,
setting a new offset of the processing thread to a second of the three additional random numbers, and
setting a new order of the processing thread to a third of the three additional random numbers.
13. The computer-implemented system of claim 8 , wherein there are conditions on the database operation, and wherein the processor further:
checks, with each of the two or more processing threads, rows of the database to determine if the rows of the database meet the conditions; and
performs the database operation on rows that meet the conditions on the database operation.
14. The computer-implemented system of claim 8 , wherein the database is not partitioned when the dynamic queries are run against the database.
15. A system comprising: one or more computers and one or more non-transitory storage devices storing instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving an instruction to perform a database operation on rows of a database;
generating two or more processing threads by, for each processing thread of the two or more processing threads:
generating three random numbers,
setting a starting point of the processing thread to a first of the three random numbers,
setting an offset of the processing thread to a second of the three random numbers, and
setting an order of the processing thread based on a third of the three random
numbers wherein the order indicates a direction of either ascending from the starting point or descending from the starting point;
generating, with each of the two or more processing threads, a dynamic query, wherein a dynamic query generated by a processing thread of the two or more processing threads comprises an instruction to perform the database operation on rows of the database that are part of a number of rows of the database based on the offset of the processing thread and located in the direction indicated by the order of the processing thread starting at a row of the database indicated by the starting point of the processing thread;
running each dynamic query against the database to perform the database operation; and
terminating or resetting a processing thread of the two or more processing threads after the processing thread finishes running its dynamic query against the database by performing the database operation on the rows of the database that are part of the number of rows of the database based on the offset of the processing thread and located in the direction indicated by the order of the processing thread starting at the row of the database indicated by the starting point of the processing thread.
16. The system of claim 15 , wherein the one or more computers and one or more non-transitory storage devices further store instructions which are operable, when executed by the one or more computers, to cause the one or more computers to further perform operations comprising:
generating one or more additional processing threads by, for each processing thread of the one or more additional processing threads:
generating three random numbers,
setting a starting point of the processing thread to a first of the three random numbers,
setting an offset of the processing thread to a second of the three random numbers, and
setting an order of the processing thread based a third of the three random numbers.
17. The system of claim 15 , wherein the one or more computers and one or more non-transitory storage devices further store instructions which are operable, when executed by the one or more computers, to cause the one or more computers to further perform operations comprising:
resetting a processing thread of the two or more processing threads when the database operation has been performed on all of the rows that are part of a number of rows of the database equal to the offset of the processing thread and in a direction indicated by the order of the processing thread starting at a row of the database indicated by the starting point of the processing thread, wherein resetting the processing thread of the two or more processing threads comprises:
generating three additional random numbers;
setting a new starting point of the processing thread to a first of the three additional random numbers,
setting a new offset of the processing thread to a second of the three additional random numbers, and
setting a new order of the processing thread to a third of the three additional random numbers.