Scaling by dynamic discovery for data processing
Aspects described herein may relate to methods, systems, and apparatuses that partitions searchable content and distributes the segments across a plurality of processing nodes, which in turn further sub-partitions the partitions for processing by local search actor in order to increase the speed with which a search request from a user is processed. Processing nodes available to receive partitioned searchable content are registered with an external storage device. The external storage device also maintains a global results collector that compiles results from the partitions of searchable content. Respective local collector actors receive compiled results from local search actors for a processing node and the compiled results are sent to the global results collector for compiling for the plurality of processing nodes. Results of the user search request are then provided to the user.
1. A method comprising:
generating, in a first computing device, a global results collector, the global results collector configured to compile results received from at least two second computing devices;
receiving, by the global results collector, a universally unique identifier (UUID) of a third computing device, different from the first computing device and the at least two second computing devices, requesting a search of content and a number of partitions of the content to be searched;
sending, to each of the at least two second computing devices, a respective one of the number of partitions and reference data of the global results collector;
receiving, by the global results collector from each of the at least two second computing devices, an expected compiled result list comprising data representative of an inventory of items matching at least one user specified criterion of the content to be searched; and
sending, by the first computing device to the third computing device, a response to the search of content, wherein the response comprises results including the expected compiled result list from each of the at least two second computing devices and financing offers associated with each item in the inventory of items.
2. The method of claim 1 , further comprising registering the global results collector with an external storage device maintaining a list of the at least two second computing devices available for assisting in processing the search of content.
3. The method of claim 1 , wherein the response comprises results matching a criterion associated with a user.
4. The method of claim 1 , wherein the global results collector further is configured to compile results received from a local collector actor in the first computing device.
5. The method of claim 1 , wherein the content to be searched comprises vehicle identification numbers (VINs).
6. The method of claim 5 , wherein the response comprises VINs matching the at least one user specified criterion of the content to be searched.
7. The method of claim 1 , wherein the number of partitions is based upon the user specified criterion.
8. The method of claim 1 , further comprising, accessing, by the first computing device, an internal memory maintaining the financing offers associated with each item in the inventory of items.
9. The method of claim 1 , further comprising, accessing, by the first computing device, an external memory maintaining the financing offers associated with each item in the inventory of items.
10. The method of claim 1 , further comprising receiving, by the global results collector from each of the at least two computing devices, financing offers associated with each item in the inventory of items received from the respective at least one second computing device.
11. A first computing device comprising:
one or more memories; and
one or more processors communicatively coupled to the one or more memories, configured to:
generate a global results collector, the global results collector configured to compile results received from at least two second computing devices;
receive, by the global results collector, a universally unique identifier (UUID) of a third computing device, different from the first computing device and the at least two second computing devices, requesting a search of content and a number of partitions of the content to be searched;
send, to each of the at least two second computing devices, a respective one of the number of partitions and reference data of the global results collector;
receive, by the global results collector from each of the at least two second computing devices, an expected compiled result list comprising data representative of an inventory of items matching at least one user specified criterion of the content to be searched; and
send, to the third computing device, a response to the search of content, wherein the response comprises results including the expected compiled result list from each of the at least two second computing devices and financing offers associated with each item in the inventory of items.
12. The first computing device of claim 11 , wherein the one or more processors, are further configured to register the global results collector with an external storage device maintaining a list of the at least two second computing devices available for assisting in processing the search of content.
13. The first computing device of claim 11 , wherein the global results collector further is configured to compile results received from a local collector actor in the first computing device.
14. The first computing device of claim 11 , wherein the response comprises VINs matching the at least one user specified criterion of the content to be searched.
15. The first computing device of claim 11 , further comprising an internal memory maintaining the financing offers associated with each item in the inventory of items, wherein the one or more processors, are further configured access the internal memory to obtain data representative of the financing offers.
16. The first computing device of claim 11 , wherein the one or more processors, are further configured access an external memory to obtain the financing offers associated with one or more item in the inventory of items.
17. A non-transitory computer readable medium storing instructions, that, when executed by one or more processors, cause a first computing device to:
generate a global results collector, the global results collector configured to compile results received from at least two second computing devices;
receive, by the global results collector, a universally unique identifier (UUID) of a third computing device, different from the first computing device and the at least two second computing devices, requesting a search of content and a number of partitions of the content to be searched;
send, to each of the at least two second computing devices, a respective one of the number of partitions and reference data of the global results collector;
receive, by the global results collector from each of the at least two second computing devices, an expected compiled result list comprising data representative of an inventory of items matching at least one user specified criterion of the content to be searched; and
send, to the third computing device, a response to the search of content, wherein the response comprises results including the expected compiled result list from each of the at least two second computing devices and financing offers associated with each item in the inventory of items.
18. The non-transitory computer readable medium of claim 17 , wherein the response comprises VINs matching the at least one user specified criterion of the content to be searched.
19. The non-transitory computer readable medium of claim 17 , wherein the instructions, when executed by the one or more processors, further cause the first computing device to access an internal memory of the first computing device to obtain data representative of the financing offers associated with each item in the inventory of items.
20. The non-transitory computer readable medium of claim 17 , wherein the instructions, when executed by the one or more processors, further cause the first computing device to access memory, external from the first computing device, to obtain data representative of the financing offers associated with each item in the inventory of items.