Search engine with two-dimensional linearly scalable parallel architecture
View Patent ↗In a search engine with two-dimensional scalable architecture for searching of a collection of documents, the search engine comprises data processing units which forms set of nodes connected in a network, a first set of nodes comprising dispatch nodes, a second set of nodes search nodes and a third set of nodes indexing nodes. The search nodes are grouped in columns, which via the network are connected in parallel between the dispatch nodes and an indexing node. The dispatch nodes are adapted for processing search queries and search answers, the search nodes are adapted to contain search software, at least some of the search nodes additionally including at least one search processor module and the indexing nodes are adapted for generally generating indexes for the search software. Optionally, acquisition nodes provided in a fourth set of nodes and adapted for processing the search answers, thus relieving the dispatch nodes of this task. The two-dimensional scaling takes place respectively through a scaling of the data volume and a scaling of the search engine performance through a respective adaptation of the architecture.
1. A search engine for searching a collection of documents, the search engine comprising:
at least one dispatch node, the at least one dispatch node being effective to receive and forward a search query from a user of the search engine;
a plurality of search nodes, coupled to the dispatch node, the search nodes each being effective to receive the search query from a respective dispatch node, and to process the search query using a searching algorithm; and
at least one indexing node, coupled to the search nodes, the at least one indexing node effective to create a respective partition-dependent data set for a respective search node; wherein
the search nodes are logically arranged in a table and stored in a computerized memory, the table comprising a plurality of columns and a plurality of a rows, each search node in one of the rows of search nodes including a distinct partition-dependent data set so that a sum of the data sets of all the search nodes in a row yields the collection of documents, and every respective search node in one of the columns of search nodes include substantially the same partition-dependent data set.
2. The search engine as recited in claim 1 , further comprising a plurality of dispatch nodes arranged in a multi-level hierarchical configuration.
3. The search engine as recited in claim 1 , further comprising at least one acquisition node, distinct from the dispatch nodes, the acquisition node effective to acquire results from the search nodes for the search query.
4. The search engine as recited in claim 3 , further comprising a plurality of acquisition nodes arranged in a multi-level hierarchical configuration.
5. The search engine as recited in claim 4 , further comprising a plurality of dispatch nodes arranged in a multi-level hierarchical configuration.
6. The search engine as recited in claim 5 , wherein the configuration of the dispatch nodes is a mirror image of the configuration of the acquisition nodes.
7. The search engine as recited in claim 1 , wherein each search node includes a search software module.
8. The search engine as recited in claim 1 , wherein a number of logical columns corresponds to the number of dispatch nodes.