DISTRIBUTED INDEX SEARCH
A distributed search system can comprise a central queue of document-based records and a group of nodes assigned to different partitions. Each partition can store indexes for a set of documents. Nodes in the same partition can independently processing the document-based records off of the central queue to construct the indexes.
1 . A distributed search system comprising:
a central queue of document-based records; and
a group of nodes assigned to different partitions, each partition storing indexes for a group of documents, nodes in the same partition independently processing the document-based records off of the central queue to construct the indexes.
2 . The distributed search system of claim 1 , wherein each node receives documents to create document-based records for the central queue.
3 . The distributed search system of claim 1 , wherein the nodes include a lexicon.
4 . The distributed search system of claims 3 , wherein the nodes include partial analytics data.
5 . The distributed search system of claim 1 , wherein the document-based records include document keys.
6 . The distributed search system of claim 5 , wherein the document keys are hashed to determine the partition whose index is updated.
7 . The distributed search system to claims 1 , wherein the indexing includes indicating what documents are associated with potential search terms.
8 . A distributed search system comprising:
a central queue of document-based records; and
a group of nodes assigned to different partitions, each partition storing indexes for a group of documents, nodes in the same partition independently processing the document-based records off of the central queue to construct the indexes, nodes can receive indexing requests and encode them onto a shared queue wherein the nodes maintain a synchronized lexicon so that aggregated query results can be decoded no matter which partition the results came from and wherein nodes independently maintain their (partial) index data by reading from the central queue.
9 . The distributed search system of claim 8 , wherein each node can receive documents to create document-based records for the central queue.
10 . The distributed search system of claim 8 , wherein the nodes include a lexicon.
11 . The distributed search system of claim 8 , wherein document-based records include document keys.
12 . The distributed search system of claim 11 , wherein the document keys are hashed to determine the partition whose index is updated.
13 . The distributed search system to claims 8 , wherein the indexing includes indicating what documents are associated with potential search terms.
14 . The distributed system of claim 8 , wherein searches include combining results from multiple partitions.
15 . A computer readable medium containing code to:
access a central queue of document-based records; and
maintain an index for a portion of the documents of the distributed search system as indicated by a document ID associated with the document-based records.