Method and system for managing a document search index

1. A method for managing a document search index, comprising:

obtaining indexing terms for documents in a document repository to generate a plurality of search index fragments;

storing the plurality of search index fragments in the document search index;

constructing a hierarchical structure from a first plurality of stored search index fragments, wherein the first plurality of stored search index fragments is arranged by size in the hierarchical structure and wherein the hierarchical structure comprises at least one virtual element that connects at least two of the first plurality of stored search index fragments in the hierarchical structure;

selecting, based on a minimal size, a second plurality of stored search index fragments from the hierarchical structure;

merging the second plurality of stored search index fragments to obtain a larger search index fragment, wherein the at least one virtual element comprises the larger search index fragment;

storing the larger search index fragment in the document search index; and

serving at least one search request using the larger search index fragment.

2. The method of claim 1 , wherein a cardinality of the first plurality of stored search index fragments is governed by a target value of the stored search index fragments in the document search index.

3. The method of claim 1 , wherein the hierarchical structure is a Huffman tree, and wherein the first plurality of stored search index fragments is organized, in the Huffman tree, by search index fragment size.

4. The method of claim 1 , wherein merging the second plurality of stored search index fragments comprises:

obtaining indexing terms from each of the stored search index fragments in the second plurality of search index fragments; and

storing the obtained indexing terms in the larger search index fragment.

5. The method of claim 1 , wherein a cardinality of the second plurality of search index fragments is governed by a maximum merge threshold that specifies a maximum number of search index fragments to be merged into the larger search index fragment.

6. The method of claim 1 , further comprising:

making a determination that additional stored search index fragments are remaining, after constructing the hierarchical structure;

based on the determination:

selecting, based on minimal size of the remaining search index fragments, a third plurality of search index fragments from the stored remaining search index fragments; and

arranging the third plurality of search index fragments in a second hierarchical structure.

7. The method of claim 1 , further comprising:

making a determination that after selecting the second plurality of search index fragments, non-selected search index fragments are remaining in the hierarchical structure, and based on the determination:

selecting, based on minimal size of the non-selected search index fragments, a third plurality of stored search index fragments;

merging the third plurality of stored search index fragments, to obtain a second larger search index fragment; and

storing the second larger search index fragment in the document search index.

8. The method of claim 1 , further comprising:

adding a document to the document repository;

obtaining additional indexing terms for the added document; and

storing the additional indexing terms in an additional search index fragment in the document search index.

9. A non-transitory computer readable medium (CRM) comprising instructions that enable a system for managing a document search index to:

obtain indexing terms for documents in a document repository to generate a plurality of search index fragments;

store the plurality of search index fragments in the document search index;

construct a hierarchical structure from a first plurality of stored search index fragments, wherein the first plurality of stored search index fragments is arranged by size in the hierarchical structure and wherein the hierarchical structure comprises at least one virtual element that connects at least two of the first plurality of stored search index fragments in the hierarchical structure;

select, based on a minimal size, a second plurality of stored search index fragments from the hierarchical structure;

merge the second plurality of stored search index fragments to obtain a larger search index fragment, wherein the at least one virtual element comprises the larger search index fragment;

store the larger search index fragment in the document search index; and

serve at least one search request using the larger search index fragment.

10. The non-transitory CRM of claim 9 , wherein the hierarchical structure is a Huffman tree, and wherein the first plurality of stored search index fragments is organized, in the Huffman tree, by search index fragment size.

11. The non-transitory CRM of claim 9 , wherein merging the second plurality of stored search index fragments comprises:

obtaining indexing terms from each of the stored search index fragments in the second plurality of search index fragments; and

storing the obtained indexing terms in the larger search index fragment.

12. The non-transitory CRM of claim 9 , wherein a cardinality of the second plurality of search index fragments is governed by a maximum merge threshold that specifies a maximum number of search index fragments to be merged into the larger search index fragment.

13. The non-transitory CRM of claim 9 , wherein the instructions further enable the system for managing the document search index to:

make a determination that additional stored search index fragments are remaining, after constructing the hierarchical structure;

based on the determination:

select, based on minimal size of the remaining search index fragments, a third plurality of search index fragments from the stored remaining search index fragments; and

arrange the third plurality of search index fragments in a second hierarchical structure.

14. The non-transitory CRM of claim 9 , wherein the instructions further enable the system for managing document search index to:

make a determination that after selecting the second plurality of search index fragments, non-selected search index fragments are remaining in the hierarchical structure, and based on the determination:

select, based on minimal size of the non-selected search index fragments, a third plurality of stored search index fragments;

merge the third plurality of stored search index fragments, to obtain a second larger search index fragment; and

store the second larger search index fragment in the document search index.

15. A system for managing a document search index, the system comprising:

a document management service; and

a repository server comprising a document repository and the document search index;

wherein the document management service:

obtains indexing terms for documents in the document repository to generate a plurality of search index fragments;

store the plurality of search index fragments in the document search index;

construct a hierarchical structure from a first plurality of stored search index fragments, wherein the selected first plurality of stored search index fragments is arranged by size in the hierarchical structure and wherein the hierarchical structure comprises at least one virtual element that connects at least two of the first plurality of stored search index fragments in the hierarchical structure;

select, based on a minimal size, a second plurality of stored search index fragments from the hierarchical structure;

merge the second plurality of stored search index fragments to obtain a larger search index fragment, wherein the at least one virtual element comprises the larger search index fragment;

store the larger search index fragment in the document search index; and

serve at least one search request using the larger search index fragment.

16. The system for managing the document search index of claim 15 , wherein the hierarchical structure is a Huffman tree, and wherein the first plurality of stored search index fragments is organized, in the Huffman tree, by search index fragment size.

17. The system for managing the document search index of claim 15 , wherein merging the second plurality of stored search index fragments comprises:

obtaining indexing terms from each of the stored search index fragments in the second plurality of search index fragments; and

storing the obtained indexing terms in the larger search index fragment.

18. The system for managing the document search index of claim 15 , wherein a cardinality of the second plurality of search index fragments is governed by a maximum merge threshold that specifies a maximum number of search index fragments to be merged into the larger search index fragment.

19. The system for managing the document search index of claim 15 , wherein the document management service further:

makes a determination that additional stored search index fragments are remaining, after constructing the hierarchical structure;

based on the determination:

selects, based on minimal size of the remaining search index fragments, a third plurality of search index fragments from the stored remaining search index fragments; and

arranges the third plurality of search index fragments in a second hierarchical structure.

20. The system for managing the document search index of claim 15 , wherein the document management service further:

makes a determination that after selecting the second plurality of search index fragments, non-selected search index fragments are remaining in the hierarchical structure, and based on the determination:

selects, based on minimal size of the non-selected search index fragments, a third plurality of stored search index fragments;

merges the third plurality of stored search index fragments, to obtain a second larger search index fragment; and

stores the second larger search index fragment in the document search index.