Multi-user search system with methodology for bypassing instant indexing
View Patent ↗Multi-user computer search system with methodology for bypassing instant indexing of documents. In some embodiments, for example, a system for bypassing instant indexing includes a token store storing a set of token for a current version of a document and a tokenizer server configured to tokenize a new version of the document and to generate a set of tokens for the new version of the document. The system further includes an instant indexer server configured to determine tokens to index the document by based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document, to determine whether the new version of the document is a bypass document version or non-bypass document version, and to generate an index mutation for the new version of the document including a specification of whether the generated index mutation is a bypass index mutation or a non-bypass index mutation based on whether the new version of the document is a bypass document version or non-bypass document version. The system also includes an index mutation server configured to provide the generated index mutation to an index server, if the generated index mutation is a non-bypass index mutation, or not provide the generated index mutation to the index server, if the generated index mutation is a bypass index mutation.
1. A computing system comprising:
one or more processors;
storage media;
a plurality of index servers comprising a distributed index, the distributed index configured to index documents by keywords;
an instant indexing mechanism configured to update the distributed index by sending index mutations to the plurality of index servers when new versions of documents are stored with a content management service, wherein a new version of a document is identified by the content management service by a respective document version identifier; and
instructions stored in the storage media for execution by the one or more processors and configured for:
detecting that a file system folder, containing a set of file system files corresponding to a set of documents stored with the content management service, is moved from a first file system location to a second file system location;
based, at least in part, on the detecting that the file system folder is moved, storing a new respective version of each document in the set of documents with the content management service, and bypassing the instant indexing mechanism for each new respective version of the set of documents stored with the content management service, the bypassing the instant indexing mechanism including not sending a respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.
2. The computing system of claim 1 , further comprising instructions stored in the storage media for execution by the one or more processors and configured for:
detecting that a file system file, corresponding to a document stored with the content management service, is updated at a computing device;
wherein the file system file is not contained in the file system folder;
wherein the distributed index indexes a pre-update version of the document; and
sending, to an index server of the plurality of index servers, an index mutation for a post-update version of the document, the index mutation reflecting one or more differences between the pre-update version of the document and the post-update version of the document.
3. The computing system of claim 1 , wherein each new respective version of the set of documents corresponds to deleting a respective file system file, of the set of file system files, from the first file system location.
4. The computing system of claim 1 , wherein each new respective version of the set of documents corresponds to adding a respective file system file, of the set of file system files, at the second file system location.
5. The computing system of claim 1 , wherein the bypassing the instant indexing mechanism includes not storing, in an index mutation journal, the respective index mutation for each new respective version of the set of documents stored with the content management service.
6. The computing system of claim 1 , wherein the bypassing the instant indexing mechanism includes: (a) storing, in an index mutation journal, the respective index mutation for each new version of the set of documents stored with the content management service, but does not include (b) sending the respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.
7. The computing system of claim 6 , further comprising instructions stored in the storage media for execution by the one or more processors and configured for:
storing the respective index mutation in the index mutation journal for each new respective version of the set of documents with a bypass flag set in the respective index mutation; and
determining not to send the respective index mutation to the plurality of index servers for each new respective version of the set of documents based on the bypass flag set in the respective index mutation.
8. A method performed by a computing system comprising one or more processors, the computing system operatively coupled to a plurality of index servers comprising a distributed index, the distributed index configured to index documents by keywords, the computing system coupled to an instant indexing mechanism configured to update the distributed index by sending index mutations to the plurality of index servers when new versions of documents are stored with a content management service, wherein a new version of a document is identified by the content management service by a respective document version identifier, the method comprising:
detecting that a file system folder, containing a set of file system files corresponding to a set of documents stored with the content management service, is moved from a first file system location to a second file system location; and
based, at least in part, on the detecting that the file system folder is moved, storing a new respective version of each document in the set of documents with the content management service, and bypassing the instant indexing mechanism for each new respective version of the set of documents stored with the content management service, the bypassing the instant indexing mechanism including not sending a respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.
9. The method of claim 8 , further comprising:
detecting that a file system file, corresponding to a document stored with the content management service, is updated at a computing device;
wherein the file system file is not contained in the file system folder;
wherein the distributed index indexes a pre-update version of the document; and
sending, to an index server of the plurality of index servers, an index mutation for a post-update version of the document, the index mutation reflecting one or more differences between the pre-update version of the document and the post-update version of the document.
10. The method of claim 8 , wherein each new respective version of the set of documents corresponds to deleting a respective file system file, of the set of file system files, from the first file system location.
11. The method of claim 8 , wherein each new respective version of the set of documents corresponds to adding a respective file system file, of the set of file system files, at the second file system location.
12. The method of claim 8 , wherein the bypassing the instant indexing mechanism includes not storing, in an index mutation journal, the respective index mutation for each new respective version of the set of documents stored with the content management service.
13. The method of claim 8 wherein the bypassing the instant indexing mechanism includes: (a) storing, in an index mutation journal, the respective index mutation for each new version of the set of documents stored with the content management service, but does not include (b) sending the respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.
14. The method of claim 13 , further comprising:
storing the respective index mutation in the index mutation journal for each new respective version of the set of documents with a bypass flag set in the respective index mutation; and
determining not to send the respective index mutation to the plurality of index servers for each new respective version of the set of documents based on the bypass flag set in the respective index mutation.
15. One or more non-transitory computer-readable media storing instructions for execution by a computing system comprising one or more processors, the computing system operatively coupled to a plurality of index servers comprising a distributed index, the distributed index configured to index documents by keywords, the computing system coupled to an instant indexing mechanism configured to update the distributed index by sending index mutations to the plurality of index servers when new versions of documents are stored with a content management service, wherein a new version of a document is identified by the content management service by a respective document version identifier, the instructions, when executed by the computing system, are capable of causing the computing system to perform:
detecting that a file system folder, containing a set of file system files corresponding to a set of documents stored with the content management service, is moved from a first file system location to a second file system location; and
based, at least in part, on the detecting that the file system folder is moved, storing a new respective version of each document in the set of documents with the content management service, and bypassing the instant indexing mechanism for each new respective version of the set of documents stored with the content management service, the bypassing the instant indexing mechanism including not sending a respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.
16. The one or more non-transitory computer-readable media of claim 15 , wherein the instructions, when executed by the computing system, are further capable of causing the computing system to perform:
detecting that a file system file, corresponding to a document stored with the content management service, is updated at a computing device;
wherein the file system file is not contained in the file system folder;
wherein the distributed index indexes a pre-update version of the document; and
sending, to an index server of the plurality of index servers, an index mutation for a post-update version of the document, the index mutation reflecting one or more differences between the pre-update version of the document and the post-update version of the document.
17. The one or more non-transitory computer-readable media of claim 15 , wherein each new respective version of the set of documents corresponds to deleting a respective file system file, of the set of file system files, from the first file system location.
18. The one or more non-transitory computer-readable media of claim 15 , wherein each new respective version of the set of documents corresponds to adding a respective file system file, of the set of file system files, at the second file system location.
19. The one or more non-transitory computer-readable media of claim 15 , wherein the bypassing the instant indexing mechanism includes not storing, in an index mutation journal, the respective index mutation for each new respective version of the set of documents stored with the content management service.
20. The one or more non-transitory computer-readable media of claim 15 , wherein the bypassing the instant indexing mechanism includes: (a) storing, in an index mutation journal, the respective index mutation for each new version of the set of documents stored with the content management service, but does not include (b) sending the respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.