System and method of managing capacity of search index partitions
View Patent ↗A search system can maintain a search index of metadata and text for objects in a repository, repositories or distributed across a network. The search index can be divided into partitions with a partition assigned a first capacity utilization threshold and a second capacity utilization threshold. If the capacity utilization of the partition is below the first threshold, the system can add, update and delete information in the partition. If the capacity utilization of the partition is above the first threshold, the system can update and delete information in the partition, but cannot add information for new objects to the partition. If the capacity utilization of the partition is above the second threshold, the system can enter a rebalancing mode in which it seeks to rebalance capacity utilization between partitions. The behavior of the system can change depending upon the size of a partition relative to its configurable thresholds.
1. A computerized search system, comprising:
a search engine communicatively connected to a client device over a physical connection;
a processor; and
a storage unit coupled to the processor and storing a search index and instructions, wherein the instructions when translated by the processor cause the search engine to perform:
when a capacity utilization of a partition in the search index is under a first threshold:
allowing any object to be added to the partition; and
allowing any existing object in the partition to be updated or deleted;
when the capacity utilization of the partition is above the first threshold:
disallowing any object to be added to the partition; and
allowing any existing object in the partition to be updated or deleted; and
when the capacity utilization of the partition is above a second threshold:
disallowing any object to be added to the partition;
allowing any existing object in the partition to be deleted; and
moving any existing object in the partition that is subject to an update to another partition in the search index along with an indexing request.
2. The system of claim 1 , wherein the indexing request is generated by a partition manager based on the update associated with the existing object.
3. The system of claim 2 , wherein the moving is automatically and proactively performed by the partition manager when the capacity utilization of the partition is above the second threshold.
4. The system of claim 2 , wherein the moving is performed by the partition manager in response to another indexing request received at the partition manager.
5. The system of claim 1 , wherein the instructions when translated by the processor further cause the search engine to perform:
changing from an Add mode to an Update mode when the capacity utilization of the partition meets or exceeds the first threshold; and
changing from the Update mode to a Rebalancing mode when the capacity utilization of the partition meets or exceeds the second threshold.
6. The system of claim 1 , wherein the instructions when translated by the processor further cause the search engine to perform:
changing from an Update mode to an Add mode when the capacity utilization of the partition drops below the first threshold.
7. The system of claim 1 , wherein the instructions when translated by the processor further cause the search engine to perform:
changing from a Rebalancing mode to an Update mode when the capacity utilization of the partition drops below a third threshold.
8. A computer program product comprising at least one non-transitory computer readable medium storing instructions translatable by a processor, wherein the instructions when translated by the processor cause a search engine to perform:
when a capacity utilization of a partition in a search index is under a first threshold:
allowing any object to be added to the partition; and
allowing any existing object in the partition to be updated or deleted;
when the capacity utilization of the partition is above the first threshold:
disallowing any object to be added to the partition; and
allowing any existing object in the partition to be updated or deleted; and
when the capacity utilization of the partition is above a second threshold:
disallowing any object to be added to the partition;
allowing any existing object in the partition to be deleted; and
moving any existing object in the partition that is subject to an update to another partition in the search index along with an indexing request, the search index being connected to the search engine, the search engine being communicatively connected to a client device over a physical connection.
9. The computer program product of claim 8 , wherein the indexing request is generated by a partition manager based on the update associated with the existing object.
10. The computer program product of claim 9 , wherein the moving is automatically and proactively performed by the partition manager when the capacity utilization of the partition is above the second threshold.
11. The computer program product of claim 9 , wherein the moving is performed by the partition manager in response to another indexing request received at the partition manager.
12. The computer program product of claim 8 , wherein the instructions when translated by the processor further cause the search engine to perform:
changing from an Add mode to an Update mode when the capacity utilization of the partition meets or exceeds the first threshold; and
changing from the Update mode to a Rebalancing mode when the capacity utilization of the partition meets or exceeds the second threshold.
13. The computer program product of claim 8 , wherein the instructions when translated by the processor further cause the search engine to perform:
changing from an Update mode to an Add mode when the capacity utilization of the partition drops below the first threshold.
14. The computer program product of claim 8 , wherein the instructions when translated by the processor further cause the search engine to perform:
changing from a Rebalancing mode to an Update mode when the capacity utilization of the partition drops below a third threshold.
15. A method for managing partitions in a computerized search system, the method performed by a search engine embodied on at least one non-transitory computer readable medium including instructions translable by a processor, the method comprising:
when a capacity utilization of a partition in a search index is under a first threshold:
allowing any object to be added to the partition; and
allowing any existing object in the partition to be updated or deleted;
when the capacity utilization of the partition is above the first threshold:
disallowing any object to be added to the partition; and
allowing any existing object in the partition to be updated or deleted; and
when the capacity utilization of the partition is above a second threshold:
disallowing any object to be added to the partition;
allowing any existing object in the partition to be deleted; and
moving any existing object in the partition that is subject to an update to another partition in the search index along with an indexing request, the search index being connected to the search engine, the search engine being communicatively connected to a client device over a physical connection.
16. The method of claim 15 , wherein the indexing request is generated by a partition manager based on the update associated with the existing object.
17. The method of claim 16 , wherein the moving is automatically and proactively performed by the partition manager when the capacity utilization of the partition is above the second threshold.
18. The method of claim 16 , wherein the moving is performed by the partition manager in response to another indexing request received at the partition manager.
19. The method of claim 15 , further comprising:
changing from an Add mode to an Update mode when the capacity utilization of the partition meets or exceeds the first threshold; and
changing from the Update mode to a Rebalancing mode when the capacity utilization of the partition meets or exceeds the second threshold.
20. The method of claim 15 , further comprising:
changing from a Rebalancing mode to an Update mode when the capacity utilization of the partition drops below a third threshold.