Index lifecycle management
Methods and systems for index lifecycle management are provided. Exemplary methods include: receiving an ILM policy; determining a first condition and a first action for a first phase using the ILM policy; performing the first action for the first phase when the first condition is met; transition from the first phase to a second phase; determining a second condition and a second action for the second phase using the ILM policy; performing the second action for the second phase when the second condition is met; transition from the second phase to a third phase; determining a third condition and a third action for the third phase using the ILM policy; performing the third action for the third phase when the third condition is met; transition from the third phase to a fourth phase; and deleting the index during the third phase.
1. A computer-implemented method for integrated lifecycle management (ILM), the method comprising:
receiving an ILM policy for an index;
determining a first condition and a first action for a first phase for the index using the ILM policy, the index located on a first node;
performing the first action for the first phase when the first condition is met;
transitioning the index from the first phase to a second phase;
moving the index from the first node to a second node, the second node having less computing power and operating expense than the first node;
determining a second condition and a second action for the index for the second phase using the ILM policy;
performing the second action for the second phase when the second condition is met;
transitioning the index from the second phase to a third phase;
moving the index from the second node to a third node, the third node having less computing power and operating expense than the second node;
determining a third condition and a third action for the index for the third phase using the ILM policy;
performing the third action for the third phase when the third condition is met;
transitioning the index from the third phase to a fourth phase; and
deleting the index during the fourth phase.
2. A system for integrated lifecycle management (ILM), the system comprising:
a processor; and
a memory communicatively coupled to the processor, the memory storing instructions executable by the processor to:
receive an ILM policy for an index;
determine a first condition and a first action for the index for a first phase using the ILM policy;
perform the first action for the first phase when the first condition is met;
transition the index from the first phase to a second phase;
move the index from a first node to a second node, the second node having less computing power and operating expense than the first node;
determine a second condition and a second action for the second phase using the ILM policy;
perform the second action for the second phase when the second condition is met;
transition the index from the second phase to a third phase;
move the index from the second node to a third node, the third node having less computing power and operating expense than the second node;
determine a third condition and a third action for the third phase using the ILM policy;
perform the third action for the third phase when the third condition is met;
transition from the third phase to a fourth phase;
move the index from the third node to a fourth node, the fourth node having less computing power and operating expense than the third node;
and
delete the index during the fourth phase.
3. The computer-implemented method of claim 1 , wherein the first, second, and third conditions are at least one of: when the index exceeds a particular age, and when the index exceeds a particular size.
4. The computer-implemented method of claim 1 , wherein:
in the first phase the index is being updated and is searched;
in the second phase the index is no longer being updated and is searched; and
in the third phase the index is no longer being updated and is seldom queried.
5. The computer-implemented method of claim 1 , wherein:
the third action is a shrink action,
the shrink action creates a copy index from the index, and
an index name of the copy index includes a predetermined prefix added to a front of the index name of the index.
6. The computer-implemented method of claim 1 , wherein:
the index is in a leader cluster being replicated in a follower cluster using cross-cluster replication, and
shards of the index in the leader cluster include a history retention lease, the history retention lease denoting operations to be saved for the follower cluster to execute.
7. The system of claim 2 , wherein the first, second, and third conditions are at least one of: when the index exceeds a particular age, and when the index exceeds a particular size.
8. The system of claim 2 , wherein:
in the first phase the index is being updated and is searched;
in the second phase the index is no longer being updated and is searched; and
in the third phase the index is no longer being updated and is seldom queried.
9. The system of claim 2 , wherein:
the third action is a shrink action,
the shrink action creates a copy index from the index, and
an index name of the copy index includes a predetermined prefix added to a front of the index name of the index.
10. The system of claim 2 , wherein:
the index is in a leader cluster being replicated in a follower cluster using cross-cluster replication,
shards of the index in the leader cluster include a history retention lease, the history retention lease denoting operations to be saved for the follower cluster to execute, and
the shards of the index in the leader cluster further include a marker when the index in the leader cluster is rolled over.
11. The computer-implemented method of claim 3 , wherein a default age is 30 days and a default size is 50 GB.
12. The computer-implemented method of claim 3 , wherein the first action can include at least one of: set priority, rollover, and unfollow.
13. The computer-implemented method of claim 6 , wherein:
shards of the index in the leader cluster include a marker when the index in the leader cluster is rolled over.
14. The system of claim 7 , wherein a default age is 30 days and a default size is 50 GB.
15. The system of claim 7 , wherein the first action can include at least one of: set priority, rollover, and unfollow.
16. The computer-implemented method of claim 12 , wherein
the second action can include at least one of: set priority, allocate, read-only, force merge, shrink, and unfollow; and
the third action can include at least one of: set priority, allocate, freeze, and unfollow.
17. The system of claim 15 , wherein:
the second action can include at least one of: set priority, allocate, read-only, force merge, shrink, and unfollow, and
the third action can include at least one of: set priority, allocate, freeze, and unfollow.
18. The computer-implemented method of claim 16 wherein the ILM policy is in JavaScript Object Notation.
19. The system of claim 17 wherein the ILM policy is in JavaScript Object Notation.