Granted Patent B2

US 11,429,626 · App. 17/244,931 · Granted Aug 30, 2022

Inventors: Pengfei Su (Shanghai, CN); Julius Jian Zhu (Shanghai, CN); Lingling Yao (Shanghai, CN)

Assignee: EMC IP HOLDING COMPANY LLC

G06F16/2477G06F16/2228G06F16/248G06F16/24568G06F16/24573

Abstract

Claims (61)

1. A method, comprising:

receiving, by a system comprising a processor, multiple events comprised in a data stream to be stored in a storage system;

determining at least one event among the multiple events that occurs within a time window according to a defined length of the time window and occurrence times of the multiple events;

creating a window index node based on the at least one event, the window index node comprising an index of the at least one event; and

adding the window index node to the index in response to determining that a current time point satisfies a threshold time point corresponding to the time window, the threshold time point indicating that a number of received events that occur within the time window in the data stream reaches at least a threshold number.

2. The method according to claim 1 , wherein the creating the window index node based on the at least one event comprises:

creating multiple segment index nodes for the time window using multiple computing resources in the storage system, respectively, wherein segment index nodes in the multiple segment index nodes correspond to events in the at least one event that are received by computing resources in the multiple computing resources; and

adding the segment index nodes to the window index node.

3. The method according to claim 2 , further comprising: setting a state of the segment index nodes in the window index node to a read-only state.

4. The method according to claim 2 , further comprising:

creating another segment index node based on an event in the data stream, in response to determining that the event is received after the threshold time point; and

adding the other segment index node to the window index node.

5. The method according to claim 2 , further comprising at least one of:

merging a first plurality of segment index nodes in the multiple segment index nodes; or

merging a second plurality of segment index nodes in two successive window index nodes in response to determining that a total length of two successive time windows corresponding to the two successive window index nodes in the index satisfies a defined length condition.

6. The method according to claim 2 , further comprising:

searching the index for a window index node corresponding to an expiration condition in response to receiving a removal request for removing an event that satisfies the expiration condition from the storage system; and

removing the event associated with the window index node from the storage system.

7. The method according to claim 6 , wherein the removing the event associated with the window index node from the storage system comprises at least one of:

removing all events associated with the window index node from the storage system in response to determining that an end time of the window index node satisfies the expiration condition; or

removing at least some events associated with at least some segment index nodes in the multiple segment index nodes from the storage system based on the multiple segment index nodes and the expiration condition, in response to the end time not satisfying the expiration condition time and a start time of the window index node satisfying the expiration condition.

8. The method according to claim 7 , further comprising at least one of:

updating the index based on all the events that have been removed from the storage system; or

updating the index based on at least the some events that have been removed from the storage system.

9. The method according to claim 1 , further comprising:

searching the index for at least one window index node corresponding to a specified time condition in response to receiving a query request to query for an event that satisfies the specified time condition in the storage system; and

obtaining the event that satisfies the specified time condition based on the at least one window index node.

10. The method according to claim 1 , wherein the threshold time point is determined based on the number of received events that occur within the time window, and wherein the method is executed in parallel at multiple computing resources in the storage system.

11. A device, comprising:

at least one processor; and

a memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the device to perform operations, comprising:

receiving multiple events comprised in a data stream to be stored in a storage system;

determining at least one event among the multiple events that occurs within a time window according to a predetermined length of the time window and occurrence times of the multiple events;

creating a window index node based on the at least one event, the window index node comprising an index of the at least one event; and

adding the window index node to the index in response to determining that a current time point meets a threshold time point corresponding to the time window, the threshold time point indicating that a number of received events that occur within the time window in the data stream has reached a threshold number.

12. The device according to claim 11 , wherein the creating the window index node based on the at least one event comprises:

creating multiple segment index nodes for the time window using multiple computing resources in the storage system, respectively, wherein segment index nodes in the multiple segment index nodes correspond to ones of the at least one event that are received by computing resources in the multiple computing resources; and

adding the segment index nodes to the window index node.

13. The device according to claim 12 , wherein the operations further comprise: setting the segment index nodes in the window index node to be in a read-only state.

14. The device according to claim 12 , wherein the operations further comprise:

creating another segment index node based on an event in the data stream, in response to determining that the event is received after the threshold time point; and

adding the other segment index node to the window index node.

15. The device according to claim 12 , wherein the operations further comprise at least one of:

merging a first group of segment index nodes in the multiple segment index nodes; or

merging a second group of segment index nodes in two successive window index nodes in response to determining that a total length of two successive time windows corresponding to the two successive window index nodes in the index meets a predetermined length condition.

16. The device according to claim 12 , wherein the operations further comprise:

searching the index for a window index node corresponding to an expiration condition in response to receiving a removal request for removal of an event that meets the expiration condition from the storage system; and

removing the event associated with the window index node from the storage system.

17. The device according to claim 16 , wherein the removing the event associated with the window index node from the storage system comprises at least one of:

removing all events associated with the window index node from the storage system in response to determining that an end time of the window index node meets the expiration condition;

removing a group of events associated with at least part of segment index nodes of the multiple segment index nodes from the storage system based on the multiple segment index nodes and the expiration condition, in response to the end time not meeting the expiration condition time and a start time of the window index node meeting the expiration condition; or

updating the index based on at least one of all the events associated with the window index node from the storage system or the group of events that have been removed from the storage system.

18. The device according to claim 11 , wherein the operations further comprise:

searching the index for a window index node corresponding to a specified time condition in response to receiving a query request to query for an event that meets the specified time condition in the storage system; and

obtaining the event that meets the specified time condition based on the window index node.

19. The device according to claim 11 , wherein the threshold time point is determined based on the number of events that occur within the time window, and the operations are executed in parallel at multiple computing resources in the storage system.

20. A computer program product that is stored on a non-transitory computer-readable medium and comprises machine-executable instructions, that when executed, cause a system comprising a processor to perform operations, comprising:

receiving events comprised in a data stream to be stored in a storage system;

determining an event of the events that occurs within a time window according to a defined length of the time window and occurrence times of the events;

based on the event, creating a window index node comprising an index of the event; and

in response to determining that a current time satisfies a function of a threshold time corresponding to the time window, adding the window index node to the index, wherein the function of the threshold time being satisfied indicates that a number of received events corresponding to times within the time window in the data stream is at least a threshold number.

Assignments (10)

Priority Claims (1)

Continuity (1)