IP Library Granted Patent US 10,025,824
Granted Patent B2
US 10,025,824 · App. 14/722,950 · Granted Jul 17, 2018

Filter optimizer for data streams

Inventors: Bin Cao (Rochester, MN); Jessica R. Eidem (Rochester, MN); Brian R. Muras (Rochester, MN); Jingdong Sun (Rochester, MN)
Assignee: International Business Machines Corporation
G06F17/30516G06F11/3034G06F17/30463G06F17/30867G06F17/30442G06F17/30486G06F17/30572
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,025,824
App. No.
14/722,950
Granted
Jul 17, 2018
Kind
B2
Abstract

Embodiments of the preset disclosure include a method, a system, and a computer program product for rearranging a set of filters within a data streams environment. The method includes setting a first filter ordering of two or more filter processing elements. The method includes determining a filtering efficiency of the first filter ordering. The determining of the filtering efficiency includes monitoring a runtime, one or more tuples, and an amount of tuples retained after filtering of the first filter ordering. The method includes creating a second filter ordering of the two or more filter processing elements. The method includes comparing the filtering efficiency of the second filter ordering to the first filter ordering. The method includes selecting the second filter ordering in response to comparing the filtering efficiency of the second filter ordering to the filtering efficiency of the first filter ordering.

Claims (63)

1. A system for managing a data streams environment, comprising:

a memory;

a processor device communicatively coupled to the memory; and

a filter optimizer communicatively coupled to the memory and the processor device, wherein the filter optimizer is configured to:

set a first filter ordering of two or more filter processing elements,

determine a filtering efficiency of the first filter ordering by:

monitoring a runtime of the first filter ordering,

monitoring one or more tuples as the one or more tuples pass through the data streams environment, and

monitoring an amount of tuples retained after filtering of the first filter ordering,

create a second filter ordering of the two or more filter processing elements, wherein the second filter ordering of the two or more filter processing elements differ compared to the first filter ordering of the two or more filter processing elements,

compare the filtering efficiency of the second filter ordering to the filtering efficiency of the first filter ordering, wherein comparing the filtering efficiency of the second filter ordering to the filtering efficiency of the first filter ordering comprises:

comparing an amount of tuples retained after filtering of the second filter ordering to the amount of tuples retained after filtering of the first filter ordering; and

comparing a total time used by the second filter ordering to a total time used by the first filter ordering,

wherein the filter optimizer is further configured to:

select the second filter ordering of the two or more filter processing elements, in response to determining that the filtering efficiency of the second filter ordering is greater than the filtering efficiency of the first filter ordering; and

select the first filter ordering of the two or more filter processing elements in response to determining that the filtering efficiency of the second filter ordering is less than the filtering efficiency of the first filter ordering.

2. The system of claim 1 , wherein the filter optimizer is further configured to create a second filter ordering by:

setting the second filter ordering of the two or more filter processing elements that are arranged as the second filter ordering;

determining a filter efficiency of the second filter ordering by:

monitoring a runtime of the second filter ordering, and

monitoring an amount tuples retained after filtering of the second filter ordering.

3. The system of claim 1 , wherein the second filter ordering increases the filtering efficiency by decreasing a runtime when compared to the first filter ordering.

4. The system of claim 1 , wherein the second filter ordering increases the filtering efficiency by decreasing an amount of tuples retained after filtering.

5. The system of claim 1 , wherein a first filter and a second filter can be dependent upon each other creating a learned correlation increasing the filtering efficiency.

6. The system of claim 1 , wherein the filter optimizer is further configured to:

reroute a tuple flow within first filter ordering to a beginning of the first filter ordering;

create the second filter ordering; and

start the tuple flow at a beginning of the second filter ordering.

7. The system of claim 1 , wherein the filter optimizer is further configured to:

place a pause on a tuple flow entering the first filter ordering;

process the tuple flow within the first filter ordering;

create the second filter ordering;

remove the pause that was placed upon the tuple flow; and

resume the tuple flow with the second filter ordering.

8. The system of claim 1 , wherein the filter optimizer is further configured to:

store the first filter ordering of the two or more filter processing elements on the memory;

access the first filter ordering from the memory; and

create the first filter ordering.

9. A computer program product for optimizing a filter ordering within a data streams environment comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

set a first filter ordering of two or more filter processing elements;

determine a filtering efficiency of the first filter ordering by:

monitoring a runtime of the first filter ordering,

monitoring one or more tuples as the one or more tuples pass through the data streams environment, and

monitoring an amount of tuples retained after filtering of the first filter ordering, create a second filter ordering of the two or more filter processing elements, wherein the second filter ordering of the two or more filter processing elements differ compared to the first filter ordering of the two or more filter processing elements;

compare the filtering efficiency of the second filter ordering to the filtering efficiency of the first filter ordering the first filter ordering, wherein comparing the filtering efficiency of the second filter ordering to the filtering efficiency of the first filter ordering comprises:

comparing an amount of tuples retained after filtering of the second filter ordering to the amount of tuples retained after filtering of the first filter ordering; and

comparing a total time used by the second filter ordering to a total time used by the first filter ordering;

wherein the computer readable program further causes the computing device to:

select the second filter ordering of the two or more filter processing elements, in response to determining that the filtering efficiency of the second filter ordering is greater than the filtering efficiency of the first filter ordering; and

select the first filter ordering of the two or more filter processing elements in response to determining that the filtering efficiency of the second filter ordering is less than the filtering efficiency of the first filter ordering.

10. The computer program product of claim 9 , wherein the computer readable program further causes the computing device to:

set the second filter ordering of the two or more filter processing elements that are arranged as the second filter ordering;

determine a filter efficiency of the second filter ordering by:

monitor a runtime of the second filter ordering, and

monitor an amount tuples retained after filtering of the second filter ordering.

11. The computer program product of claim 9 , wherein the computer readable program causes the computing device to:

select, in response the to the second filter ordering increases the filtering efficiency by decreasing a runtime when compared to the first filter ordering, the second filter ordering.

12. The computer program product of claim 9 , wherein the computer readable program causes the computing device to:

select, in response to the second filter ordering increases the filtering efficiency by decreasing an amount of tuples retained after filtering, the second filter ordering.

13. The computer program product of claim 9 , wherein the computer readable program causes the computing device to:

store the first filter ordering of the two or more filter processing elements on the computer readable storage medium;

access the first filter ordering from the computer readable storage medium; and

create the first filter ordering.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2015
From: CAO, BIN; EIDEM, JESSICA R.; MURAS, BRIAN R.; SUN, JINGDONG
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 035724/0219 →
Continuity (1)
Related Publication 20160350381A1 · Dec 1, 2016