IP Library Granted Patent US 11,475,008
Granted Patent B2
US 11,475,008 · App. 16/860,154 · Granted Oct 18, 2022

Systems and methods for monitoring user-defined metrics

Inventors: Koon Heng Ivan Teo (San Francisco, CA); Qingyi Sui (Falls Church, VA); Mohammad Shami (Foster City, CA); Yoonseong Kim (Berkley, CA); Fernando San Martin Jorquera (Los Altos, CA); Francisco Perez Leon (Richmond, CA)
Assignee: Capital One Services, LLC
G06F16/2455G06F16/242G06F16/283
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,475,008
App. No.
16/860,154
Granted
Oct 18, 2022
Kind
B2
Abstract

Disclosed are systems and methods for monitoring user-defined metrics. A method may include: receiving, from a user device, a metric definition usable to generate queries to obtain data for a metric to be monitored; receiving, from the user device, a monitoring configuration indicative of a manner in which a metric monitoring process associated with the metric definition is to be repeatedly performed; storing the metric definition in a metric definition database; and repeatedly performing the metric monitoring process in accordance with the monitoring configuration. The metric monitoring process may include: retrieving the metric definition from the metric definition database; generating a database query based on the metric definition, the database query including one or more executable database statements defined by the metric definition; executing the database query to obtain query result data, the query result data being data for the metric; and storing the query result data.

Claims (81)

1. A computer-implemented method for monitoring metrics, the method comprising:

receiving, by a configuration component, from a user device, a metric definition usable to generate queries to obtain data for a metric to be monitored, the metric definition including code that indicates one or more variables to be resolved at a time of execution of the queries based on a context of the execution;

receiving, by the configuration component, from the user device, a monitoring configuration indicative of a manner in which a metric monitoring process associated with the metric definition is to be repeatedly performed;

storing, by the configuration component, the metric definition in a metric definition database; and

repeatedly performing the metric monitoring process in accordance with the monitoring configuration, the metric monitoring process including:

retrieving, by a scheduler, the metric definition from the metric definition database based on a user-defined frequency;

generating, by the scheduler, a database query based on the metric definition, the database query including one or more executable database statements defined by the metric definition, and the generating including resolving the one or more variables based on an execution context of the database query;

sending, by the scheduler, the database query to a data warehouse;

executing, by the data warehouse, the database query to obtain query result data, the query result data being data for the metric; and

storing, by the data warehouse, the query result data in the data warehouse,

wherein the scheduler is configured to initiate metric monitoring processes respectively associated with different respective metric definitions stored in the metric definition database in a sequential manner such that the scheduler avoids concurrent performance of two or more metric monitoring processes.

2. The method of claim 1 , wherein

the monitoring configuration specifies a frequency of performing the metric monitoring process, and

in the repeatedly performing the metric monitoring process, the metric monitoring process is performed periodically at the specified frequency.

3. The method of claim 2 , wherein

the scheduler is configured to initiate additional metric monitoring processes, respectively associated with additional metric definitions stored in the metric definition database, in accordance with monitoring configurations associated with the additional metric definitions.

4. The method of claim 1 , wherein, to resolve each of the one or more variables,

the generating the database query includes:

determining, by the scheduler, an execution value of the variable, and

determining, by the scheduler, the one or more executable database statements by setting the variable to the execution value, such that the one or more executable database statements includes the execution value of the variable.

5. The method of claim 1 , wherein, in the metric monitoring process, the storing the query result data includes storing the query result data as part of collected metric data associated with the metric.

6. The method of claim 5 , wherein in the metric monitoring process,

the collected metric data includes values of the metric for a first period of time,

the query result data includes values of the metric for a second period of time, and

the storing the query result data includes appending the query result data to the collected metric data.

7. The method of claim 5 , wherein the metric monitoring process is performed a plurality of times, such that the collected metric data is updated a plurality of times.

8. The method of claim 5 , further comprising:

presenting the collected metric data to the user device or another user device.

9. The method of claim 1 , wherein the method is performed by a computer system and further comprises:

transmitting, to the user device, a web page including a user interface enabling a user of the user device to submit the metric definition to the computer system.

10. The method of claim 1 , further comprising:

transmitting, to the user device, a web page including a user interface including a list of metric definitions stored in the metric definition database.

11. The method of claim 1 , wherein

the metric to be monitored is a measure or evaluation of events described by data recorded in the data warehouse, and

the query result data is based on the data recorded in the data warehouse.

12. A computer system for monitoring metrics, the computer system comprising:

a memory storing instructions; and

one or more processors configured to execute the instructions to perform operations including;

receiving, by a configuration component, from one or more user devices, a plurality of metric definitions, each of the plurality of metric definitions being usable to generate queries to obtain data for a respective metric to be monitored, the respective metric being a measure of an event associated with a particular period of time;

storing, by the configuration component, the plurality of metric definitions in a metric definition database; and

scheduling and performing a plurality of metric monitoring processes respectively associated with the plurality of metric definitions, such that each of the plurality of metric monitoring processes is periodically performed at a respective user-specified frequency, wherein each of the plurality of metric monitoring processes includes:

retrieving, by a scheduler, the respective metric definition from the metric definition database;

generating, by the scheduler, a database query based on the respective metric definition, the database query including one or more executable database statements defined by the respective metric definition;

sending, by the scheduler, the database query to a data warehouse;

executing, by the data warehouse, the database query to obtain query result data, the query result data being data for the respective metric; and

storing, by the data warehouse, the query result data as part of collected metric data associated with the respective metric in a table of the data warehouse that is accessible by a plurality of user devices including the one or more user devices, the query result data including values of the respective metric for a first period of time corresponding to the particular period of time,

wherein, as the respective metric monitoring process is repeatedly performed for each of the plurality of metric monitoring processes, additional query result data obtained is appended to the collected metric data in the table of the data warehouse, the additional query result data including values of the respective metric for a second period of time corresponding to the particular period of time, and

wherein the scheduler is configured to initiate the plurality of metric monitoring processes in a sequential manner such that the scheduler avoids concurrent performance of two or more metric monitoring processes.

13. The computer system of claim 12 , wherein

the plurality of metric definitions and the plurality of metric monitoring processes respectively include a first metric definition and a first metric monitoring process associated with the first metric definition, the first metric definition including a query template indicative of a variable whose value is to be determined during performance of the first metric monitoring process, and

for the first metric monitoring process, the generating the database query includes:

determining, by the scheduler, an execution value of the variable, and

determining, by the scheduler, one or more executable database statements by setting the variable to the execution value, such that the one or more executable database statements includes the execution value of the variable.

14. The computer system of claim 12 , wherein

a first metric definition of the plurality of metric definitions is received from a user device, and

the operations further include transmitting, to the user device, a web page including a user interface enabling a user of the user device to submit the first metric definition to the computer system.

15. The computer system of claim 14 , wherein the operations further include:

transmitting, to the user device, a web page including a user interface including a list of the plurality of metric definitions stored in the metric definition database, the list including one or more metric definitions submitted by another user.

16. A computer system for monitoring metrics, the computer system comprising:

a memory storing instructions; and

one or more processors configured to execute the instructions to perform operations including;

receiving, by a configuration component, from a user device, a metric definition usable to generate queries to obtain data for a metric to be monitored, the metric definition including code that indicates one or more variables to be resolved at a time of execution of the queries based on a context of the execution, and the metric being a measure or evaluation of events associated with a particular period of time that are described by data recorded in a data warehouse;

receiving, by the configuration component, from the user device, a monitoring configuration indicative of a manner in which a metric monitoring process associated with the metric definition is to be repeatedly performed;

storing, by the configuration component, the metric definition in a metric definition database; and

repeatedly performing the metric monitoring process in accordance with the monitoring configuration, the metric monitoring process including:

retrieving, by a scheduler, the metric definition from the metric definition database based on a user-defined frequency;

generating, by the scheduler, a database query based on the metric definition, the database query including one or more executable database statements defined by the metric definition, and the generating including resolving the one or more variables based on an execution context of the database query;

sending, by the scheduler, the database query to a data warehouse;

executing, by the data warehouse, the database query to obtain query result data, the query result data being data for the metric and being based on the data recorded in the data warehouse; and

storing, by the data warehouse, the query result data as part of collected metric data associated with the metric in a table of the data warehouse that is accessible by a plurality of user devices including the user device, the query result data including values of the metric for a first period of time corresponding to the particular period of time,

wherein, as the metric monitoring process is repeatedly performed, additional query result data obtained is appended to the collected metric data in the table of the data warehouse, the additional query result data including values of the metric for a second period of time corresponding to the particular period of time; and

executing a web service enabling the plurality of user devices to retrieve the collected metric data such that, in response to receiving a request from one of the plurality of user devices for the values of the metric for one or more of the first period of time or the second period of time, the requested values are retrieved from the table of the data warehouse and provided to the one of the plurality of user devices for display rather than causing the database query to be re-executed,

wherein the scheduler is configured to initiate metric monitoring processes respectively associated with different respective metric definitions stored in the metric definition database in a sequential manner such that the scheduler avoids concurrent performance of two or more metric monitoring processes.

17. The computer system of claim 12 , wherein

the event of which the metric is a measure of is described by data recorded in the data warehouse, and

the query result data is based on the data recorded in the data warehouse.

18. The computer system of claim 12 , wherein, in response to receiving a request from one of the plurality of user devices for the values of the respective metric for one or more of the first period of time or the second period of time, retrieving the requested values from the table of the data warehouse to provide to the one of the plurality of user devices for display rather than causing the database query to be re-executed.

19. The method of claim 4 , wherein the execution value of the variable is determined by the scheduler based on an execution context of the database query, the execution context including one or more of a date, a time, or an environment associated with the execution.

20. The computer system of claim 16 , wherein, to resolve each of the one or more variables, the generating the database query includes:

determining, by the scheduler, an execution value of the variable based on the execution context of the database query, the context including one or more of a date, a time, or an environment associated with the execution; and

determining, by the scheduler, the one or more executable database statements by setting the variable to the execution value, such that the one or more executable database statements includes the execution value of the variable.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2020
From: TEO, KOON HENG IVAN; SUI, QINGYI; SHAMI, MOHAMMAD; KIM, YOONSEONG; SAN MARTIN JORQUERA, FERNANDO; PEREZ LEON, FRANCISCO
To: CAPITAL ONE SERVICES, LLC
Reel/Frame 052520/0823 →
Continuity (1)
Related Publication 20210334274A1 · Oct 28, 2021