Interest-Driven Business Intelligence Systems and Methods of Data Analysis Using Interest-Driven Data Pipelines

1 . An interest-driven business intelligence system, comprising:

raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes, where the raw data comprises unstructured data;

metadata storage configured to contain metadata that describes the raw data;

aggregate data storage configured to contain aggregate data, where aggregate data comprises structured data generated using ETL processes from the raw data;

an intermediate processing layer;

a business intelligence reporting engine;

wherein the intermediate processing layer is configured to automatically:

generate metadata describing the raw data;

derive reporting data requirements from the at least one report specification based on the metadata; and

compile an interest-driven data pipeline based upon the reporting data requirements, where compiling the interest-driven data pipeline comprises:

generating ETL processing jobs to generate aggregate data from the raw data by:

filtering the raw data using the structure of the raw data; and

applying transformations to the raw data based on the structure of the raw data;

storing the aggregate data in the aggregate data storage;

generating reporting data including reporting data satisfying the reporting data requirements using the aggregate data; and

storing the reporting data in a data mart within the intermediate processing layer for exploration by the business intelligence reporting engine;

wherein the business intelligence reporting engine is configured to:

receive metadata describing the raw data from the intermediate processing layer; and

generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data using the metadata.

2 . The interest-driven business intelligence system of claim 1 , wherein the raw data storage is a data warehouse.

3 . The interest-driven business intelligence system of claim 2 , wherein the data warehouse is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database.

4 . The interest-driven business intelligence system of claim 2 , wherein the data warehouse is a distributed computing system implemented utilizing Hadoop.

5 . The interest-driven business intelligence system of claim 2 , wherein the data warehouse is configured to store data generated utilizing the intermediate processing layer.

6 . The interest-driven business intelligence system of claim 2 , wherein the intermediate processing layer is configured to generate data warehouse requests.

7 . The interest-driven business intelligence system of claim 6 , wherein the data warehouse requests are requests selected from the group consisting of Hive queries and MapReduce operations.

8 . The interest-driven business intelligence system of claim 1 , wherein the intermediate processing layer is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database system.

9 . The interest-driven business intelligence system of claim 1 , wherein the business intelligence reporting engine is configured to display an indication based upon the interactive exploration of the at least one report.

10 . The interest-driven business intelligence system of claim 9 , wherein the indication is an estimate of the time needed to update the reporting data requirements.

11 . A method for creating a report utilizing an interest-driven business intelligence system, comprising:

storing raw data in raw data storage using an interest-driven business intelligence system, where the raw data comprises unstructured data;

generating metadata describing the raw data using the interest-driven business intelligence system;

receiving report definitions generated utilizing the metadata describing the raw data using the interest-driven business intelligence system;

automatically generating reporting data requirements derived from report definitions using the interest-driven business intelligence system;

automatically compiling an interest-driven data pipeline that is part of the interest-driven business intelligence system using the reporting data requirements and the raw data;

generating reporting data from the reporting data requirements using the interest-driven data pipeline, where the reporting data comprises structured data generated using the raw data; and

storing the generated reporting data in a data mart within the interest-driven business intelligence system.