System and method to analyse and predict impact of textual data
System and method to analyze and predict impact of textual data are provided. The system also includes a processing subsystem configured to select textual data from a plurality of data sets stored in a memory, to extract data from external sources using crawling, to identify at least one context of the textual data using one or more identification methods. The processing subsystem includes an NLP module configured to match the textual data with NLP frameworks using a mapping method based on a plurality of parameters, to apply feature engineering and transformation on the textual data to extract a plurality of features from the plurality of data sets and to analyze matched textual data of the textual using at least one analysis method. The processing subsystem also includes a predictive module configured to predict one or more future values of the analyzed textual data using the one or more predictive methods.
1. A system to analyze and predict impact of textual data comprising:
a memory configured to store a plurality of data sets acquired from one or more sources;
a processing subsystem operatively coupled to the memory, and configured to:
select textual data from the plurality of data sets;
extract data from one or more external sources through web crawling;
identify at least one context of the textual data using one or more context identification methods, wherein at least one or more context identification methods comprises at least one or more machine learning models;
wherein the processing subsystem comprises:
a natural language processing (NLP) module configured to:
match the textual data with at least one natural language processing (NLP) framework from a plurality of frameworks obtained from the one or more sources using a mapping method based on a plurality of parameters;
apply feature engineering and transformation on the textual data to extract a plurality of features from the plurality of data sets wherein the feature engineering and the transformation is applied on the textual data based on a use case, a data quality, a data type and a data volume for extracting the plurality of features from the plurality of data sets;
analyze matched textual data using at least one analysis method, wherein the at least one analysis method comprises at least one of a part of speech (POS) tagging, a sentiment method, a topic modelling, a clustering method and a document classification method;
store an analyzed result of the textual data in the memory;
a predictive module operatively coupled to the natural language processing (NLP) module, and configured to:
obtain the analyzed result of the textual data from the memory;
predict one or more future values of the analyzed textual data using one or more machine learning models based on the analyzed result;
a data visualization engine configured to:
generate one or more of a chart, a graph and a table based on the predicted one or more future values; and
generate a model summary based on the predicted one or more future values.
2. The system as claimed in claim 1 , wherein the plurality of data sets comprises at least one of a plurality of structured data sets, a plurality of unstructured data sets and a plurality of semi-structured data sets.
3. The system as claimed in claim 1 , wherein the plurality of parameters comprises at least one of a use case, a statistical influence and a previous predictive sample.
4. The system as claimed in claim 1 , further comprises a representation module operatively coupled to the processing subsystem, and configured to represent one or more predicted future values in one or more forms.
5. A method for analyzing and predicting impact of textual data comprising:
acquiring a plurality of data sets from one or more sources;
selecting textual data from the plurality of data sets;
identifying at least one context of the textual data using one or more context identification methods;
matching the textual data with at least one natural language processing (NLP) framework from a plurality of frameworks obtained from the one or more sources using a mapping method based on a plurality of parameters;
applying feature engineering and transformation on the textual data to extract a plurality of features from the plurality of data sets wherein the feature engineering and the transformation is applied on the textual data based on a use case, a data quality, a data type and a data volume for extracting the plurality of features from the plurality of data sets;
analyzing matched textual data using at least one analysis method;
predicting one or more future values of the analyzed textual data using one or more machine learning models based on an analysis result;
generating one or more of a chart, a graph and a table based on the predicted one or more future values; and
generating a model summary based on the predicted one or more future values.
6. The method as claimed in claim 5 , wherein acquiring the plurality of data sets from one or more sources comprises acquiring the plurality of data from at least one of a web, a manual entry of data, a local data set, an internal storage, an external storage and an experimental data set.
7. The method as claimed in claim 5 , wherein analyzing the matched textual data using the at least one analysis method comprises analyzing the matched textual data using at least one of a part of speech (POS) tagging, a sentiment method, a topic modelling, a clustering method and a document classification method.
8. The method as claimed in claim 5 , further comprises representing one or more predicted future values of the textual data in one or more forms.