System and method for automatic data verification through concurrent processing paths
A data analytic system for conducting automated analytics of content within a network-based system. The data analytic system features query management logic that, responsive to a triggering event, initiates queries for retrieval of particular type of content to be verified. The data analytic system further features multi-stage statistical analysis logic, automated intelligence and reporting logic. The statistical analysis logic is configured to concurrently conduct a plurality of statistical analyses on the content and generate corresponding plurality of statistical results, apply weightings to each of the statistical results, perform arithmetic operation(s) on the weighted statistical results to produce an analytic result, and determine whether the analytic result signifies that the content constitutes errored content. The reporting logic generates one or more issue alert messages including information for rendering a dashboard representing the analyses conducted by the multi-stage statistical analysis logic and the automated intelligence.
1. A system for conducting automated analytics of content within a network-based system, comprising:
a processor; and
a memory communicatively coupled to the processor and configured to store
query management logic that, responsive to a triggering event, initiates one or more queries, each query of the one or more queries is directed for retrieval of particular type of content to be verified,
multi-stage statistical analysis logic to concurrently (i) conduct a plurality of statistical analyses on the content included as part of a dataset provided to the multi-stage statistical analysis logic and (ii) generate a corresponding plurality of statistical results associated with determined accuracy of the content, the plurality of statistical analyses are directed to different types of natural language processing analytics;
automated intelligence configured to (i) receive the plurality of statistical results, (ii) apply weightings to each of the statistical results based, at least in part, on a type of query associated with the dataset, (iii) perform an arithmetic operation on the weighted statistical results to produce an analytic result, and (iv) determine whether the analytic result signifies that the content constitutes errored content; and
reporting logic communicatively coupled to the automated intelligence, the reporting logic being configured to receive at least the analytic result and metadata associated with the content included as part of the dataset and generate one or more issue alert messages including information for rendering a dashboard representing the analyses conducted by the multi-stage statistical analysis logic and the automated intelligence.
2. The system of claim 1 , wherein the metadata includes a textual description of the content.
3. The system of claim 1 , wherein the plurality of statistical analyses are conducted by execution, by the processor, of at least two of (1) a first statistical module conducting an Edit Distance analysis, (2) a second statistical module conducting a Normal Distribution analysis, (3) a third statistical module conducting an Unigram Probability analysis and (4) a fourth statistical module conducting a Conditional Probability analysis.
4. The system of claim 3 , wherein the plurality of statistical analyses conducted concurrently in verification of the particular type of content being an incentive-based content include the first statistical module conducting the Edit Distance analysis, the second statistical module conducting the Normal Distribution analysis, and the fourth statistical module conducting the Conditional Probability analysis.
5. The system of claim 3 , wherein the plurality of statistical analyses conducted concurrently in verification of the particular type of content with respect to accuracy of textual information of the content include the first statistical module conducting the Edit Distance analysis and the third statistical module conducting the Unigram Probability analysis.
6. The system of claim 1 , wherein the query management logic provides the dataset to the multi-stage statistical analysis logic, the dataset includes a collection of information received from an administrative system including the content and the metadata associated with the content including context information.
7. The system of claim 1 , wherein the automated intelligence being configured to receive training data, being data based on correct or incorrect determinations of verified content is in error, and adjust the weightings applied to each of the statistical results produced by the plurality of statistical analyses.
8. The system of claim 1 further comprising:
a data analytic system communicatively coupled to the reporting logic to aggregate content associated with one or more analytics results, including the analytic result, to generate the alert messages to be rendered as part of the dashboard to visually illustrate a presence of any potential errored content under verification within a data feed including the content.
9. The system of claim 1 , wherein the query management logic is configured to retrieve the one or more queries, where the one or more queries are selected based on the particular type of content to be verified and each query provides a framework for data associated with the content and the metadata to be retrieved from one or more domain servers deployed within a centralized storage system.
10. A system comprising:
an administrative system including a non-transitory storage medium including a plurality of queries, each query provides a framework for data to be retrieved from one or more servers; and
a data analytic system communicatively coupled to the administrative system, the data analytic system includes
query management logic configured to retrieve one or more queries of the plurality of queries from the administrative system, each query of the one or more queries is directed for retrieval of particular type of content to be verified,
multi-stage statistical analysis logic configured to (i) concurrently conduct a plurality of statistical analyses on the content included as part of a dataset provided to the multi-stage statistical analysis logic from the query management logic and (ii) generate a corresponding plurality of statistical results associated with determined accuracy of the content, the plurality of statistical analyses are directed to different types of natural language processing analytics,
automated intelligence configured to (i) receive the plurality of statistical results, (ii) apply weightings to each of the statistical results based, at least in part, on a type of query associated with the dataset, (iii) perform an arithmetic operation on the weighted statistical results to produce an analytic result, and (iv) determine whether the analytic result signifies that the content constitutes errored content, and
reporting logic communicatively coupled to the automated intelligence, the reporting logic being configured to receive at least the analytic result and metadata associated with the content included as part of the dataset and generate one or more issue alert messages including information for rendering a dashboard representing the analyses conducted by the multi-stage statistical analysis logic and the automated intelligence.
11. The system of claim 10 , wherein the data analytic system operates as a cloud service.