System and method for detecting drug adverse effects in social media and mobile applications data
Some implementations provide a computer-implemented method for identifying, from on-line postings, reports of potential adverse effects resulting from consuming a healthcare product, the method including: receiving a log of on-line postings regarding consuming the healthcare product; receiving a database comprising a healthcare taxonomy and a set of linguistic rules; analyzing, based on the healthcare taxonomy, the log of on-line postings to identify a report of at least one adverse effect resulting from consuming the healthcare product; generating a score for the identified report according to the healthcare taxonomy and the set of linguistic rules; comparing the generated score with a threshold; and in response to determining that the generated score is above the threshold, flagging the identified report as a report of a potential adverse effect.
1. A computer-implemented method, comprising:
performing, by a processor, a metasearch query on a plurality of on-line databases for electronic postings relating to a healthcare product, the plurality of on-line databases comprising a plurality of on-line communities;
aggregating, by an interface, the electronic postings from the plurality of on-line databases into a log of postings in response to the metasearch query;
receiving healthcare taxonomy data from one or more taxonomy databases;
identifying, by a text mining engine, textual content present within the electronic postings in the log of postings and analyzing the textual content present within the electronic postings based on the healthcare taxonomy data and according to load balancing tactics of or more computing systems, the load balancing tactics comprising factoring a proximity of each of the one or more computing systems to stored data representing the electronic postings and a likelihood of hits when searching a particular portion of data representing the electronic postings;
assigning, by a scoring engine, a score to each electronic posting in the log of postings based on the analyzed textual content of that electronic posting;
comparing, by the processor, the assigned score of each electronic posting in the log of postings to a scoring threshold to identify if the scoring threshold has been exceeded;
identifying, by the processor, an electronic posting in the log of postings as relating to an adverse effect resulting from consuming the healthcare product based on the scoring threshold being exceeded;
updating, by the processor and based on the score assigned to one or more electronic postings in the log of postings, the healthcare taxonomy data to include one or more new terms associated with the healthcare product; and
determining, by the processor and based on the score assigned to one or more electronic postings in the log of postings, that one or more additional terms in the healthcare taxonomy data are obsolete,
wherein the updating of the healthcare taxonomy data further includes removing the one or more additional terms from the healthcare taxonomy data.
2. The method of claim 1 , further comprising:
removing each of the electronic postings with assigned scores that do not exceed the scoring threshold.
3. The method of claim 1 , wherein the assigning of each score includes applying a set of linguistic rules to the electronic postings.
4. The method of claim 1 , further comprising:
generating a digest comprising the electronic posting relating to the adverse effect based on the scoring threshold being exceeded.
5. The method of claim 1 , further comprising:
submitting a report identifying safety issues based on the scoring threshold being exceeded.
6. The method of claim 1 , comprising identifying information on consumption of the healthcare product relating to the adverse effect.
7. The method of claim 1 , wherein the healthcare taxonomy data includes definitions for one or more medical conditions.
8. A non-transitory computer-readable medium encoded with processor-readable instructions that, when executed by one or more processors, cause the one or more processors to:
perform, by the one or more processors, a metasearch query on a plurality of on-line databases for electronic postings relating to a healthcare product, the plurality of on-line databases comprising a plurality of on-line communities;
aggregate, by an interface, the electronic postings from the plurality of on-line databases into a log of postings in response to the metasearch query;
receive healthcare taxonomy data from one or more taxonomy databases;
identify, by a text mining engine, textual content present within the electronic postings in the log of postings and analyze the textual content present within the electronic postings based on the healthcare taxonomy data and according to load balancing tactics of one or more computing systems, the load balancing tactics comprising factoring a proximity of each of the one or more computing systems to stored data representing the electronic postings and a likelihood of hits when searching a particular portion of data representing the electronic postings;
assign, by a scoring engine, a score to each electronic posting in the log of postings based on the analyzed textual content of that electronic posting;
compare, by the one or more processors, the assigned score of each electronic posting in the log of postings to a scoring threshold to identify if the scoring threshold has been exceeded;
identify, by the one or more processors, an electronic posting in the log of postings as relating to an adverse effect resulting from consuming the healthcare product based on the scoring threshold being exceeded;
updating, by the one or more processors and based on the score assigned to one or more electronic postings in the log of postings, the healthcare taxonomy data to include one or more new terms associated with the healthcare product; and
determining, by the one or more processors and based on the score assigned to one or more electronic postings in the log of postings, that one or more additional terms in the healthcare taxonomy data are obsolete,
wherein the updating of the healthcare taxonomy data further includes removing the one or more additional terms from the healthcare taxonomy data.
9. The non-transitory computer-readable medium of claim 8 , wherein the scoring threshold is used to identify the electronic postings that do not exceed the scoring threshold.
10. The non-transitory computer-readable medium of claim 8 , wherein the comparison to the scoring threshold identifies a level of importance for electronic postings in multiple dimensions.
11. The non-transitory computer-readable medium of claim 8 , wherein a digest is generated based on keywords identified in the electronic postings.
12. The non-transitory computer-readable medium of claim 8 , wherein one or more potential leads based on the identified adverse effect are analyzed.
13. The non-transitory computer-readable medium of claim 8 , wherein a report is generated linking the adverse effect to consuming the healthcare product.
14. A computer system connected to a network, the system comprising:
a memory configured to store instructions;
one or more processors configured to execute the instructions to:
perform a metasearch query on a plurality of on-line databases for electronic postings relating to a healthcare product, the plurality of on-line databases comprising a plurality of on-line communities;
aggregate, by an interface, the electronic postings from the plurality of on-line databases into a log of postings in response to the metasearch query;
receive healthcare taxonomy data from one or more taxonomy databases;
identify, by a text mining engine, textual content present within the electronic postings in the log of postings and analyze the textual content present within the electronic postings based on the healthcare taxonomy data and according to load balancing tactics of one or more computing systems, the load balancing tactics comprising factoring a proximity of each of the one or more computing systems to stored data representing the electronic postings and a likelihood of hits when searching a particular portion of data representing the electronic postings;
assign, by a scoring engine, a score to each electronic posting in the log of postings based on the analyzed textual content of that electronic posting;
compare the assigned score of each electronic posting in the log of postings to a scoring threshold to identify if the scoring threshold has been exceeded;
identify an electronic posting in the log of postings as relating to an adverse effect resulting from consuming the healthcare product based on the scoring threshold being exceeded;
update, based on the score assigned to one or more electronic postings in the log of postings, the healthcare taxonomy data to include one or more new terms associated with the healthcare product; and
determine, based on the score assigned to one or more electronic postings in the log of postings, that one or more additional terms in the healthcare taxonomy data are obsolete,
wherein the updating of the healthcare taxonomy data further includes removing the one or more additional terms from the healthcare taxonomy data.
15. The system of claim 14 , wherein the scoring threshold is used to identify which of the electronic postings contain relevant information.
16. The system of claim 14 , wherein the comparison to the scoring threshold identifies electronic postings that do not include the adverse effect.
17. The system of claim 14 , wherein the electronic posting relating to the adverse effect is identified to determine one or more healthcare products that caused the adverse effect.
18. The method of claim 1 , wherein the score of at least some electronic postings in the log of postings is based on a space proximity of keywords identified in the at least some electronic postings.