Systems and methods for information categorization
This invention relates to a computer-based method and system for facilitating the classification of information items. Information items are searched for references to entities of interest, and associated with such entities based on calculated confidence levels that the information items contains a reference to the entities and a category confidence level that the information items relate to the entities.
1. A computer-implemented method for classifying information items, the computer comprising a processor and a memory, the method comprising:
storing on the memory a list of entities in a database;
receiving an information item via a receiver;
using an entity extractor module operating on a computing device to (i) electronically search the information item for references to the entities within the information item; (ii) identify at least one possible reference to at least one of the entities within the information item; and (iii) calculate an accuracy confidence level that the possible reference to at least one of the entities contains the at least one of the entities; and
using a categorization engine operating on the computing device to: (i) associate the information item with the at least one of the entities based at least in part on the accuracy confidence level and a set of categorization rules; (ii) calculate a category confidence level that the information item relates to the at least one of the entities; and (iii) associate the information with the at least one entity based at least in part on the category confidence level.
2. The computer-implemented method of claim 1 wherein the list of entities includes one or more company names, ticker symbols, personal names, product names, industries, indices, abbreviations, or subjects.
3. The computer-implemented method of claim 1 wherein the information item comprises at least one of an electronic message, an email, an instant message, a financial report, a voicemail, a web page, a research report, an electronic document, an RSS feed, a subscription, an electronic filing, a wire service message, or a press release.
4. The computer-implemented method of claim 1 wherein searching the information item comprises the processor electronically scanning the information item for occurrences of text that matches at least one entity in the list of entities.
5. The computer-implemented method of claim 4 wherein scanning the information item for occurrences of text that matches at least one entity in the list of entities further comprises using phonetic variants of the at least one entity in the list of entities.
6. The computer-implemented method of claim 1 wherein the accuracy confidence level is based at least in part on a location within the information item that the possible reference to at least one of the entities is identified.
7. The computer-implemented method of claim 1 wherein the accuracy confidence level is based at least in part on a frequency that the possible references to at least one of the entities is identified within the information item.
8. The computer-implemented method of claim 1 wherein the accuracy confidence level is based at least in part on a source of the information item.
9. The computer-implemented method of claim 1 wherein the categorization rules consider a source of the information item in determining the category confidence level.
10. The computer-implemented method of claim 1 wherein the categorization rules consider time of receipt of the information item in determining the category confidence level.
11. The computer-implemented method of claim 1 wherein the categorization rules consider historical behaviors of users comprising one or more of websites visited, messages received, messages read or messages forwarded.
12. The computer-implemented method of claim 1 further comprising incorporating a cost function, wherein the cost function describes a user's tolerance for one or more falsely associated information items.
13. The computer-implemented method of claim 12 further comprising using a relevancy engine to calculate a recall probability for a set of information items, the recall probability comprising the probability that an information item within the set of information items is associated with an entity given that the information item is actually about the entity.
14. The computer-implemented method of claim 12 further comprising using a relevancy engine to calculate a precision probability for a set of information items, the precision probability comprising the probability that an information item within the set of information items is associated with an entity given the information item is actually classified according to the entity.
15. The computer-implemented method of claim 12 further comprising adjusting one or more of the categorization rules or the cost function such that the recall probability maintains a minimum value over a set of information items.
16. The computer-implemented method of claim 12 further comprising using a relevancy engine to calculate a relevancy for the information item based at least in part on the cost function.
17. The computer-implemented method of claim 12 wherein the cost function varies among a population of users.
18. The computer-implemented method of claim 12 wherein falsely associated information items comprise information items associated with an entity that do not relate to the entity.
19. The computer-implemented method of claim 12 wherein falsely associated information items comprise information items not associated with an entity that relate to the entity.
20. The computer-implemented method of claim 12 wherein the cost function is generated based on historical behaviors of the user.
21. The computer-implemented method of claim 12 wherein the cost function is generated based on a user-specific interest list.
22. The computer-implemented method of claim 1 further comprising using a categorization engine to associate the information item with a second entity, the second association based at least in part on a defined relationship between the at least one of the entities and the second entity.
23. The computer-implemented method of claim 22 wherein the defined relationship between the at least one of the entities and the second entity is one of a legal relationship, a parent-subsidiary relationship, a licensor-licensee relationship, a business relationship, a customer-supplier relationship or a competitive relationship.
24. A system for classifying information items, the system comprising:
a database for storing a plurality of entity identifiers;
a receiver for receiving an information item;
an entity extractor for (i) searching the information item for references to the entities within the information item, (ii) identifying at least one possible reference to at least one of the entities within the information item, and (iii) calculating an accuracy confidence level that the possible reference to at least one of the entities contains the at least one of the entities; and
a categorization engine for (i) associating the information item with the at least one of the entities based at least in part on the accuracy confidence level and a set of categorization rules, (ii) calculating a category confidence level that the information item relates to the at least one of the entities, and (iii) associating the information item with the at least one entity based at least in part on the category confidence level.
25. The system of claim 24 further comprising a database for storing the categorization rules.
26. The system of claim 24 further comprising a database for storing a plurality of user-specific cost functions, each cost function describing a user's tolerance for falsely associated information items.
27. The system of claim 26 further comprising a relevancy engine for calculating a relevancy for a specific user and information item based at least in part on the category confidence level and the cost function attributed to the user.
28. The system of claim 24 further comprising a categorization client agent that facilitates the manual association of the information item with the at least one entity.