Systems and methods for facilitating the gathering of open source intelligence
Systems and methods (e.g., utilities) for use in providing automated, lightweight collection of online, open source data which may be content-based to reduce website source bias. In one aspect, a utility is disclosed for use in extracting content of interest from at least one website or other online data source (e.g., where the extracted content can be used in a subsequent search query). In other aspects, utilities are disclosed that are operable to perform various types of analyses on such extracted content and present graphical representations of such analyses on a display of a client device.
1. A system for use in determining an influence of sentiment of an author of a post in an online information flow network on the posts of one or more other authors in the network, the system comprising:
a processor; and
a memory logically connected to the processor and comprising a set of computer readable instructions executable by the processor to:
receive information related to a plurality of online posts, wherein each of the online posts is associated with at least one author;
obtain, from the information, a listing of online posts that refer to one or more other online posts, wherein any online post that refers to another online post comprises a “secondary post” and wherein any online post that is referred to by another online post comprises a “primary post”;
build an information flow network that includes a plurality of information flow links that connect and represent information flows from primary posts to secondary posts;
first determine a sentiment of each of a primary post and one or more secondary posts of the primary post;
second determine a sentiment influence of the at least one author of the primary post in the information flow network using the sentiment of the primary post and the sentiments of the one or more secondary posts by:
determining a first average sentiment of the one or more secondary posts;
determining a second average sentiment of other posts in the information flow network initiated by the at least one author of the primary post; and
determining a quotient of the first average sentiment and the second average sentiment to obtain the sentiment influence of the at least one author of the primary post in the information flow network; and
present, on a display, a graphical representation of the sentiment influence of the at least one author of the primary post in the information flow network.
2. The system of claim 1 , wherein the primary post, one or more secondary posts, and other posts in the information flow network associated with the at least one author of the primary post relate to a common topic or discussion.
3. The system of claim 1 , wherein the sentiments of the posts are normalized.
4. The system of claim 3 , wherein the sentiment of each post is normalized by way of executing the computer readable instructions by the processor to:
identify a quantity of at least one of positively or negatively charged terms in the post;
identify a total number of terms in the post; and
divide the quantity of the at least one of positively or negatively charged terms in the post by the total number of terms in the post to obtain a normalized sentiment of the post.
5. The system of claim 1 , wherein the computer readable instructions are further executable by the processor to:
use the sentiment influence of the at least one author to determine at least one online role of the at least one author in the information flow network.
6. The system of claim 5 , wherein the at least one online role comprises a provocateur role.
7. A method for use in determining an influence of sentiment of an author of a post in an online information flow network on the posts of one or more other authors in the network, the method comprising:
receiving information related to a plurality of online posts, wherein each of the online posts is associated with at least one author;
obtaining, from the information, a listing of online posts that refer to one or more other online posts, wherein any online post that refers to another online post comprises a “secondary post” and wherein any online post that is referred to by another online post comprises a “primary post”;
building an information flow network that includes a plurality of information flow links that connect and represent information flows from primary posts to secondary posts;
first determining, using a processor, a sentiment of each of a primary post and one or more secondary posts of the primary post;
second determining, using the processor, a sentiment influence of the at least one author of the primary post in the information flow network using the sentiment of the primary post and the sentiments of the one or more secondary posts by:
determining a first average sentiment of the one or more secondary posts;
determining a second average sentiment of other posts in the information flow network initiated by the at least one author of the primary post; and
determining a quotient of the first average sentiment and the second average sentiment to obtain the sentiment influence of the at least one author of the primary post in the information flow network; and
presenting, on a display, a graphical representation of the sentiment influence of the at least one author of the primary post in the information flow network.
8. The method of claim 7 , wherein the primary post, one or more secondary posts, and other posts in the information flow network associated with the at least one author of the primary post relate to a common topic or discussion.
9. The method of claim 7 , wherein the sentiments of the posts are normalized.
10. The method of claim 9 , wherein the sentiment of each post is normalized by way of:
identifying a quantity of at least one of positively or negatively charged terms in the post;
identifying a total number of terms in the post; and
dividing the quantity of the at least one of positively or negatively charged terms in the post by the total number of terms in the post to obtain a normalized sentiment of the post.
11. The method of claim 7 , further comprising:
using the sentiment influence of the at least one author to determine at least one online role of the at least one author in the information flow network.
12. The method of claim 11 , wherein the at least one online role comprises a provocateur role.