IP Library Granted Patent US 9,348,934
Granted Patent B2
US 9,348,934 · App. 14/553,796 · Granted May 24, 2016

Systems and methods for facilitating open source intelligence gathering

Inventors: Abha Moitra (Scotia, NY); David Brian Bracewell (Niskayuna, NY); Steven Matt Gustafson (Niskayuna, NY); T. Michael Baylor (Morgantown, PA); Tina H. Chau (Fairfax, VA)
Assignee: Lockheed Martin Corporation
G06F17/3089G06F3/0482G06F3/04817G06F17/30864G06N5/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,348,934
App. No.
14/553,796
Granted
May 24, 2016
Kind
B2
Abstract

Systems and methods (e.g., utilities) for use in providing automated, lightweight collection of online, open source data which may be content-based to reduce website source bias. In one aspect, a utility is disclosed for use in extracting content of interest from at least one website or other online data source (e.g., where the extracted content can be used in a subsequent search query). In other aspects, utilities are disclosed that are operable to perform various types of analyzes on such extracted content and present graphical representations of such analyzes on a display of a client device.

Claims (69)

1. A system for use in creating a hierarchical signature for a website, the system comprising:

a processor; and

a non-transitory computer-readable memory logically connected to the processor and comprising a set of computer readable instructions executable by the processor to:

receive parameters used to perform a query that identifies the at least one website from a plurality of websites; and

identify, from the received parameters, at least one textual hierarchy including at least first and second levels, wherein the first level comprises at least one textual category and the second level comprises at least one term that is encompassed by the at least one textual category;

determine a number of occurrences of the at least one term from a number of pages of at least one website during a time period;

first obtain a hierarchical signature of the at least one term that represents a prevalence of the at least one term on the at least one website;

second obtain, from the first obtaining step, a hierarchical signature of the at least one textual category that represents a prevalence of the at least one textual category on the at least one website;

establish a hierarchical signature of the at least one website utilizing the hierarchical signature of one or more of the at least one term and the at least one textual category; and

present, on a display, a graphical representation of the hierarchical signature of the at least one website, wherein the graphical representation illustrates the prevalence of one or more of the at least one term and the at least one textual category.

2. The system of claim 1 , wherein the computer readable instructions that first obtain the hierarchical signature of the at least one term include instructions executable by the processor to:

divide the number of occurrences of the at least one term by the number of pages to obtain the hierarchical signature for the at least one term.

3. The system of claim 2 , wherein the at least one term comprises first and second terms.

4. The system of claim 3 , wherein the computer readable instructions further include instructions executable by the processor to:

normalize the signatures of the first and second terms.

5. The system of claim 4 , wherein the computer readable instructions that normalize the signatures of the first and second terms include instructions executable by the processor to:

ascertain a mean and standard deviation of the first and second term hierarchical signatures;

subtract the mean from each of the first and second term hierarchical signatures; and

divide each of the values from the subtracting by the standard deviation to obtain z-scores for each of the first and second term hierarchical signatures, wherein the graphical representations of the first and second term hierarchical signatures respectively correspond to the z-scores of the first and second term hierarchical signatures.

6. The system of claim 1 , wherein a first color of the graphical representation represents a first prevalence of the at least one term and/or at least one textual category on the at least one website and a second color of the graphical representation represents a second prevalence of the at least one term and/or at least one textual category on the at least one website, and wherein the first prevalence is higher than the second prevalence.

7. The system of claim 1 , wherein each of the pages of the at least one website comprises an object that includes content of interest that has been extracted from the at least one website by way of computer readable instructions executable by the processor to:

obtain source code used to generate the page on a display, wherein the source code includes a plurality of elements and each element includes at least one tag comprising at least one tag type;

parse the source code to obtain a node tree including a plurality of nodes arranged in a hierarchical structure, wherein each node comprises one of the elements, and wherein one of the plurality of nodes comprises a root node;

determine a tag type of a node under the root node;

assign a heuristic score to the node based at least in part on the tag type of the node;

continue to determine and assign for one or more additional nodes of the node tree; and

generate the object, wherein the content of interest is associated with nodes of the node tree having heuristic scores indicating that such content is of interest.

8. The system of claim 7 , wherein the computer readable instructions that first obtain the hierarchical signature of the at least one term include instructions executable by the processor to:

divide the number of occurrences of the at least one term by the number of pages to obtain the hierarchical signature for the at least one term.

9. The system of claim 8 , wherein the at least one term comprises first and second terms.

10. The system of claim 9 , wherein the computer readable instructions that second obtain the hierarchical signature of the at least one textual category include instructions executable by the processor to:

average the hierarchical signatures of the first and second terms to obtain the hierarchical signature for the at least one textual category.

11. The system of claim 9 , wherein the computer readable instructions further include instructions executable by the processor to:

normalize the signatures of the first and second terms.

12. The system of claim 11 , wherein the computer readable instructions that normalize the signatures of the first and second terms include instructions executable by the processor to:

ascertain a mean and standard deviation of the first and second term hierarchical signatures;

subtract the mean from each of the first and second term hierarchical signatures; and

divide each of the values from the subtracting by the standard deviation to obtain z-scores for each of the first and second term hierarchical signatures, wherein the graphical representations of the first and second term hierarchical signatures respectively correspond to the z-scores of the first and second term hierarchical signatures.

13. The system of claim 1 , wherein at least one of the received parameters is a search term, and wherein the at least one term of the second level is the search term.

14. A system for use in creating a hierarchical signature for a website, the system comprising:

a processor; and

a non-transitory computer-readable memory logically connected to the processor and comprising a set of computer readable instructions executable by the processor to:

identify at least one textual hierarchy including at least first and second levels, wherein the first level comprises at least one textual category and the second level comprises at least one term that is encompassed by the at least one textual category;

determine a number of occurrences of the at least one term from a number of pages of at least one website during a time period;

first obtain a hierarchical signature of the at least one term that represents a prevalence of the at least one term on the at least one website by dividing the number of occurrences of the at least one term by the number of pages to obtain the hierarchical signature for the at least one term;

second obtain, from the first obtaining step, a hierarchical signature of the at least one textual category that represents a prevalence of the at least one textual category on the at least one website;

establish a hierarchical signature of the at least one website utilizing the hierarchical signature of one or more of the at least one term and the at least one textual category; and

present, on a display, a graphical representation of the hierarchical signature of the at least one website, wherein the graphical representation illustrates the prevalence of one or more of the at least one term and the at least one textual category, wherein the at least one term comprises first and second terms, and wherein the computer readable instructions that second obtain the hierarchical signature of the at least one textual category include instructions executable by the processor to:

average the hierarchical signatures of the first and second terms to obtain the hierarchical signature for the at least one textual category.

15. The system of claim 14 , wherein the computer readable instructions further include instructions executable by the processor to:

receive parameters used to perform a query that identifies the at least one website from a plurality of websites, wherein the at least one textual hierarchy is identified from the received parameters.

16. A system for use in creating a hierarchical signature for a website, the system comprising:

a processor; and

a non-transitory computer-readable memory logically connected to the processor and comprising a set of computer readable instructions executable by the processor to:

identify at least one textual hierarchy including at least first and second levels, wherein the first level comprises at least one textual category and the second level comprises at least one term that is encompassed by the at least one textual category;

determine a number of occurrences of the at least one term from a number of pages of at least one website during a time period;

first obtain a hierarchical signature of the at least one term that represents a prevalence of the at least one term on the at least one website;

second obtain, from the first obtaining step, a hierarchical signature of the at least one textual category that represents a prevalence of the at least one textual category on the at least one website;

establish a hierarchical signature of the at least one website utilizing the hierarchical signature of one or more of the at least one term and the at least one textual category; and

present, on a display, a graphical representation of the hierarchical signature of the at least one website, wherein the graphical representation illustrates the prevalence of one or more of the at least one term and the at least one textual category, wherein the at least one textual category comprises a plurality of textual categories, wherein the website hierarchical signature comprises the hierarchical signatures of each of the plurality of textual categories, wherein the computer readable instructions that establish the hierarchical signature of the website include instructions executable by the processor to establish hierarchical signatures for a plurality of websites, and wherein the computer readable instructions further include instructions executable by the processor to:

discover at least one website community from the plurality of web sites; and

utilize the hierarchical signatures of the websites making up the website community to obtain a hierarchical signature for the website community.

17. The system of claim 16 , wherein the computer readable instructions that utilize the hierarchical signatures of the websites include instructions executable by the processor to:

average the hierarchical signatures of the websites making up the website community to obtain the hierarchical signature for the website community.

18. The system of claim 16 , wherein the computer readable instructions further include instructions executable by the processor to:

receive parameters used to perform a query that identifies the at least one website from a plurality of websites, wherein the at least one textual hierarchy is identified from the received parameters.

19. The system of claim 16 , wherein the computer readable instructions that first obtain the hierarchical signature of the at least one term include instructions executable by the processor to:

divide the number of occurrences of the at least one term by the number of pages to obtain the hierarchical signature for the at least one term.

20. The system of claim 19 , wherein the at least one term comprises first and second terms.

Assignments (4)
NUNC PRO TUNC ASSIGNMENT Recorded Sep 17, 2019
From: LOCKHEED MARTIN CORPORATION
To: LEIDOS INNOVATIONS TECHNOLOGY, INC.; LOCKHEED MARTIN CORPORATION
Reel/Frame 050403/0506 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2014
From: BAYLOR, T. MICHAEL; CHAU, TINA H.
To: LOCKHEED MARTIN CORPORATION
Reel/Frame 034270/0947 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2014
From: MOITRA, ABHA; BRACEWELL, DAVID BRIAN; GUSTAFSON, STEVEN MATT
To: GENERAL ELECTRIC COMPANY
Reel/Frame 034270/0958 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2014
From: GENERAL ELECTRIC COMPANY
To: LOCKHEED MARTIN CORPORATION
Reel/Frame 034270/0967 →
Continuity (4)
Division 14091024 · Nov 26, 2013
Division 13045128 · Mar 10, 2011
Provisional Application 61312341 · Mar 10, 2010
Related Publication 20150121265A1 · Apr 30, 2015