IP Library › Granted Patent US 12,388,732
Granted Patent B2
US 12,388,732 · App. 18/243,967 · Granted Aug 12, 2025

Historic netflow analysis system and method

Inventors: Mattias Harrysson (Tokyo, JP); Yasuyuki Hamada (Tokyo, JP)
Assignee: NTT Security Holdings Corporation
H04L43/062H04L43/067
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,388,732
App. No.
18/243,967
Granted
Aug 12, 2025
Kind
B2
Abstract

A system and method perform historic netflow searching for an IP address, such as an IPv4 address, in a cost, time and storage efficient manner using blob indexes and bitsets.

Claims (31)

1. A method for finding a historic netflow having a particular IP address, the method comprising:

retrieving, by a historic netflow computer system, a plurality of netflow blobs each containing a plurality of netflows wherein each netflow contains data about data traffic between an internet protocol (IP) address of a source host and the IP address of a destination host during a certain period of time in the past, each of the plurality of netflow blobs containing the plurality of netflows over a different period of time in the past;

receiving, by the historic netflow computer system, a request to find a requested IP address during a searchable period of time, wherein the requested IP address is findable in at least two of the network blobs that have netflows over different periods of time in the past;

generating, by the historic netflow computer system, a blob index file for each of the at least two netflow blobs, each blob index file containing each of a plurality of IP addresses in the netflow blob for a different period of time in the past;

generating, by the historic netflow computer system, a bitset for one of the at least two netflow blobs having a longest time period, wherein the bitset contains a set of bits in a word wherein each IP address of the plurality of IP addresses in the netflow blob having the longest period of time is converted into a bit in the bitset, wherein each bit in the bitset corresponds to the actual IP address;

performing, by the historic netflow computer system, a search, using the bitset and the at least two blob index files, to find a historic netflow having the requested IP address in a netflow during the searchable period of time; and

displaying, on a display of a user computing device, the found historic netflow having the requested IP address.

2. The method of claim 1 further comprising forming a search tree having a root that is the generated bitset and a plurality of leaves wherein each leaf is one of the generated blob index files and wherein performing the search further comprises checking, in each blob index file, if the requested IP address appears in the blob index file.

3. The method of claim 2 , wherein the performing the search further comprises identifying a particular netflow blob to be processed having the requested IP address using the blob index file corresponding to the particular netflow blob and one or more bitsets corresponding to the particular netflow blob and searching the identified particular netflow blob to find a historic netflow having the request IP address during a time interval of the particular netflow blob.

4. The method of claim 3 further comprising queuing the identified particular netflow blobs and finding the requested IP address in each identified particular netflow blob in the queue.

5. The method of claim 1 , wherein the period of time is one of an hour, a month and a day.

6. The method of claim 1 , wherein the generated bitset is a sixty-four bit word.

7. The method of claim 1 further comprising generating a plurality of bitsets wherein each generated bitset has a time period that varies between a shortest time interval, a short time interval and a longest time interval and wherein performing the search for the historic netflow further comprises performing the search using the generated bitset having the longest time interval, performing the search using the generated bitset having the short time interval if the requested IP address is contained in the longest time interval bitset and performing the search using the generated bitset having the shortest time interval if the requested IP address is contained in the short time interval bitset to find the historic netflow with the requested IP address.

8. The method of claim 1 , wherein the requested IP address is an IPv4 IP address.

9. A system for finding a historic netflow having a particular IP address, the system comprising:

a historic network finding computer system having a processor and a memory;

a computing device of a user that is connectable to the historic network finding computer system;

wherein the processor of the historic network finding computer system is configured to:

retrieve a plurality of historic netflow blobs each containing a plurality of netflows wherein each netflow contains data about data traffic between an internet protocol (IP) address of a source host and the IP address of a destination host during a certain period of time in the past, each of the plurality of netflow blobs containing the plurality of netflows over a different period of time in the past;

receive a request to find a requested IP address during a searchable period of time, wherein the requested IP address is findable in at least two of the network blobs that have different periods of time in the past;

generate a blob index file for each of the at least two netflow blobs, each blob index file containing each of a plurality of IP addresses in the netflow blob for a different period of time in the past;

generate a bitset for one of the at least two netflow blobs having a longest time period, wherein the bitset contains a set of bits in a word wherein each IP address of the plurality of IP addresses in the netflow blob having the longest period of time is converted into a bit in the bitset, wherein each bit in the bitset corresponds to the actual IP address; and

perform a search, using the bitset and the at least two blob index files, to find a historic netflow having the requested IP address in a netflow during the searchable period of time; and

wherein a display of the computing device displays the found historic netflow having the requested IP address.

10. The system of claim 9 , wherein the processor is further configured to form a search tree having a root that is the generated bitset and a plurality of leaves wherein each leaf is one of the generated blob index files and check, in each blob index file, if the requested IP address appears in the blob index file.

11. The system of claim 10 , wherein the processor is further configured to identify a particular netflow blob to be processed having the requested IP address using the blob index file corresponding to the particular netflow blob and one or more bitsets corresponding to the particular netflow blob and search the identified particular netflow blob to find a historic netflow having the request IP address during a time interval of the particular netflow blob.

12. The system of claim 11 , wherein the processor is further configured to queue the identified particular netflow blobs and find the requested IP address in each identified particular netflow blob in the queue.

13. The system of claim 9 , wherein the period of time is one of an hour, a month and a day.

14. The system of claim 9 , wherein the generated bitset is a sixty-four bit word.

15. The system of claim 9 , wherein the processor is further configured to generate a plurality of bitsets wherein each generated bitset has a time period that varies between a shortest time interval, a short time interval and a longest time interval, perform the search for the requested IP address in the netflow blob using the generated bitset having the longest time interval, perform the search for the requested IP address in the netflow blob using the generated bitset having the short time interval if the requested IP address is contained in the longest time interval bitset and perform the search for the requested IP address in the netflow blob using the generated bitset having the shortest time interval if the requested IP address is contained in the short time interval bitset to find the historic network with the requested IP address.

16. The system of claim 9 , wherein the requested IP address is an IPv4 IP address.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2023
From: HARRYSSON, MATTIAS; HAMADA, YASUYUKI
To: NTT SECURITY HOLDINGS CORPORATION
Reel/Frame 065061/0273 →
Continuity (1)
Related Publication 20250088440A1 · Mar 13, 2025
References Cited (69)
US 6575902B1 · Burton · 2003 [cited by applicant]
US 7225343B1 · Honig et al. · 2007 [cited by applicant]
US 7912698B2 · Statnikov et al. · 2011 [cited by applicant]
US 8135718B1 · Das et al. · 2012 [cited by applicant]
US 8402543B1 · Ranjan et al. · 2013 [cited by applicant]
US 8510830B2 · Reilly · 2013 [cited by applicant]
US 8762298B1 · Ranjan · 2014 [cited by applicant]
US 9144389B2 · Srinivasan et al. · 2015 [cited by applicant]
US 9183387B1 · Altman et al. · 2015 [cited by applicant]
US 9674298B1 · Edwards · 2017 [cited by examiner]
US 9674880B1 · Egner et al. · 2017 [cited by applicant]
US 9769189B2 · Mohalsen et al. · 2017 [cited by applicant]
US 9781160B1 · Irimie · 2017 [cited by examiner]
US 9787640B1 · Xie et al. · 2017 [cited by applicant]
US 10566084B2 · Kataoka · 2020 [cited by applicant]
US 10742669B2 · Takahashi · 2020 [cited by examiner]
US 20030137109A1 · Vancura · 2003 [cited by applicant]
US 20040128535A1 · Cheng · 2004 [cited by applicant]
US 20060187060A1 · Colby · 2006 [cited by applicant]
US 20060189377A1 · Gomez et al. · 2006 [cited by applicant]
US 20070256141A1 · Nakano et al. · 2007 [cited by applicant]
US 20070298874A1 · Baerlocher · 2007 [cited by applicant]
US 20070298875A1 · Baerlocher · 2007 [cited by applicant]
US 20080076515A1 · Baeriocher et al. · 2008 [cited by applicant]
US 20090066521A1 · Atlas et al. · 2009 [cited by applicant]
US 20090280891A1 · Filipour et al. · 2009 [cited by applicant]
US 20090319457A1 · Cheng et al. · 2009 [cited by applicant]
US 20100066509A1 · Okuizaimi et al. · 2010 [cited by applicant]
US 20100286572A1 · Moersdorf et al. · 2010 [cited by applicant]
US 20110118011A1 · Filipour · 2011 [cited by applicant]
US 20120005755A1 · Kitazawa et al. · 2012 [cited by applicant]
US 20130074186A1 · Muttik · 2013 [cited by applicant]
US 20130195326A1 · Bear · 2013 [cited by applicant]
US 20140153478A1 · Kazmi et al. · 2014 [cited by applicant]
US 20150088791A1 · Lin et al. · 2015 [cited by applicant]
US 20170063893A1 · Franc et al. · 2017 [cited by applicant]
US 20170092068A1 · Vann · 2017 [cited by applicant]
US 20170251005A1 · Niv · 2017 [cited by applicant]
US 20170318033A1 · Holland et al. · 2017 [cited by applicant]
US 20170337776A1 · Herring · 2017 [cited by applicant]
US 20180047253A1 · Vann · 2018 [cited by applicant]
US 20180069885A1 · Patterson · 2018 [cited by applicant]
US 20180082530A1 · Upton et al. · 2018 [cited by applicant]
US 20180082533A1 · Hallerbach et al. · 2018 [cited by applicant]
US 20180083988A1 · Kataoka et al. · 2018 [cited by applicant]
US 20180329958A1 · Choudhury · 2018 [cited by applicant]
US 20190132342A1 · Arlitt et al. · 2019 [cited by applicant]
US 20190305957A1 · Reddy et al. · 2019 [cited by applicant]
JP 2003242124A · 2003 [cited by applicant]
JP 2007318745A · 2007 [cited by applicant]
JP 2008049602A · 2018 [cited by applicant]
JP 2018148267A · 2018 [cited by applicant]
WO WO2008117544A · 2008 [cited by applicant]
WO WO2012075336A1 · 2012 [cited by applicant]
WO WO2018140335A1 · 2018 [cited by applicant]
WO WO2019032745A1 · 2019 [cited by applicant]
Website Traffic, Statistics and Analytics @ ALEXA—Webpage: https://www.alexa.com/siteinfo retrieved from internet Jan. 30, 2018, 5 pages. [cited by applicant]
Kegelman, J.C., et al., entitled “Insights into vehicle trajectories at the handling limits:analyzing open data from racecar drivers; Taylor & Francis, Vellicle System Dynamics” dated Nov. 3, 2016, 18 pages. [cited by applicant]
Theodosis et al., “Nonlinear Optimization of a Racing Line for an Autonomous Racecar Using Professional Driving Techniques”, dated Oct. 2012, 7 pages, Citation and abstract, Theodosis retrieved from the web at: https://… [cited by applicant]
Tulabandhula, T. et al. “Tire Changes, Fresh Air, and Yellow Flags: Challenges in Predictive Analytics for Professional Racing” MIT, dated Jun. 2014 (17 pages.). [cited by applicant]
Takagahara, K. et al.: “hitoe”—A Wearable Sensor Developed through Crossindustrial Collaboration, NTT Technical Review, dated Sep. 4, 2014 (5 pages.). [cited by applicant]
Lee, J.H., et al., Development of a novel Tympanic temperature monitoring system for GT car racing athletes—Abstract Only—pp. 2062-2065, dated 2013, (3 pages.), Lee retrieved from the web at https://link.springer.com/ch… [cited by applicant]
Kataoka et al., “Mining Muscle Use Data for Fatigue Reduction in IndyCar,” MIT Sloan Sports Analytics Conference (Mar. 4, 2017), pp. 1-12., Kataoka retrieved from the Internet. http://www.sloansportsconference.com/wp-co… [cited by applicant]
Malik et al. “Automatic training data cleaning for text classification.” In: 2011 IEEE 11th international conference on data mining workshops. Dated : Dec. 11, 2011, Malik retrieved on Feb. 22, 2022, entire document, ht… [cited by applicant]
Chawla et al., “SMOTE: synthetic minority over-sampling technique.” In: Journal of artificial intelligence research. Dated: Jun. 2, 2002, 37 pages, Chawla retrieved on Feb. 22, 2022 at https://patents.google.com/patent/… [cited by applicant]
Yen et al., “Cluster-based under—sampling approaches for imbalanced data distributions.” In: Expert Systems with Applications. Dated: Apr. 2009, 10 pages, Yen retrieved at: https://sci2s.ugr.es/keel/pdf/specific/articul… [cited by applicant]
Analytics Vidhya. “Imbalanced Data: How to handle Imbalanced Classification Problems in machine learning?” In: Dated: Mar. 17, 2017, 17 pages., retrieved at: https://www.analyticsvidhya.com/blog/2017/03/imbalanced-data-… [cited by applicant]
Fullmer, et al., “The OSU flow-tools Package and Cisco Netflow Logs”, 2000 Lisa XIV—Dec. 3-8, 2000—New Orleans, LA, pp. 291-304. [cited by applicant]
Fusco, et al., “Real-time creation of bitmap indexes on streaming network data” The VLDB Journal (2012) 21:287-307. [cited by applicant]