IP Library › Granted Patent US 12,585,771
Granted Patent B2
US 12,585,771 · App. 18/084,045 · Granted Mar 24, 2026

Learned control flow monitoring and enforcement of unobserved transitions

Inventors: Andrew Zawadowskiy (Hollis, NH); Vincent E. Parla (North Hampton, NH); Oleg Bessonov (Littleton, MA)
Assignee: Cisco Technology, Inc.
G06F21/566G06F8/433G06F8/75G06F9/44589G06F11/3616G06F21/51G06F21/53G06F21/54G06F21/552G06F21/577H04L63/1416H04L63/1425H04L63/1433G06F2221/033
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,585,771
App. No.
18/084,045
Granted
Mar 24, 2026
Kind
B2
Abstract

Techniques and systems described herein relate to monitoring executions of computer instructions on computing devices based on learning and generating a control flow directed graph. The techniques and systems include determining a learned control flow diagram for a process on a computing system and monitoring execution of the process on the computing system using the control flow diagram. An unobserved transition is determined based on the learned control flow diagram and the unobserved transition is classified as safe or unsafe based on a monitoring component analysis. An action is performed based on the safety classification and the learned control flow diagram.

Claims (75)

1 . A method for monitoring a computing system, comprising:

determining a learned control flow diagram for a process executed on a computing system by observing executions of transitions during an observation period during which safe executions of transitions are permitted to execute and be observed;

monitoring execution of the process on the computing system using the learned control flow diagram;

determining an unobserved transition of the process based at least in part on the unobserved transition not being represented in the learned control flow diagram;

determining a classification of the unobserved transition as being safe by analyzing, using a monitoring component, the unobserved transition; and

performing an action based at least in part on the classification and the learned control flow diagram, the action including adding the unobserved transition to the learned control flow diagram.

2 . The method of claim 1 , wherein analyzing the unobserved transition comprises statically analyzing the transition.

3 . The method of claim 1 , wherein determining the classification for the unobserved transition comprises:

determining a context for the transition;

performing a static analysis of the unobserved transition; and

determining the unobserved transition is safe in response to determining a reason for the transition based at least in part on the context and the static analysis.

4 . The method of claim 1 , wherein determining the classification for the unobserved transition comprises:

determining a type of transition for the unobserved transition;

determining a destination for the unobserved transition;

determining a characteristic of the destination; and

determining the classification using a machine learned model using inputs of the type of transition, the destination, the characteristic of the destination, and the learned control flow diagram.

5 . The method of claim 1 , wherein determining the classification for the unobserved transition comprises:

determining a destination linked by the unobserved transition; and

determining a risk score associated with the destination, and wherein the classification is based at least in part on the risk score of the destination.

6 . The method of claim 5 , wherein the risk score is further based on at least one of:

a presence of a system call at the destination;

permissions associated with the destination;

a presence of propagating transitions to additional destinations; or

a presence of the destination within the learned control flow diagram.

7 . A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

determining a learned control flow diagram for a process executed on a computing system by observing executions of transitions during an observation period during which safe executions of transitions are permitted to execute and be observed;

monitoring execution of the process on the computing system using learned control flow diagram;

determining an unobserved transition of the process based at least in part on the unobserved transition not being represented in the learned control flow diagram;

determining a classification of the unobserved transition as being safe by analyzing, using a monitoring component, the unobserved transition; and

performing an action based at least in part on the classification and the learned control flow diagram, the action including adding the unobserved transition to the learned control flow diagram.

8 . The system of claim 7 , wherein determining the classification for the unobserved transition comprises:

determining a context for the transition;

performing a static analysis of the unobserved transition; and

determining the unobserved transition is safe in response to determining a reason for the transition based at least in part on the context and the static analysis.

9 . The system of claim 7 , wherein determining the classification for the unobserved transition comprises:

determining a type of transition for the unobserved transition;

determining a destination for the unobserved transition;

determining a characteristic of the destination; and

determining the classification using a set of heuristics defining the classification in response to the type of transition, the destination, the characteristic of the destination, and the learned control flow diagram.

10 . The system of claim 7 , wherein determining the classification for the unobserved transition comprises:

determining a destination linked by the unobserved transition; and

determining a risk score associated with the destination, and wherein the classification is based at least in part on the risk score of the destination.

11 . The system of claim 7 , wherein determining the classification for the unobserved transition comprises:

determining a type of transition for the unobserved transition;

determining a destination for the unobserved transition;

determining a characteristic of the destination; and

determining the classification using a machine learned model using inputs of the type of transition, the destination, the characteristic of the destination, and the learned control flow diagram.

12 . The system of claim 7 , wherein analyzing the unobserved transition comprises statically analyzing the transition.

13 . One or more non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to:

determine a learned control flow diagram for a process executed on a computing system by observing executions of transitions during an observation period during which safe executions of transitions are permitted to execute and be observed;

monitor execution of the process on the computing system using the learned control flow diagram;

determine an unobserved transition of the process based at least in part on the unobserved transition not being represented in the learned control flow diagram;

determine a classification of the unobserved transition as being safe by analyzing, using a monitoring component, the unobserved transition; and

performing an action based at least in part on the classification and the learned control flow diagram, the action adding the unobserved transition to the learned control flow diagram.

14 . The one or more non-transitory computer-readable media of claim 13 , wherein:

the action comprises allowing execution of the unobserved transition in response to the classification being safe.

15 . The one or more non-transitory computer-readable media of claim 13 , wherein the instructions to determine the classification for the unobserved transition comprises further instructions to:

determine a context for the transition;

perform a static analysis of the unobserved transition; and

determine the unobserved transition is safe in response to determining a reason for the transition based at least in part on the context and the static analysis.

16 . The one or more non-transitory computer-readable media of claim 13 , wherein the instructions to determine the classification for the unobserved transition comprises further instructions to:

determine a type of transition for the unobserved transition;

determine a destination for the unobserved transition;

determine a characteristic of the destination; and

determine the classification using a set of heuristics defining the classification in response to the type of transition, the destination, the characteristic of the destination, and the learned control flow diagram.

17 . The one or more non-transitory computer-readable media of claim 13 , wherein the instructions to determine the classification for the unobserved transition comprises further instructions to:

determine a type of transition for the unobserved transition;

determine a destination for the unobserved transition;

determine a characteristic of the destination; and

determine the classification using a machine learned model using inputs of the type of transition, the destination, the characteristic of the destination, and the learned control flow diagram.

18 . The one or more non-transitory computer-readable media of claim 13 , wherein the instructions to determine the classification for the unobserved transition comprises further instructions to:

determine a destination linked by the unobserved transition; and

determine a risk score associated with the destination, and wherein the classification is based at least in part on the risk score of the destination.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2022
From: ZAWADOWSKIY, ANDREW; PARLA, VINCENT E; BESSONOV, OLEG
To: CISCO TECHNOLOGY, INC.
Reel/Frame 062146/0234 →
Continuity (3)
Provisional Application 63391518 · Jul 22, 2022
Provisional Application 63391560 · Jul 22, 2022
Related Publication 20240028742A1 · Jan 25, 2024
References Cited (157)
US 5933640A · Dion · 1999 [cited by applicant]
US 7984304B1 · Waldspurger · 2011 [cited by applicant]
US 8230505B1 · Ahrens et al. · 2012 [cited by applicant]
US 8407322B1 · Sasikumar et al. · 2013 [cited by applicant]
US 8682985B2 · Pulla et al. · 2014 [cited by applicant]
US 9021589B2 · Anderson · 2015 [cited by examiner]
US 9158915B1 · Yumer · 2015 [cited by applicant]
US 9454659B1 · Daymont · 2016 [cited by examiner]
US 9696989B1 · Korotaev · 2017 [cited by applicant]
US 9953158B1 · Benameur et al. · 2018 [cited by applicant]
US 10084815B2 · Falkowitz et al. · 2018 [cited by applicant]
US 10203968B1 · Lawson · 2019 [cited by applicant]
US 10310863B1 · Poimboeuf · 2019 [cited by applicant]
US 10691575B2 · Lengauer et al. · 2020 [cited by applicant]
US 10706144B1 · Moritz · 2020 [cited by applicant]
US 10915636B1 · Kaplan · 2021 [cited by applicant]
US 11003746B1 · Vashishtha et al. · 2021 [cited by applicant]
US 11018959B1 · Neill · 2021 [cited by applicant]
US 11138314B1 · Gettys et al. · 2021 [cited by applicant]
US 11349670B1 · Miller · 2022 [cited by applicant]
US 11556444B1 · Rohithakshappa et al. · 2023 [cited by applicant]
US 11556452B2 · Danielson et al. · 2023 [cited by applicant]
US 11809535B2 · Magi et al. · 2023 [cited by applicant]
US 11847044B2 · Krishnan et al. · 2023 [cited by applicant]
US 11947663B2 · Sethumadhavan et al. · 2024 [cited by applicant]
US 11971807B2 · Kroehling · 2024 [cited by applicant]
US 12326936B2 · Zawadowskiy · 2025 [cited by examiner]
US 20030078782A1 · Blair · 2003 [cited by applicant]
US 20050010804A1 · Bruening · 2005 [cited by examiner]
US 20060161978A1 · Abadi et al. · 2006 [cited by applicant]
US 20060174077A1 · Abadi et al. · 2006 [cited by applicant]
US 20070160191A1 · Blair · 2007 [cited by applicant]
US 20090055571A1 · Budko et al. · 2009 [cited by applicant]
US 20090328211A1 · Abraham · 2009 [cited by examiner]
US 20100180258A1 · Takahashi · 2010 [cited by applicant]
US 20100275186A1 · McGarvey et al. · 2010 [cited by applicant]
US 20120066166A1 · Curbera et al. · 2012 [cited by applicant]
US 20120222019A1 · Gounares · 2012 [cited by examiner]
US 20130031531A1 · Keynes · 2013 [cited by applicant]
US 20130055210A1 · Murthy et al. · 2013 [cited by applicant]
US 20130151831A1 · Bealkowski et al. · 2013 [cited by applicant]
US 20130291113A1 · Dewey · 2013 [cited by applicant]
US 20130326625A1 · Anderson et al. · 2013 [cited by applicant]
US 20140337086A1 · Asenjo · 2014 [cited by applicant]
US 20150161383A1 · Chen et al. · 2015 [cited by applicant]
US 20150356294A1 · Tan · 2015 [cited by applicant]
US 20160283712A1 · Kanhere et al. · 2016 [cited by applicant]
US 20160300063A1 · Daymont · 2016 [cited by examiner]
US 20170017789A1 · Daymont · 2017 [cited by examiner]
US 20170024562A1 · Rombouts et al. · 2017 [cited by applicant]
US 20170090929A1 · Muttik · 2017 [cited by applicant]
US 20170116108A1 · Miskelly · 2017 [cited by applicant]
US 20170169229A1 · Brucker et al. · 2017 [cited by applicant]
US 20170185775A1 · Boehm et al. · 2017 [cited by applicant]
US 20170212829A1 · Bales et al. · 2017 [cited by applicant]
US 20170255416A1 · Zhang et al. · 2017 [cited by applicant]
US 20170359220A1 · Weith · 2017 [cited by applicant]
US 20180004946A1 · LeMay et al. · 2018 [cited by applicant]
US 20180060209A1 · Kim · 2018 [cited by applicant]
US 20180095764A1 · Sultana · 2018 [cited by applicant]
US 20180096147A1 · Ince et al. · 2018 [cited by applicant]
US 20180101565A1 · Pike · 2018 [cited by applicant]
US 20180121654A1 · Bobritsky · 2018 [cited by applicant]
US 20180211046A1 · Muttik et al. · 2018 [cited by applicant]
US 20180225446A1 · Liu · 2018 [cited by applicant]
US 20180316698A1 · David et al. · 2018 [cited by applicant]
US 20180349603A1 · Yamada et al. · 2018 [cited by applicant]
US 20190042730A1 · Yamada et al. · 2019 [cited by applicant]
US 20190050566A1 · LeMay et al. · 2019 [cited by applicant]
US 20190065743A1 · Shibahara et al. · 2019 [cited by applicant]
US 20190108342A1 · Conikee et al. · 2019 [cited by applicant]
US 20190121979A1 · Chari · 2019 [cited by examiner]
US 20190129825A1 · Bardin et al. · 2019 [cited by applicant]
US 20190171423A1 · Mishra · 2019 [cited by applicant]
US 20190207969A1 · Brown · 2019 [cited by applicant]
US 20190266322A1 · Jones et al. · 2019 [cited by applicant]
US 20190294790A1 · Chang · 2019 [cited by examiner]
US 20190347415A1 · Yavo · 2019 [cited by applicant]
US 20200004954A1 · Zawadowskiy · 2020 [cited by applicant]
US 20200012786A1 · Lamothe-Brassard · 2020 [cited by applicant]
US 20200057856A1 · Daymont · 2020 [cited by applicant]
US 20200143061A1 · Kim · 2020 [cited by applicant]
US 20200162483A1 · Farhady et al. · 2020 [cited by applicant]
US 20200242239A1 · Loman et al. · 2020 [cited by applicant]
US 20200314124A1 · Reybok, Jr. et al. · 2020 [cited by applicant]
US 20200382560A1 · Woolward · 2020 [cited by applicant]
US 20200394313A1 · Ionescu · 2020 [cited by applicant]
US 20210089400A1 · Kashani et al. · 2021 [cited by applicant]
US 20210097168A1 · Patel · 2021 [cited by examiner]
US 20210133324A1 · Chari et al. · 2021 [cited by applicant]
US 20210152588A1 · Cruz · 2021 [cited by applicant]
US 20210157583A1 · Yuile et al. · 2021 [cited by applicant]
US 20210182393A1 · Chevalier et al. · 2021 [cited by applicant]
US 20210216634A1 · Kenyon · 2021 [cited by applicant]
US 20210232680A1 · Hecht · 2021 [cited by examiner]
US 20210264031A1 · Dhillon et al. · 2021 [cited by applicant]
US 20210279338A1 · Bowman et al. · 2021 [cited by applicant]
US 20210286600A1 · Calvano et al. · 2021 [cited by applicant]
US 20210312082A1 · Conikee · 2021 [cited by applicant]
US 20210390182A1 · Boutnaru · 2021 [cited by applicant]
US 20220019657A1 · Sethumadhavan et al. · 2022 [cited by applicant]
US 20220092179A1 · Zhang et al. · 2022 [cited by applicant]
US 20220108007A1 · Zatutschne-Marom · 2022 [cited by examiner]
US 20220156365A1 · Garba et al. · 2022 [cited by applicant]
US 20220391532A1 · Le et al. · 2022 [cited by applicant]
US 20230012722A1 · Del Rosario · 2023 [cited by applicant]
US 20230017384A1 · Woodward · 2023 [cited by examiner]
US 20230020547A1 · Katkoori et al. · 2023 [cited by applicant]
US 20230049789A1 · Seletskiy et al. · 2023 [cited by applicant]
US 20230195860A1 · Porter et al. · 2023 [cited by applicant]
US 20230289451A1 · Mackenbach · 2023 [cited by applicant]
US 20230315439A1 · Castrejon, III et al. · 2023 [cited by applicant]
US 20230367882A1 · Bussell et al. · 2023 [cited by applicant]
US 20230379342A1 · Gilad et al. · 2023 [cited by applicant]
US 20240028701A1 · Zawadowskiy · 2024 [cited by applicant]
US 20240028708A1 · Zawadowskiy · 2024 [cited by applicant]
US 20240028709A1 · Zawadowskiy · 2024 [cited by applicant]
US 20240028712A1 · Parla · 2024 [cited by applicant]
US 20240028724A1 · Parla · 2024 [cited by applicant]
US 20240028741A1 · Parla · 2024 [cited by applicant]
US 20240028743A1 · Parla · 2024 [cited by applicant]
US 20240031394A1 · Parla · 2024 [cited by applicant]
US 20240089272A1 · Gilad et al. · 2024 [cited by applicant]
US 20250272402A1 · Zawakowskiy · 2025 [cited by applicant]
CN 110268411A · 2019 [cited by applicant]
CN 111062038 · 2020 [cited by applicant]
CN 112818356A · 2021 [cited by applicant]
CN 113569244 · 2021 [cited by applicant]
CN 113434870B · 2022 [cited by applicant]
KR 102392642 · 2022 [cited by applicant]
Altinay, et al; “BinRec: Dynamic Binary Lifting and Recompilation”, Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing, Apr. 15, 2020, pp. 1-16. [cited by applicant]
Bardin, et al; “Backward-Bounded DSE: Targeting Infeasibility Questions on Obfuscated Codes”, 2017 IEEE Symposium on Security and Privacy (SP), IEEE, May 22, 2017, pp. 633-651. [cited by applicant]
DeLozier et al., “Hurdle, Securing Jump Instructions Against Code Reuse Atttacks”, Proceeduings of the 25th International Conference on Architechtural Support for Programming Languages and Operating Systems, ACM, New Yo… [cited by applicant]
Kumar, S., Moolchandani, D., & Sarangi, S. R. (2022). Hardware-assisted mechanisms to enforce control flow integrity: A comprehensive survey. Journal of Systems Architecture, 102644. [cited by applicant]
Liu, et al; “Transparent and Efficient CFI Enforcement with Intel Processor Trace,” 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), IEEE, US, Feb. 4, 2017, pp. 529-540. [cited by applicant]
The PCT Search Report and Written Opinion mailed Apr. 15, 2024 for PCT application No. PCT/US2023/028282, 15 pages. [cited by applicant]
The PCT Search Report and Written Opinion mailed Apr. 15, 2024 for PCT application No. PCT/US2023/084664, 14 pages. [cited by applicant]
The PCT Search Report and Written Opinion mailed Apr. 24, 2024 for PCT application No. PCT/US2023/084670, 14 pages. [cited by applicant]
The PCT Search Report and Written Opinion mailed Apr. 29, 2024 for PCT application No. PCT/US2023/084901, 13 pages. [cited by applicant]
The PCT Search Report and Written Opinion mailed May 3, 2024 for PCT application No. PCT/US2023/084659, 14 pages. [cited by applicant]
The PCT Search Report and Written Opinion Dated Apr. 25, 2024 for PCT Application No. PCT/US2023/084668, 15 pges. [cited by applicant]
Qiang, “CloudFI: Context-Sensitive and Incremental CFI in the Cloud Environment”, IEEE Transactions on Cloud Computing, IEEE Computer Society, USA vo1. 9, No. 3, Mar. 1, 2019 pp. 938-957. [cited by applicant]
Xinyang, et al; Griffin: Guarding Control Flows Using Intel Processor Trace, Architectural support for programming languages and Operating systems, Apr. 4, 2017, pp. 585-598. [cited by applicant]
Yuan, et al., “Hardware-Assisted 1-23 Fine-Grained Code-Reuse Attack Detection”, Dec. 12, 2015, SAT 2015 18th International Conference , Austin, TX, USA, Sep. 24-27, 2015 ; [Lecture Notes in Computer Science; Lect . Not… [cited by applicant]
Zhang et al: “DeepCheck: A Non-intrusive Control-flow Integrity Checking based on Deep Learning”, arxiv.org, Cornell University Library, 201 Olin Library Cornell University Ithaca, NY1, May 6, 2019, 10 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,007, Dated Oct. 22, 2024, Parla, “Control Flow Integrity Monitoring Based Insights”, 37 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,093, dated Aug. 29, 2024, 16 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,121, dated Sep. 11, 2024, 24 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,147, dated Sep. 6, 2024, 17 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,093, Dated Feb. 27, 2025, Parla, “Control Flow Prevention Using Software Bill of Materials Analysis ,” 33 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,147, Dated Mar. 20, 2025, Parla,“Control Flow Integrity Enforcement for Applications Running on Platforms,” 24 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,177, dated Jan. 13, 2025, 31 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/084,007, dated Jan. 29, 2025, 26 pages. [cited by applicant]
Da Silva, A. C. F., et al. (2022). Using a cyber digital twin for continuous automotive security requirements verification. IEEE Software. 10 pages. [cited by applicant]
Hiremagalore, Sharath “Zero-Day Web Attack Detection Using Collaborative and Transduction-Based Anomaly Detectors” George Mason University; published Nov. 9, 2021, 99 pages. [cited by applicant]
Maunero, et al. “A FPGA-Based Control-Flow Integrity Solution for Securing Bare-Metal Embedded Systems” Institute of Electrical and Electronics Engineers Inc. 2020, 11 pages. [cited by applicant]
Office Action for U.S. Appl. No. 18/083,838, Dated Oct. 31, 2024, Zawadowsky, “Control Flow Directed Graph for Use With Program Disassembler,” 9 pages. [cited by applicant]