IP Library › Granted Patent US 12,399,808
Granted Patent B2
US 12,399,808 · App. 18/144,949 · Granted Aug 26, 2025

Fuzzy testing a software system

Inventors: Martin Soukup (Ottawa, CA); Daniel Murdock (Ottawa, CA); Lama Moukahal (Ajax, CA); Mohammad Zulkernine (Kingston, CA)
Assignee: IRDETO B.V.
G06F11/3688G06F11/3644G06F11/3692
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,399,808
App. No.
18/144,949
Granted
Aug 26, 2025
Kind
B2
Abstract

A method for a testing system to perform fuzzy testing of a software system, wherein the software system comprises a plurality of callable units and is arranged to receive input for the software system to process, the method comprising: determining, for each callable unit of the plurality of callable units, based on one or more security vulnerability metrics, a target number of times that callable unit is to be tested; initializing a ranked plurality of queues, each queue for storing one or more seeds, said initializing comprising storing one or more initial seeds in a corresponding queue of the ranked plurality of queues; performing a sequence of tests, wherein performing each test comprises: obtaining a seed from the highest ranked non-empty queue; performing a mutation process on the obtained seed to generate a test seed, wherein the mutation process is configured, at least in part, by mutation guidance information; providing the test seed as input to the software system for the software system to process; and evaluating the processing of the test seed by the software system to generate a result for the test; wherein each queue in the ranked plurality of queues has an associated seed addition criterion and wherein performing each test comprises either (a) adding the test seed to the highest ranked queue in the ranked plurality of queues for which the test seed meets the seed addition criterion associated with that queue; or (b) discarding the test seed if the test seed does not meet the seed addition criterion associated with any of the queues in the ranked plurality of queues; wherein the seed addition criteria are configured so that, if processing of a first test seed by the software system involves execution of, or an execution path approaching, a callable unit of interest and if processing of a second test seed by the software system does not involve execution of, or an execution path approaching, a callable unit of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added, wherein a callable unit is a callable unit of interest if the current number of tests that have resulted in execution of that callable unit is less than the target number of times that callable unit is to be tested.

Claims (55)

1. A method for a testing system to perform fuzzy testing of a software system, wherein the software system comprises a plurality of callable units and is arranged to receive input for the software system to process, the method comprising:

determining, for each callable unit of the plurality of callable units, based on one or more security vulnerability metrics, a target number of times that callable unit is to be tested;

initializing a ranked plurality of queues, each queue for storing one or more seeds, said initializing comprising storing one or more initial seeds in a corresponding queue of the ranked plurality of queues;

performing a sequence of tests, wherein performing each test comprises:

obtaining a seed from the highest ranked non-empty queue;

performing a mutation process on the obtained seed to generate a test seed, wherein the mutation process is configured, at least in part, by mutation guidance information;

providing the test seed as input to the software system for the software system to process; and

evaluating the processing of the test seed by the software system to generate a result for the test;

wherein each queue in the ranked plurality of queues has an associated seed addition criterion and wherein performing each test comprises either (a) adding the test seed to the highest ranked queue in the ranked plurality of queues for which the test seed meets the seed addition criterion associated with that queue; or (b) discarding the test seed if the test seed does not meet the seed addition criterion associated with any of the queues in the ranked plurality of queues;

wherein the seed addition criteria are configured so that, if processing of a first test seed by the software system involves execution of, or an execution path approaching, a callable unit of interest and if processing of a second test seed by the software system does not involve execution of, or an execution path approaching, a callable unit of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added, wherein a callable unit is a callable unit of interest if the current number of tests that have resulted in execution of that callable unit is less than the target number of times that callable unit is to be tested.

2. The method of claim 1 , wherein the mutation guidance information is arranged to configure the mutation process so that test seeds generated by the mutation process are less likely to be invalid inputs for the software system.

3. The method of claim 1 , wherein the mutation guidance information is arranged to configure the mutation process to increase the likelihood that processing, by the software system, of a test seed generated by the mutation process involves execution of, or an execution path approaching, a callable unit of interest.

4. The method of claim 1 , wherein the mutation guidance information specifies a range of values for a quantity represented by the test seed, and wherein the mutation guidance information is arranged to configure the mutation process to (a) ensure that the value of said quantity as represented by the test seed is within said range of values; or (b) ensure that the value of said quantity as represented by the test seed is outside said range of values; or (c) bias the value of said quantity as represented by the test seed to be within said range of values; or (d) bias the value of said quantity as represented by the test seed to be outside said range of values.

5. The method of claim 1 , wherein the mutation guidance information is arranged to configure the mutation process with a target distribution for values of a quantity represented by generated test seeds.

6. The method of claim 5 , wherein the target distribution is based on an algorithmic characteristic of one or more callable units of interest.

7. The method of claim 1 , wherein the mutation guidance information is arranged to configure the mutation process to use, for at least some of the generated test seeds, one or more predetermined values as the value of a corresponding quantity represented by the at least some of the generated test seeds.

8. The method of claim 4 , wherein the mutation guidance information specifies that the range of values for the quantity represented by the test seed is determined, at least in part, based on a value of another quantity represented by that test seed or by the corresponding obtained seed.

9. The method of claim 5 , wherein the mutation guidance information specifies that the target distribution for a value of a quantity represented by a generated test seed is determined, at least in part, based on a value for another quantity represented by that test seed or by the corresponding obtained seed.

10. The method of claim 1 , wherein the mutation guidance information is arranged to configured the mutation process to implement a change to a value of at least one quantity represented by obtained seeds more frequently than implementing a change to a value of at least one other quantity represented by the obtained seeds.

11. The method of claim 10 , wherein the mutation guidance information is arranged to configure the mutation process to avoid implementing a change to a value of one or more of the at least one other quantity represented by the obtained seeds.

12. The method of claim 1 , wherein the mutation guidance information is provided and/or generated by an operator of the testing system.

13. The method of claim 1 , wherein the seed addition criteria are configured for one or more of the following:

(a) if processing of a first test seed by the software system involves an execution path approaching a callable unit of interest but does not involve execution of a callable unit of interest and if processing of a second test seed by the software system involves execution of a callable unit of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added;

(b) if processing of a first test seed by the software system involves an execution path approaching a callable unit of interest but does not involve execution of a callable unit of interest and if processing of a second test seed by the software system involves execution of a callable unit of interest, then the queue to which the first test seed is added is of lower rank than the queue to which the second test seed is added;

(c) if processing of a first test seed by the software system involves execution of, or an execution path approaching, one or more first callable units of interest and if processing of a second test seed by the software system involves execution of, or an execution path approaching, one or more second callable units of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added if: at least one of the one or more first callable units of interest has a remaining number of times to be tested greater than a remaining number of times each of the one or more second callable units of interest are to be tested; or a sum of a remaining number of times each of the one or more first callable units of interest are to be tested is greater than a sum of a remaining number of times each of the one or more second callable units of interest are to be tested;

(d) the seed addition criterion for a first queue is that processing of the test seed by the software system involves execution of, or an execution path approaching, a callable unit of interest; and

(e) the seed addition criterion for a second queue is that processing of the test seed by the software system reaches a branch point in the software system that has not been reached when performing a previous test, wherein optionally the first queue has a higher rank than the second queue and wherein further optionally the ranked plurality of queues is the set containing the first queue and the second queue.

14. The method of claim 1 , wherein performing a mutation process on the obtained seed to generate a test seed comprises one of:

(a) mutating the obtained seed to form the test seed; and

(b) setting the test seed to be the obtained seed if the obtained seed is an initial seed; and mutating the obtained seed to form the test seed otherwise.

15. The method of claim 1 , wherein:

(a) for each callable unit of the plurality of callable units, determining the target number of times that callable unit is to be tested generates a higher target number when the one or more security vulnerability metrics indicate a higher level of security vulnerability for the callable unit; and/or

(b) each callable unit is a respective one of: a routine; a subroutine; a function; a procedure; a process; a class method; an interface; a component; or a subsystem of a larger system; and/or

(c) the one or more security vulnerability metrics comprise one or more of: a metric representing a degree of security vulnerability and/or security criticality of a callable unit; a metric representing a risk that a malicious message may be passed from one callable unit to another callable unit; a metric based on a number of and/or types of communication techniques used by a callable unit; a metric based on a level of complexity of code of a callable unit; a metric based on a number of input and output parameters of a callable function which have varying values and/or a degree to which input and output parameters of a callable function can have varying values;

and a metric based on historical vulnerability data relating to a callable unit.

16. A testing system for fuzzy testing a software system, wherein the software system comprises a plurality of callable units and is arranged to receive input for the software system to process, the testing system comprising one or more processors arranged to:

determine, for each callable unit of the plurality of callable units, based on one or more security vulnerability metrics, a target number of times that callable unit is to be tested;

initialize a ranked plurality of queues, each queue for storing one or more seeds, said initializing comprising storing one or more initial seeds in a corresponding queue of the ranked plurality of queues;

perform a sequence of tests, wherein performing each test comprises:

obtaining a seed from the highest ranked non-empty queue;

performing a mutation process on the obtained seed to generate a test seed;

providing the test seed as input to the software system for the software system to process; and

evaluating the processing of the test seed by the software system to generate a result for the test;

wherein each queue in the ranked plurality of queues has an associated seed addition criterion and wherein performing each test comprises either (a) adding the test seed to the highest ranked queue in the ranked plurality of queues for which the test seed meets the seed addition criterion associated with that queue: or (b) discarding the test seed if the test seed does not meet the seed addition criterion associated with any of the queues in the ranked plurality of queues;

wherein the seed addition criteria are configured so that, if processing of a first test seed by the software system involves execution of, or an execution path approaching, a callable unit of interest and if processing of a second test seed by the software system does not involve execution of, or an execution path approaching, a callable unit of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added, wherein a callable unit is a callable unit of interest if the current number of tests that have resulted in execution of that callable unit is less than the target number of times that callable unit is to be tested.

17. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors of a testing system, cause the testing system to perform fuzzy testing of a software system, wherein the software system comprises a plurality of callable units and is arranged to receive input for the software system to process, the fuzzy testing of the software system comprising:

determining, for each callable unit of the plurality of callable units, based on one or more security vulnerability metrics, a target number of times that callable unit is to be tested;

initializing a ranked plurality of queues, each queue for storing one or more seeds, said initializing comprising storing one or more initial seeds in a corresponding queue of the ranked plurality of queues;

performing a sequence of tests, wherein performing each test comprises:

obtaining a seed from the highest ranked non-empty queue;

performing a mutation process on the obtained seed to generate a test seed, wherein the mutation process is configured, at least in part, by mutation guidance information;

providing the test seed as input to the software system for the software system to process; and

evaluating the processing of the test seed by the software system to generate a result for the test;

wherein each queue in the ranked plurality of queues has an associated seed addition criterion and wherein performing each test comprises either (a) adding the test seed to the highest ranked queue in the ranked plurality of queues for which the test seed meets the seed addition criterion associated with that queue: or (b) discarding the test seed if the test seed does not meet the seed addition criterion associated with any of the queues in the ranked plurality of queues;

wherein the seed addition criteria are configured so that, if processing of a first test seed by the software system involves execution of, or an execution path approaching, a callable unit of interest and if processing of a second test seed by the software system does not involve execution of, or an execution path approaching, a callable unit of interest, then the queue to which the first test seed is added is of higher rank than the queue to which the second test seed is added, wherein a callable unit is a callable unit of interest if the current number of tests that have resulted in execution of that callable unit is less than the target number of times that callable unit is to be tested.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2025
From: SOUKUP, MARTIN
To: IRDETO CANADA CORPORATION
Reel/Frame 071310/0654 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2025
From: MURDOCK, DAN
To: IRDETO CANADA CORPORATION
Reel/Frame 071310/0764 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2025
From: MOUKAHAL, LAMA
To: IRDETO CANADA CORPORATION
Reel/Frame 071310/0784 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2025
From: ZULKERNINE, MOHAMMAD
To: IRDETO CANADA CORPORATION
Reel/Frame 071310/0924 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2025
From: IRDETO CANADA CORPORATION
To: IRDETO B.V.
Reel/Frame 071311/0037 →
Priority Claims (1)
EP 22173068.2 · May 12, 2022 · regional
Continuity (1)
Related Publication 20230367704A1 · Nov 16, 2023
References Cited (90)
US 20150379187A1 · Gou · 2015 [cited by examiner]
US 20180365139A1 · Rajpal et al. · 2018 [cited by applicant]
US 20190384697A1 · Phan et al. · 2019 [cited by applicant]
US 20200341887A1 · Landsborough et al. · 2020 [cited by applicant]
US 20210326246A1 · Li · 2021 [cited by examiner]
S. Parkinson, et al., “Cyber threats facing autonomous and connected vehicles: Future challenges,” IEEE transactions on intelligent transportation systems, vol. 18, No. 11, pp. 2898-2915, Jan. 1, 2017. [cited by applicant]
S. V. E. S. S. Committee et al., “Sae j3061-cybersecurity guidebook for cyber-physical automotive systems,” SAE—Society of Automotive Engineers, Jan. 1, 2016. [cited by applicant]
L. J. Moukahal, et al., “Vehicle software engineering (vse): Research and practice,” IEEE Internet of Things Journal, vol. 7, No. 10, pp. 10 137-10 149, Jan. 1, 2020. [cited by applicant]
B. Potter and G. McGraw, “Software security testing,” IEEE Security & Privacy, vol. 2, No. 5, pp. 81-85, Jan. 1, 2004. [cited by applicant]
U. Drolia, et al., “Autoplug: An automotive test-bed for electronic controller unit testing and verification,” in 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), Jan. 1, 2011, pp. 1… [cited by applicant]
S. Bayer, T. Enderle, D.-K. Oka, and M. Wolf, “Security crash test-practical security evaluations of automotive onboard it components,” Automotive-Safety & Security, Jan. 1, 2015. [cited by applicant]
D. S. Fowler, J. Bryans, M. Cheah, P. Wooderson, and S. A. Shaikh, “A method for constructing automotive cybersecurity tests, a can fuzz testing example,” in 2019 IEEE 19th International Conference on Software Quality, … [cited by applicant]
A. Pretschner, M. Broy, I. H. Kruger, and T. Stauner, “Software engineering for automotive systems: A roadmap,” in Future of Software Engineering (FOSE'07). IEEE, Jan. 1, 2007, pp. 55-71. [cited by applicant]
M. Broy, I. H. Kruger, A. Pretschner, and C. Salzmann, “Engineering automotive software,” Proceedings of the IEEE, vol. 95, No. 2, pp. 356-373, Jan. 1, 2007. [cited by applicant]
S. Moshtari, A. Sami, and M. Azimi, “Using complexity metrics to improve software security,” Computer Fraud & Security, vol. 2013, No. 5, pp. 8-17, Jan. 1, 2013. [cited by applicant]
M. Broy, “Challenges in automotive software engineering,” in Proceedings of the 28th International Conference on Software Engineering. ACM, Jan. 1, 2006, pp. 33-42. [cited by applicant]
S. A. Haider, G. Samdani, M. Ali, and M. Kamran, “A comparative analysis of in-house and outsourced development in software industry,” International Journal of Computer Applications, vol. 141, No. 3, pp. 18-22, Jan. 1, … [cited by applicant]
C. Hubmann, M. Becker, D. Althoff, D. Lenz, and C. Stiller, “Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles,” in 2017 IEEE Intelligent Vehicles Symposium … [cited by applicant]
P. Koopman and M. Wagner, “Challenges in autonomous vehicle testing and validation,” SAE International Journal of Transportation Safety, vol. 4, No. 1, pp. 15-24, Jan. 1, 2016. [cited by applicant]
P. Koopman and M. Wagner, “Autonomous vehicle safety: An interdisciplinary challenge,” IEEE Intelligent Transportation Systems Magazine, vol. 9, No. 1, pp. 90-96, Jan. 1, 2017. [cited by applicant]
J. De Winter, P. M. van Leeuwen, and R. Happee, “Advantages and disadvantages of driving simulators: A discussion,” in Proceedings of measuring behavior, vol. 2012. Citeseer, Jan. 1, 2012, p. 8th. [cited by applicant]
C. Obermaier, R. Riebl, C. Facchi, A. Al-Bayatti, and S. Khan, “Limitations of hil test architectures for car2x communication devices and applications,” in ACM Computer Science in Cars Symposium, Jan. 1, 2019, pp. 1-9. [cited by applicant]
I. Pekaric, C. Sauerwein, and M. Felderer, “Applying security testing techniques to automotive engineering,” in Proceedings of the 14th International Conference on Availability, Reliability and Security, Jan. 1, 2019, p… [cited by applicant]
A. Imparato, R. R. Maietta, S. Scala, and V. Vacca, “A comparative study of static analysis tools for autosar automotive software components development,” in 2017 IEEE International Symposium on Software Reliability Eng… [cited by applicant]
S. Keul, “Tuning static data race analysis for automotive control software,” in 2011 IEEE 11th International Working Conference on Source Code Analysis and Manipulation. IEEE, Jan. 1, 2011, pp. 45-54. [cited by applicant]
A. G. Bardas et al., “Static code analysis,” Journal of Information Systems & Operations Management, vol. 4, No. 2, pp. 99-107, Jan. 1, 2010. [cited by applicant]
Z. B. Celik, E. Fernandes, E. Pauley, G. Tan, and P. McDaniel, “Program analysis of commodity iot applications for security and privacy: Challenges and opportunities,” ACM Computing Surveys (CSUR), vol. 52, No. 4, pp. 1… [cited by applicant]
K. A. Koscher, “Securing embedded systems: analyses of modern automotive systems and enabling near-real time dynamic analysis,” Ph.D. dissertation, Jan. 1, 2014. [cited by applicant]
G. Cabodi, D. F. S. Finocchiaro, and D. Montisci, “Security-oriented dynamic code analysis in automotive embedded systems.” Jan. 1, 2017, Politecnico Di Torino, Master's Degree thesis in Ingegneria Informatica (Computer… [cited by applicant]
M. Ring, J. Durrwang, F. Sommer, and R. Kriesten, “Survey on vehicular attacks-building a vulnerability database,” in 2015 IEEE International Conference on Vehicular Electronics and Safety (ICVES). IEEE, Jan. 1, 2015, p… [cited by applicant]
F. Sommer, J. Durrwang, and R. Kriesten, “Survey and classification of automotive security attacks,” Information, vol. 10, No. 4, p. 148, Jan. 1, 2019. [cited by applicant]
K. Koscher, A. Czeskis, F. Roesner, S. Patel, T. Kohno, S. Checkoway, D. McCoy, B. Kantor, D. Anderson, H. Shacham et al., “Experimental security analysis of a modern automobile,” in 2010 IEEE Symposium on Security and … [cited by applicant]
M. Cheah, S. A. Shaikh, O. Haas, and A. Ruddle, “Towards a systematic security evaluation of the automotive bluetooth interface,” Vehicular Communications, vol. 9, pp. 8-18, Jan. 1, 2017. [cited by applicant]
C. Corbett, T. Basic, T. Lukaseder, and F. Kargl, “A testing framework architecture for automotive intrusion detection systems,” Automotive-Safety & Security 2017-Sicherheit und Zuverlässigkeit für automobile Informatio… [cited by applicant]
H. Liang, X. Pei, X. Jia, W. Shen, and J. Zhang, “Fuzzing: State of the art,” IEEE Transactions on Reliability, vol. 67, No. 3, pp. 1199-1218, Jan. 1, 2018. [cited by applicant]
D. K. Oka, T. Fujikura, and R. Kurachi, “Shift left: Fuzzing earlier in the automotive software development lifecycle using hil systems,” Jan. 1, 2018. [cited by applicant]
D. S. Fowler, J. Bryans, S. A. Shaikh, and P. Wooderson, “Fuzz testing for automotive cyber-security,” in 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), Jan. 1,… [cited by applicant]
P. Wang and X. Zhou, “Sok: The progress, challenges, and perspectives of directed greybox fuzzing,” arXiv preprint arXiv:2005.11907, Jan. 1, 2020. [cited by applicant]
M. Zalewski, “American fuzzy lop.(2015),” URL http://lcamtuf.coredump.cx/afl, Jan. 1, 2015. [cited by applicant]
V.-T. Pham, M. Bohme, A. E. Santosa, A. R. Caciulescu, and A. Roy-choudhury, “Smart greybox fuzzing,” IEEE Transactions on Software Engineering, Jan. 1, 2019. [cited by applicant]
C. Lemieux and K. Sen, “Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineer┐ing, Jan. 1, 2018… [cited by applicant]
M. Böhme, V.-T. Pham, and A. Roychoudhury, “Coverage-based grey-box fuzzing as markov chain,” IEEE Transactions on Software Engineering, vol. 45, No. 5, pp. 489-506, Jan. 1, 2017. [cited by applicant]
P. Chen and H. Chen, “Angora: Efficient fuzzing by principled search,” in 2018 IEEE Symposium on Security and Privacy (SP). IEEE, Jan. 1, 2018, pp. 711-725. [cited by applicant]
V. Wüstholz and M. Christakis, “Learning inputs in greybox fuzzing,” arXiv preprint arXiv:1807.07875, Jan. 1, 2018. [cited by applicant]
V. Jain, S. Rawat, C. Giuffrida, and H. Bos, “Tiff: using input type inference to improve fuzzing,” in Proceedings of the 34th Annual Computer Security Applications Conference, Jan. 1, 2018, pp. 505-517. [cited by applicant]
S. Rawat and L. Mounier, “Offset-aware mutation based fuzzing for buffer overflow vulnerabilities: Few preliminary results,” in 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation … [cited by applicant]
R. C. Bhushan and D. D. Yadav, “Number of test cases required in achieving statement, branch and path coverage using ‘gcov’: An analysis,” in 7th International Workshop on Computer Science and Engineering (WCSE 2017) Be… [cited by applicant]
Wikipedia, “Fuzzing,” Oct. 24, 2020. Available: https://en.wikipedia.org/w/index.php?title=Fuzzing&oldid=985264480. [cited by applicant]
Team Upstream, “Q1 2019 sees a rapid growth of automotive cyber incidents.” Jun. 1, 2019. Available: https://www.upstream.auto/blog/q1-2019-sees-a-rapid-growth-of-automotive-cyber-incidents/. [cited by applicant]
International Organization for Standardization, “Iso/iec 27005”. Jan. 1, 2018. Available: https://www.iso.org/standard/43464.html. [cited by applicant]
SAE Mobilus, “Road vehicles—cybersecurity engineering iso/sae dis 21434”. Feb. 12, 2020, SAE International, Vehicle Cybersecurity Systems Engineering Committee. Available: https://www.sae.org/standards/content/iso/sae21… [cited by applicant]
D. Oka, Ponemon Institute, “Securing the modern vehicle: A study of automotive industry cybersecurity practices,” An Independent study commissioned by SAE International and Synopsys, Dec. 1, 2018. [cited by applicant]
Moukahal, L., et al., “Vulnerability-Oriented Fuzz Testing for Connected Autonomous Vehicle Systems.” IEEE Transactions on Reliability, vol. 70, No. 4, (2021), pp. 1422-1437. [cited by applicant]
Moukahal, L., et al., “Boosting Grey-box Fuzzing for Connected Autonomous Vehicle Systems.” In 2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C) Dec. 2021 (pp. 516-5… [cited by applicant]
ETAS, “Successful security tests using fuzzing and hil test systems.,” Translated article “Effektive Security-Tests am HiL-System,” Hanser Automotive Nov. 1, 2016. Available: https://www.etas.com/download-center-files/p… [cited by applicant]
John Mcshane, Escrypt, “The fuzz on automotive cybersecurity testing.” Escrypt North America blog, Oct. 5, 2020. Available at: https://securitybyescrypt.com/fuzztesting/. [cited by applicant]
Synopsys, “Defensics fuzz testing.” Jan. 1, 2019. Available: https://www.synopsys.com/software-integrity/security-testing/fuzz-testing.html. [cited by applicant]
Pearl-Doughty White, et al., Informationisbeautiful.net. “Million Lines of Code—Information is Beautiful”, Sep. 24, 2015. Available: https://www.informationisbeautiful.net/visualizations/million-lines-of-code/. [cited by applicant]
Vector Informatik GmbH, “Testing ecus and networks with CANoel.” Jan. 1, 2017. Available: https://www.vector.com/int/en/products/products-a-z/software/canoe/. [cited by applicant]
Opal-rt., “Opal-rt testing platform for automotive simulation.” Mar. 1, 2019. Available: https://www.opal-rt.com/automotive-overview/. [cited by applicant]
Simulationx, “Design and Analyze Your Multi-Physics System with Simulation Software.” Feb. 1, 2019. Available: https://www.simulationx.com/industries/simulation-automotive.html. [cited by applicant]
Mathworks, “Polyspace code prover.” Jan. 1, 2017. Available: https://www.mathworks.com/products/polyspace-code-prover.html. [cited by applicant]
Misra, “What is Misra?”, Jan. 1, 2017. Available: https://www.misra.org.uk/MISRAHome/WhatisMISRA/tabid/66/Default.aspx. [cited by applicant]
G. Zhang and X. Zhou, afl extended with test case prioritization techniques, Int. J. Model. Optim, vol. 8, No. 1, pp. 41-45, Feb. 1, 2018. [cited by applicant]
Omnet++, “Simulation Model and Tools,” Jan. 1, 2017, Available at: https://omnetpp.org/download/models-and-tools. [cited by applicant]
QA Systems, “Integrated Static Analysis;” Jan. 1, 2018, Available at: https://www.qa-systems.com/tools/Integrated-static-analysis/. [cited by applicant]
Auto-Isac, “Automotive information sharing and analysis center,” Membership and Best Practices, Jan. 1, 2017, Available at https://automotiveisac.com/. [cited by applicant]
Openpilot, “What is Openpilot?”, Jun. 11, 2018, Available at: https://github.com/commaai/openpilot#/what-is-openpilot. [cited by applicant]
Openpilot, “Process Replay,” Jan. 1, 2018. Available at: https://github.com/commaai/openpilot/tree/master/selfdrive/test/process_replay. [cited by applicant]
Brennich, T., Moser, M. Putting Automotive Security to the Test. ATZ Electron Worldw 15, 46-51 (Jan. 1, 2020). https://doi.org/10.1007/s38314-019-0155-9. [cited by applicant]
D. S. Fowler, J. Bryans, S. A. Shaikh, and P. Wooderson, “Fuzz testing for automotive cyber-security,” in 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W). IEEE, J… [cited by applicant]
D. Durisic, M. Nilsson, M. Staron, and J. Hansson, “Measuring the impact of changes to the complexity and coupling properties of automotive software systems,” Journal of Systems and Software, vol. 86, No. 5, pp. 1275-12… [cited by applicant]
Y. Shin and L. Williams, “Can traditional fault prediction models be used for vulnerability prediction?” Empirical Software Engineering, vol. 18, No. 1, pp. 25-59, Dec. 9, 2011. [cited by applicant]
I. Chowdhury and M. Zulkernine, “Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities,” Journal of Systems Architecture, vol. 57, No. 3, pp. 294-313, Jun. 19, 2010. [cited by applicant]
A. Taylor, S. Leblanc, and N. Japkowicz, “Probing the limits of anomaly detectors for automobiles with a cyberattack framework,” IEEE Intelligent Systems, vol. 33, No. 2, pp. 54-62, Mar. 1, 2018. [cited by applicant]
T. Huang, J. Zhou, and A. Bytes, “Atg: An attack traffic generation tool for security testing of in-vehicle CAN bus,” in Proceedings of the 13th International Conference on Availability, Reliability and Security, Aug. 2… [cited by applicant]
L. Moukahal and M. Zulkernine, “Security vulnerability metrics for connected vehicles,” in 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE, Jan. 1, 2019, pp.… [cited by applicant]
Autosar, “Autosar—enabling continuous innovations—About—Organization,” Jan. 1, 2020, Available at https://www.autosar.org/about/organization/. [cited by applicant]
National Instruments Corp., “What is the iso 26262 functional safety standard?” Jan. 1, 2020. Available: https://www.ni.com/en-ca/innovations/white-papers/11/what-is-the-iso-26262-functional-safety-standard-.html. [cited by applicant]
SAE International, “Society of automotive engineers—Overview,” Jan. 1, 2020. Available at: https://www.sae.org/about. [cited by applicant]
InsectIndentification.org, “Bugfinder—insect search and identification tool.” Jan. 1, 2017. Available: https://www.Insectidentification.org/bugfinder-start.asp. [cited by applicant]
A. Zeller, R. Gopinath, M. Böhme, G. Fraser, and C. Holler, “The fuzzing book, Introduction to Software Testing” Jan. 1, 2019. Available: https://www.fuzzingbook.org/html/Intro_Testing.html. [cited by applicant]
Comma.ai., “FAQ/What is openpilot?” Jan. 1, 2020. Available: https://comma.ai/faq. [cited by applicant]
D. K. Oka, A. Yvard, S. Bayer, and T. Kreuzinger, “Enabling cyber security testing of automotive ecus by adding monitoring capabilities,” in Embedded SECUrity in Cars Conference, 15th Escar Europe, Nov. 13, 2016. [cited by applicant]
Zhang, “AFL Extended with Test Case Prioritization Techniques”, 2018, International Journal of Modeling and Optimization, vol. 8, No. 1, Feb. 2018 (Year: 2018). [cited by applicant]
Bohme, “Coverage-based Greybox Fuzzing as Markov Chain”, 2017, Transactions on Software Engineering, vol. XX, No. X, Dec. 2017 (Year: 2017). [cited by applicant]
EP21209587.1 Extended European Search Report dated Feb. 5, 2022. [cited by applicant]
Pengfei, W., et al., “SoK: The Progress, Challenges, and Perspectives of Directed Greybox Fuzzing.” arxiv.org, Cornell University Library; Ithaca, NY 14853; Jun. 4, 2020; 16 pages. [cited by applicant]
Liang, H., et al., “Sequence Directed Hybrid Fuzzing” 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE; Feb. 18, 2020; pp. 127-137. [cited by applicant]
Moukahal, L., et al., “Vulnerability-Oriented Fuzz Testing for Connected Autonomous Vehicle Systems.” IEEE Transactions on Reliability, IEEE Service Center, Piscataway, NJ, US; Oct. 4, 2021; pp. 1422-1437; vol. 7, No. 4. [cited by applicant]