IP Library › Granted Patent US 12,315,233
Granted Patent B1
US 12,315,233 · App. 18/079,869 · Granted May 27, 2025

Optical fuzzer

Inventor: Robi Sen (Richland, WA)
G06V10/776G05D1/042G06V10/141G06V10/82G06V10/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,233
App. No.
18/079,869
Granted
May 27, 2025
Kind
B1
Abstract

A machine-learning system is trained to adapt light transmitted from an array of light emitters in order to disrupt a navigation system that employs a camera. Training comprises receiving image-processing outputs and/or navigation signals from the navigation system; configuring the array of light emitters to occupy a portion of a field of view of the camera; adapting a modulation pattern of light emitted by the array; and determining, from the image-processing outputs and/or navigation signals, if the modulation pattern affects at least one of image processing or navigation control performed in the navigation system.

Claims (23)

1. A method to manipulate a vehicle that employs one or more optical sensors for autonomous navigation, the method comprising:

receiving measurements from at least one sensor that is responsive to the vehicle's movement;

employing an artificial neural network (ANN) to adapt a modulation pattern used to modulate an array of light emitters;

transmitting the modulation pattern from the array of light emitters to the one or more optical sensors; and

determining, from the measurements, if the modulation pattern affects the vehicle's movement and configuring the ANN to adapt the modulation pattern to cause a predetermined type of movement.

2. The method of claim 1 , wherein the at least sensor comprises at least one of a camera, an optical sensor, a LIDAR, a RADAR, or an acoustic sensor.

3. The method of claim 1 , wherein receiving measurements comprises measuring the vehicle's movement.

4. The method of claim 3 , wherein measuring the vehicle's movement comprises measuring at least one of yaw, pitch, roll, heading, speed, velocity, altitude, acceleration, deceleration, vibration, ascent, descent, or derivatives thereof with respect to time.

5. The method of claim 1 , wherein the vehicle is an unmanned aerial vehicle, a ground vehicle, a boat, a submarine, or a missile.

6. The method of claim 1 , wherein the light emitters comprise at least one of light-emitting diodes and lasers.

7. The method of claim 1 , wherein the array is a linear array, a planar array, or a volumetric array.

8. The method of claim 1 , wherein the modulation pattern comprises at least one of amplitude modulation, on-off keying, frequency modulation, phase modulation, index modulation, or spatial modulation.

9. The method of claim 8 , wherein the spatial modulation comprises at least one of linear modulation, planar modulation, or volumetric modulation.

10. The method of claim 1 , wherein determining, from the measurements, if the modulation pattern affects the vehicle's movement comprises producing labeled data sets comprising modulation patterns as input data and measurements of the vehicle's movement as associated ground truths.

11. The method of claim 1 , wherein the ANN is a deep-learning neural network.

12. The method of claim 1 , further comprising configuring the array of light emitters to affect only a portion of the one or more optical sensors' field of view.

13. A method of disrupting a navigation system that employs a camera, the method comprising:

receiving output signals from the navigation system comprising at least one of image-processing outputs and navigation signals;

configuring an array of light emitters to occupy a portion of a field of view of the camera;

employing an artificial neural network (ANN) to adapt a modulation pattern of light emitted by the array; and

determining, from the output signals, if the modulation pattern affects at least one of image processing or navigation control performed in the navigation system and configuring the ANN to adapt the modulation pattern to cause a predetermined type of movement.

14. The apparatus of claim 13 , wherein the image-processing outputs comprise outputs of a camera neural network that is responsive to an image captured by the camera; wherein the ANN denotes a target image-processing output as a ground truth; and wherein the ANN computes an error function as a difference between the image-processing outputs and the ground truth.

15. The apparatus of claim 14 , wherein the ANN performs gradient descent to update the modulation pattern, the gradient descent comprising a function of the error function.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 5, 2024
From: SEN, ROBI, MR.
To: TIDAL IQ, LLC
Reel/Frame 067024/0968 →
CONFIRMATORY LICENSE Recorded Dec 14, 2023
From: COGNOSCENTI LABS, LLC
To: GOVERNMENT OF THE UNITED STATES AS REPRESENTED BY THE SECRETARY OF THE AIR FORCE
Reel/Frame 066005/0368 →
GOVERNMENT INTEREST AGREEMENT Recorded Jul 13, 2023
From: COGNOSCENTI LABS, LLC
To: GOVERNMENT OF THE UNITED STATES AS REPRESENTED BY THE SECRETARY OF THE AIR FORCE
Reel/Frame 064268/0111 →
Continuity (1)
Provisional Application 63288577 · Dec 11, 2021
References Cited (49)
US 7817085B2 · Harris · 2010 [cited by applicant]
US 8254847B2 · Sen · 2012 [cited by applicant]
US 8779912B2 · Sverrisson et al. · 2014 [cited by applicant]
US 10051475B2 · Shattil et al. · 2018 [cited by applicant]
US 10122694B2 · Shattil et al. · 2018 [cited by applicant]
US 10198645B2 · Graumann et al. · 2019 [cited by applicant]
US 10239637B2 · Ashdown et al. · 2019 [cited by applicant]
US 10469186B2 · Shattil et al. · 2019 [cited by applicant]
US 10852433B2 · Chen · 2020 [cited by examiner]
US 10985861B2 · Nguyen et al. · 2021 [cited by applicant]
US 11273841B2 · Goldman · 2022 [cited by applicant]
US 11281921B2 · Nikitidis et al. · 2022 [cited by applicant]
US 11310439B2 · Djakovic et al. · 2022 [cited by applicant]
US 11341778B1 · Joshi et al. · 2022 [cited by applicant]
US 20080103649A1 · Shen et al. · 2008 [cited by applicant]
US 20120169522A1 · Huneycutt · 2012 [cited by applicant]
US 20160226892A1 · Sen et al. · 2016 [cited by applicant]
US 20170132477A1 · Kim et al. · 2017 [cited by applicant]
US 20170228586A1 · Morishita · 2017 [cited by applicant]
US 20180034852A1 · Goldenberg · 2018 [cited by applicant]
US 20180130168A1 · Nayshtut et al. · 2018 [cited by applicant]
US 20190020439A1 · Fawaz et al. · 2019 [cited by applicant]
US 20200083979A1 · Roberts et al. · 2020 [cited by applicant]
US 20210110018A1 · Rowe et al. · 2021 [cited by applicant]
US 20210110185A1 · Rowe et al. · 2021 [cited by applicant]
US 20210303890A1 · Wei · 2021 [cited by applicant]
US 20210343165A1 · Baumgart et al. · 2021 [cited by applicant]
US 20230076107A1 · Rudow et al. · 2023 [cited by applicant]
US 20230139748A1 · Fox et al. · 2023 [cited by applicant]
Rini, Controlling a Laser's Phase, Sep. 20, 2016, Physics, https://physics.aps.org/articles/v9/s102 (Year: 2016). [cited by examiner]
Wallace, Photonics Products: High-power Laser-Diode Arrays: Diode arrays are compact, high-power light dynamos, Jan. 16, 2015, Lasers and Sources, https://www.laserfocusworld.com/lasers-sources/article/16551520/photonic… [cited by examiner]
Davidson et al., Controlling UAVs with Sensor Input Spoofing Attacks, 2016, 10th USENIX Workshop on Offensive Technologies (WOOT 16) (Year: 2016). [cited by examiner]
Cao et al., Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving, 2019, CCS (Year: 2019). [cited by examiner]
Yan et al., Can You Trust Autonomous Vehicles: Contactless Attacks against Sensors of Self-Driving Vehicle, 2016, Def Con 24 (Year: 2016). [cited by examiner]
Y. Zhang, et al.; “Perception and sensing for autonomous vehicles under adverse weather conditions: A survey”; ISPRS Journal of Photogrammetry and Remote Sensing, Jan. 9, 2023. [cited by applicant]
Z. Goldberg, et al.; “Jamming and Spoofing Attacks: Physical Layer Cybersecurity Threats to Autonomous Vehicle Systems”; Before the National Highway Traffic Safety Administration Washington, DC; Nov. 21, 2016. [cited by applicant]
P. Szoldra; “Hackers show how they tricked a Tesla into hitting objects in its path”; Insider, https://www.businessinsider.com/defcon-tesla-jamming-spoofing-autopilot-2016-8 Aug. 8, 2016. [cited by applicant]
J. Torchinsky, “Hackers Show that Tesla Autonomous Sensors Can Be Fooled, But It's All a Bit Stupid”; Jalopnik.com, https://www.forbes.com/sites/thomasbrewster/2016/08/04/tesla-autopilot-hack-crash/?sh=4ca803f62b87; Aug… [cited by applicant]
C. Yan, et al.; “Can You Trust Autonomous Vehicles: Contactless Attacks against Sensors of Self-Driving Vehicles”; Def Con 24 (2016). [cited by applicant]
K. Okokpujie, et al.; “Anomaly-Based Intrusion Detection for a Vehicle Can Bus: A Case for Hyundai Avante CN7”; Journal of Southwest Jiaotong University, vol. 56 No. 5, Oct. 2021. [cited by applicant]
M. Lichtman, et al.; “A Communications Jamming Taxonomy”; EEE Computer and Reliability Societies, Jan./Feb. 2016. [cited by applicant]
M. Pham, et al.; “A Survey on Security Attacks and Defense Techniques for Connected and Autonomous Vehicles”; https://arxiv.org/pdf/2007.08041.pdf, Jul. 16, 2020. [cited by applicant]
H. Shin, et al.; “Illusion and Dazzle: Adversarial Optical Channel Exploits against Lidars for Automotive Applications”; International Conference on Cryptographic Hardware and Embedded Systems, pp. 445 467, Springer, 20… [cited by applicant]
J. Petit, et al.; “Remote Attacks on Automated Vehicles Sensors: Experiments on Camera and LiDAR”; In Black Hat Europe, 2015. [cited by applicant]
J. Sun, et al.; “Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures”; In USENIX Security, 2020. [cited by applicant]
Y. Cao, et al.; “Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving”; https://arxiv.org/pdf/1907.06826.pdf, Aug. 20, 2019. [cited by applicant]
I. Sobh, et al.; “Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving”; Journal of Imaging Science and Technology, Nov.-Dec. 2021. [cited by applicant]
S. Thys, et al.; “Fooling automated surveillance cameras: adversarial patches to attack person detection”; Proc. CVPR Workshops (IEEE, Piscataway, NJ, 2019). [cited by applicant]
B. Tarchoun, et al.; “Adversarial Attacks in a Multi-view Setting: An Empirical Study of the Adversarial Patches Inter-view Transferability”; arXiv:2110.04887v1 [cs.CR] Oct. 10, 2021. [cited by applicant]