IP Library Granted Patent US 11,244,564
Granted Patent B2
US 11,244,564 · App. 15/878,512 · Granted Feb 8, 2022

Vehicle acoustic-based emergency vehicle detection

Inventor: Heinz B. Seifert (Ortonville, MI)
Assignee: MAGNA ELECTRONICS INC.
G08G1/0965B60Q9/00B60W30/08G06K9/00791G06K9/00805G06K9/209G06K9/78H04R1/406H04R3/005B60W2420/42B60W2420/54B60W2554/00B60W2710/18B60W2710/20H04R2499/13
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,244,564
App. No.
15/878,512
Filed
Jan 24, 2018
Granted
Feb 8, 2022
Kind
B2
Art Unit
3661
USPC
701/41
Abstract

An emergency vehicle detection system for a vehicle includes a plurality of camera assemblies having a camera and at least one microphone. At least one processor processes image data captured by the cameras, and processes outputs of the microphones. The at least one processor processes the outputs of the microphones to detect a sound source indicative of a siren emanating from an emergency vehicle exterior of the equipped vehicle, and to determine the direction of the detected sound source. Responsive to detection of the sound source and determination of the direction of the detected sound source, the at least one processor processes image data captured by at least one of the cameras that has its field of view encompassing the detected sound source to confirm that the detected sound source is a siren emanating from an emergency vehicle.

Claims (50)

1. An emergency vehicle detection system for a vehicle, said emergency vehicle detection system comprising:

a plurality of camera modules disposed at a vehicle equipped with said emergency vehicle detection system, wherein said camera modules are spaced apart, one from another, at the equipped vehicle;

wherein each camera module of said plurality of camera modules comprises a camera having a respective field of view exterior of the equipped vehicle, each said camera capturing image data;

wherein each of said camera modules comprises at least one microphone for sensing sounds emanating from exterior of the equipped vehicle;

wherein each of said camera modules comprises circuitry that is shared by said camera and by said at least one microphone of the respective camera module;

at least one processor disposed at the equipped vehicle, wherein said at least one processor is operable to process image data captured by said cameras, and wherein said at least one processor is operable to process outputs of said microphones;

wherein said at least one processor processes the outputs of said microphones to detect a sound source indicative of a siren emanating from an emergency vehicle exterior of the equipped vehicle;

wherein said at least one processor processes the outputs of said microphones to determine (i) the direction of the detected sound source indicative of the siren of the emergency vehicle from the equipped vehicle and (ii) whether the detected sound source is approaching an intersection ahead of the equipped vehicle;

wherein, responsive to detection of the sound source and determination of the direction of the detected sound source from the equipped vehicle, said at least one processor processes image data captured by at least one of said cameras that has its field of view encompassing the detected sound source to subsequently confirm that the detected sound source is a siren emanating from an emergency vehicle; and

wherein, responsive to confirmation, via image processing by said image processor of image data captured by the at least one of said cameras, that the detected sound source is a siren emanating from an emergency vehicle, said emergency vehicle detection system at least one of (i) generates an alert to a driver of the equipped vehicle and (ii) controls braking or steering of the equipped vehicle.

2. The emergency vehicle detection system of claim 1 , wherein said at least one processor processes the outputs of said microphones to determine whether the detected sound source is approaching the equipped vehicle or moving away from the equipped vehicle.

3. The emergency vehicle detection system of claim 1 , wherein, responsive to confirmation that the detected sound source is a siren emanating from an emergency vehicle, said emergency vehicle detection system generates an alert to a driver of the equipped vehicle.

4. The emergency vehicle detection system of claim 1 , wherein, responsive to confirmation that the detected sound source is a siren emanating from an emergency vehicle, said emergency vehicle detection system controls braking or steering of the equipped vehicle.

5. The emergency vehicle detection system of claim 1 , wherein, responsive to confirmation that the detected sound source is a siren emanating from an emergency vehicle, said emergency vehicle detection system (i) generates an alert to a driver of the equipped vehicle and (ii) controls braking and steering of the equipped vehicle.

6. The emergency vehicle detection system of claim 1 , wherein said plurality of camera modules comprises at least three spaced apart camera modules disposed at the exterior of the equipped vehicle.

7. The emergency vehicle detection system of claim 1 , wherein said plurality of camera modules comprises at least a driver-side camera module at a driver-side exterior mirror assembly of the equipped vehicle, and a passenger-side camera module at a passenger-side exterior mirror assembly of the equipped vehicle.

8. The emergency vehicle detection system of claim 7 , further comprising a center microphone at a shark-fin antenna of the equipped vehicle.

9. The emergency vehicle detection system of claim 7 , further comprising a center microphone at a center portion of the equipped vehicle.

10. The emergency vehicle detection system of claim 1 , wherein each of said camera modules comprises a plurality of microphones.

11. The emergency vehicle detection system of claim 1 , wherein, responsive to detection of the sound source indicative of a siren emanating from an emergency vehicle and the determined direction of the detected sound source, a sensing system of the vehicle processes data captured by at least one non-imaging sensor disposed at the vehicle and having a field of sensing that encompasses the detected sound source to confirm that the detected sound source is a siren emanating from an emergency vehicle.

12. The emergency vehicle detection system of claim 11 , wherein the sensing system comprises a sensing system selected from the group consisting of (i) an ultrasonic sensing system, (ii) a LIDAR sensing system and (iii) a RADAR sensing system.

13. An emergency vehicle detection system for a vehicle, said emergency vehicle detection system comprising:

a plurality of camera modules disposed at a vehicle equipped with said emergency vehicle detection system, wherein said camera modules are spaced apart, one from another, at the equipped vehicle;

wherein said plurality of camera modules comprises at least three spaced apart camera modules disposed at the exterior of the equipped vehicle;

wherein said plurality of camera modules comprises at least a driver-side camera module at a driver side of the equipped vehicle, and a passenger-side camera module at a passenger side of the equipped vehicle;

wherein each camera module of said plurality of camera modules comprises a camera having a respective field of view exterior of the equipped vehicle, each said camera capturing image data;

wherein each of said camera modules comprises at least one microphone for sensing sounds emanating from exterior of the equipped vehicle;

wherein each of said camera modules comprises circuitry that is shared by said camera and by said at least one microphone of the respective camera module;

at least one processor disposed at the equipped vehicle, wherein said at least one processor is operable to process image data captured by said cameras, and wherein said at least one processor is operable to process outputs of said microphones;

wherein said at least one processor processes the outputs of said microphones to detect a sound source indicative of a siren emanating from an emergency vehicle exterior of the equipped vehicle;

wherein said at least one processor processes the outputs of said microphones to determine (i) the direction of the detected sound source indicative of the siren of the emergency vehicle from the equipped vehicle and (ii) whether the detected sound source is approaching an intersection ahead of the equipped vehicle;

wherein, responsive to detection of the sound source and determination of the direction of the detected sound source, said at least one processor processes image data captured by at least one of said cameras that has its field of view encompassing the detected sound source to subsequently confirm that the detected sound source is a siren emanating from an emergency vehicle;

wherein, responsive to confirmation that the detected sound source is a siren emanating from an emergency vehicle, said emergency vehicle detection system at least one of (i) generates an alert to a driver of the equipped vehicle and (ii) controls braking or steering of the equipped vehicle; and

wherein said at least one processor processes the outputs of said microphones to determine whether the detected sound source is approaching the equipped vehicle or moving away from the equipped vehicle.

14. The emergency vehicle detection system of claim 13 , further comprising a center microphone at a center portion of the equipped vehicle.

15. The emergency vehicle detection system of claim 13 , wherein each of said camera modules comprises a plurality of microphones.

16. An emergency vehicle detection system for an autonomous vehicle, said emergency vehicle detection system comprising:

a plurality of camera modules disposed at a vehicle equipped with said emergency vehicle detection system, wherein said camera modules are spaced apart, one from another, at the equipped vehicle;

wherein said plurality of camera modules comprises at least (i) a first camera module disposed at a first side of the equipped vehicle and (ii) a second camera module disposed at a second side of the equipped vehicle;

wherein each camera module of said plurality of camera modules comprises a camera having a respective field of view exterior of the equipped vehicle, each said camera capturing image data;

wherein each of said camera modules comprises at least one microphone for sensing sounds emanating from exterior of the equipped vehicle;

wherein each of said camera modules comprises circuitry that is shared by said camera and by said at least one microphone of the respective camera module;

at least one other microphone disposed at a center portion of the equipped vehicle;

at least one processor disposed at the equipped vehicle, wherein said at least one processor is operable to process image data captured by said cameras, and wherein said at least one processor is operable to process outputs of said microphones of said camera modules and said at least one other microphone;

wherein said at least one processor processes the outputs of said microphones of said camera modules and said at least one other microphone to detect a sound source indicative of a siren emanating from an emergency vehicle exterior of the equipped vehicle;

wherein said at least one processor processes the outputs of said microphones of said camera modules and said at least one other microphone to determine (i) the direction of the detected sound source indicative of the siren of the emergency vehicle from the equipped vehicle and (ii) whether the detected sound source is approaching an intersection ahead of the equipped vehicle;

wherein, responsive to detection of the sound source and determination of the direction of the detected sound source, said at least one processor processes image data captured by at least one of said cameras that has its field of view encompassing the detected sound source to subsequently confirm that the detected sound source is a siren emanating from an emergency vehicle; and

wherein, responsive to confirmation that the detected sound source is a siren emanating from an emergency vehicle, said emergency vehicle detection system controls braking or steering of the equipped vehicle.

17. The emergency vehicle detection system of claim 16 , wherein said at least one processor processes the outputs of said microphones of said camera modules and said at least one other microphone to determine whether the detected sound source is approaching the equipped vehicle or moving away from the equipped vehicle.

18. The emergency vehicle detection system of claim 16 , wherein each of said camera modules comprises a plurality of microphones.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 14, 2018
From: SEIFERT, HEINZ B.
To: MAGNA ELECTRONICS INC.
Reel/Frame 046878/0005 →
Continuity (2)
Provisional Application 62450680 · Jan 26, 2017
Related Publication 20180211528A1 · Jul 26, 2018
Cited By (2)
US 12,223,832 US 12,597,344