IP Library Granted Patent US 10,730,450
Granted Patent B2
US 10,730,450 · App. 16/533,531 · Granted Aug 4, 2020

Road sign announcer

Inventors: Katherine Wilbert (Livonia, MI); Maria Rohrer (Bloomfield, MI); Garrett Laskowski (Fenton, MI); John Qiu (Northville, MI); Padmanaban Kanagaraj (Farmington Hills, MI)
Assignee: Panasonic Automotive Systems Company of America, Division of Panasonic Corporation of North America
B60R11/04B60Q5/005G06K9/00818G06K9/3258G10L13/00G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,730,450
App. No.
16/533,531
Granted
Aug 4, 2020
Kind
B2
Abstract

A vehicle includes a camera capturing an image of a road sign associated with a road that the vehicle is traveling on. A first processing device is communicatively coupled to the camera and recognizes text characters in the captured image. A second processing device is communicatively coupled to the first processing device and converts the recognized text characters into an electronic speech signal. A loudspeaker is communicatively coupled to the second processing device and converts the electronic speech signal into audible speech that can be heard by the driver of the vehicle.

Claims (47)

1. A vehicle, comprising:

a geographical positioning system configured to determine a geographical location of the vehicle;

a processing device communicatively coupled to the geographical positioning system and configured to:

wirelessly transmit the geographical location of the vehicle to an external database;

wirelessly receive from the external database a textual message associated with a road sign that is visible to a driver of the vehicle at the geographical location of the vehicle; and

wirelessly receive from the external database a literal word-for-word recitation of the textual message on the visible road sign;

a text-to-speech module communicatively coupled to the processing device and configured to convert the textual message from the external database into an electronic speech signal;

a loudspeaker communicatively coupled to the text-to-speech module and configured to convert the electronic speech signal into audible speech that can be heard by the driver of the vehicle;

a camera configured to capture an image of the road sign; and

a text recognition module communicatively coupled to the camera and configured to recognize text characters in the captured image, wherein the processing device is communicatively coupled to the text recognition module and is configured to produce a confidence score based on a comparison of the recognized text characters in the captured image and the literal word-for-word recitation from the database.

2. The vehicle of claim 1 wherein the confidence score is indicative of a likelihood that the literal word-for-word recitation from the database is associated with the road sign in the image captured by the camera.

3. The vehicle of claim 1 wherein the text-to-speech module is configured to convert the textual message from the external database into an electronic speech signal only if the confidence score exceeds a threshold.

4. The vehicle of claim 3 wherein the text-to-speech module is configured to convert the recognized text characters in the captured image into an electronic speech signal only if the confidence score is less than the threshold.

5. The vehicle of claim 1 further comprising a second processing device configured to recognize a shape of the road sign in the captured image, the electronic speech signal being dependent upon the recognized shape of the road sign in the captured image.

6. The vehicle of claim 1 further comprising a second processing device configured to recognize a color of the road sign in the captured image, the electronic speech signal being dependent upon the recognized color of the road sign in the captured image.

7. The vehicle of claim 6 wherein the second processing device is configured to interpret a meaning of the color of the road sign.

8. The vehicle of claim 1 further comprising a second processing device configured to recognize a symbol within the road sign in the captured image, the electronic speech signal being dependent upon the recognized symbol within the road sign in the captured image.

9. The vehicle of claim 8 wherein the second processing device is configured to interpret a meaning of the symbol within the road sign.

10. The vehicle of claim 1 wherein the camera comprises a first camera directed to a right-hand shoulder of the road, the vehicle further comprising a second camera directed to an area above and in front of the vehicle.

11. A vehicle, comprising:

a camera configured to capture an image of a road sign associated with a road that the vehicle is traveling on;

a text recognition module communicatively coupled to the camera and configured to recognize text characters in the captured image;

a geographical positioning system configured to determine a geographical location of the vehicle;

a processing device communicatively coupled to the text recognition module and the geographical positioning system and configured to:

wirelessly transmit the geographical location of the vehicle to an external database;

wirelessly receive from the external database a textual message associated with a road sign that is visible to a driver of the vehicle at the geographical location of the vehicle; and

produce a confidence score indicative of a likelihood that the textual message from the database is associated with the road sign in the image captured by the camera;

a text-to-speech module communicatively coupled to the processing device and configured to convert the textual message from the external database into an electronic speech signal only if the confidence score exceeds a threshold; and

a loudspeaker communicatively coupled to the text-to-speech module and configured to convert the electronic speech signal into audible speech that can be heard by the driver of the vehicle.

12. The vehicle of claim 11 wherein the text-to-speech module is configured to convert the recognized text characters in the captured image into an electronic speech signal only if the confidence score is less than a threshold.

13. The vehicle of claim 11 wherein the confidence score is based on a comparison of the recognized text characters in the captured image to the textual message from the external database.

14. The vehicle of claim 11 wherein the processing device is configured to wirelessly receive from the external database both the textual message and a literal word-for-word recitation of the textual message on the visible road sign.

15. The vehicle of claim 14 wherein the confidence score is based on a comparison of the recognized text characters in the captured image to the literal word-for-word recitation of the textual message from the external database.

16. A method of operating a motor vehicle, comprising:

determining a geographical location of the motor vehicle;

wirelessly transmitting the geographical location of the vehicle to an external database;

wirelessly receiving from the external database a textual message associated with a road sign that is visible to a driver of the vehicle at the geographical location of the vehicle;

wirelessly receiving from the external database a literal word-for-word recitation of the textual message on the visible road sign;

converting the textual message from the external database into an electronic speech signal;

converting the electronic speech signal into audible speech that can be heard by the driver of the vehicle:

capturing an image of the road sign;

recognizing text characters in the captured image; and

producing a confidence score based on a comparison of the recognized text characters in the captured image and the literal word-for-word recitation from the database.

17. The method of claim 16 wherein the confidence score is indicative of a likelihood that the literal word-for-word recitation from the database is associated with the road sign in the captured image.

18. The method of claim 16 the textual message is converted from the external database into an electronic speech signal only if the confidence score exceeds a threshold.

19. The method of claim 18 wherein the recognized text characters in the captured image are converted into an electronic speech signal only if the confidence score is less than the threshold.

20. The method of claim 16 further comprising recognizing a shape of the road sign in the captured image, the electronic speech signal being dependent upon the recognized shape of the road sign in the captured image.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE SUPPORTIVE DOCUMENTS PREVIOUSLY RECORDED ON REEL 72222 FRAME 267. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Sep 23, 2025
From: PANASONIC CORPORATION OF NORTH AMERICA
To: PANASONIC AUTOMOTIVE SYSTEMS AMERICA, LLC.
Reel/Frame 072930/0871 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 31, 2025
From: PANASONIC CORPORATION OF NORTH AMERICA
To: PANASONIC AUTOMOTIVE SYSTEMS AMERICA, LLC
Reel/Frame 072222/0267 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 6, 2019
From: WILBERT, KATHERINE; ROHRER, MARIA; LASKOWSKI, GARRETT; QIU, JOHN; KANAGARAJ, PADMANABAN
To: PANASONIC AUTOMOTIVE SYSTEMS COMPANY OF AMERICA, DIVISION OF PANASONIC CORPORATION OF NORTH AMERICA
Reel/Frame 049980/0065 →
Continuity (3)
Continuation 15409824 · Jan 19, 2017
Provisional Application 62280436 · Jan 19, 2016
Related Publication 20190359148A1 · Nov 28, 2019