IP Library › Granted Patent US 11,671,857
Granted Patent B2
US 11,671,857 · App. 17/110,972 · Granted Jun 6, 2023

Roadside communication system for monitoring and maintaining sensor data transmission

Inventor: Kuen Yehliu (Ann Arbor, MI)
Assignee: Mitsubishi Electric Corporation
H04W24/08G05D1/021G05D2201/0213G08G1/09H04W84/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,671,857
App. No.
17/110,972
Granted
Jun 6, 2023
Kind
B2
Abstract

Provided is a processing system to effectuate sensor processing of obstacle information for use in autonomous driving. The apparatus includes a communication module that receives sensor data collected by multiple sensors disposed to capture sensor data in a specified region to detect the obstacles. Additionally, a health status detection modules, in the system, determine the health status of the system and the sensors based on information from the sensors and a communication latency between the communication module and the sensors. The system also includes a sensor processing module to process the sensor data to identify anomalies in the data due to harsh conditions to eliminate any erroneous data.

Claims (42)

1. A first processing apparatus comprising:

at least one hardware processor configured to implement;

a communication module that receives sensor data collected by multiple sensors disposed to capture sensor data in a specified region to detect obstacles;

a health status detection module that determines a health status of the first processing apparatus and the sensors based on information from the sensors and a communication status between the communication module and the sensors,

where the first processing apparatus communicates, to a second processing apparatus, the determined health status and processed sensor data based on the detected health status of the sensors.

2. The first processing apparatus according to claim 1 , wherein health status detection module determines the communication status with each sensor based on a message transmitted from the first processing apparatus to a corresponding sensor and by measuring the latency time between the transmitted message and a return message received from the corresponding sensor.

3. The first processing apparatus according to claim 2 , wherein the latency time is measured by including a first timestamp indicating when the message was transmitted to the corresponding sensor, and measuring the difference in time between the first timestamp and a second timestamp indicating when the message was received back from the corresponding sensor.

4. The first processing apparatus according to claim 2 , wherein the latency time is measured by including a first timestamp indicating when the message was transmitted to the corresponding sensor, and measuring the difference in time between the first timestamp and a second timestamp indicating when the message is retransmitted by the corresponding sensor.

5. The first processing apparatus according to claim 1 , wherein the at least one processor comprises a first central processing portion including a first main processing core and a first checker processing core, and a second central processing portion including a second main processing core and a second checker core,

wherein data is processed on both the first main processing core and the first checker processing core and processing from each of the first main processing core and the first checker core are compared by the health status detection module, and

in response to the health status detection module determining that the comparison results are not consistent, the first main processing core is disabled and the data processing is changed to be processed by the second main processing core and the second checker core.

6. The first processing apparatus according to claim 5 , wherein when the data processing is initially implemented on the second main processing core and the second checker core, the health status detection module data communicates the processing change to the second processing apparatus.

7. The first processing apparatus according to claim 1 further comprising:

a sensor processing and calibration module that processes sensor data and performs sensor calibration based on the processed sensor data,

wherein the sensor processing and calibration module processes sensor data at a frequency lower than a normal frequency of operation and determines whether the low frequency processed data indicates a harsh condition representing external impacts on the sensors prevent useful data, and

in response to determining the harsh condition exists, the sensor processing and calibration module requests that the sensor be self-calibrated to adjust sensor characteristics at the sensor.

8. The first processing apparatus according to claim 7 , wherein in response to the sensor processing and calibration module determining that the low frequency data provides useful data, the sensor processing and calibration module determines whether sensor data obtained from the sensors exhibits anomalies representing harsh conditions exist for each sensor, and

in response to anomalies being detected in the sensor data from a corresponding sensor, the sensor processing and calibration module instructs the sensor processing and calibration module to perform post-processing on the sensor data received from the corresponding sensor.

9. The first processing apparatus according to claim 8 , wherein the post-processing detects portions of the sensor data from a corresponding sensors data that correspond to anomalies to identify anomalous portions, and transmits the other portions of the sensor data to the second processing apparatus, or transmits the sensor data with annotated portions identifying the portions of the sensor data that correspond to the anomalies to the second processing apparatus.

10. A processing system for processing sensor data comprising:

a first processing apparatus having at least one processor configured to implement:

a first communication module that receives sensor data collected by multiple sensors disposed to capture sensor data in a specified region to detect obstacles;

a first health status detection module that determines a health status of the first processing apparatus and the sensors based on information from the sensors and a communication status between the communication module and the sensors;

a sensor processing and calibration module that processes sensor data and performs sensor calibration based on the processed sensor data based on the determined health status; and

a second processing apparatus having at least one processor configured to implement:

a second communication module that receives the health status and the sensor data processed by the sensor processing and calibration module;

a second health status detection module that determines a health status of the second processing apparatus based the health status of the first information processing apparatus and a communication status between the second communication module and the first communication module; and

a detection response module that processes the sensor data processed by the sensor processing and calibration module based on the health status of the second processing apparatus.

11. The processing system for processing sensor data according to claim 10 , wherein first health status detection module determines the communication status with each sensor based on a message transmitted from the first processing apparatus to a corresponding sensor and measuring the latency time between the transmitted message and a return message received from the corresponding sensor, and

the second health status detection module determines the communication status with the first processing apparatus based on a message transmitted from the second processing apparatus to the first processing apparatus and by measuring the latency time between the transmitted message and a return message received from the first processing apparatus.

12. The processing system for processing sensor data according to claim 11 , wherein the latency time is measured by including a first timestamp indicating when the message was transmitted to the corresponding sensor, and measuring the difference in time between the first timestamp and a second timestamp indicating when the message was received back from the corresponding sensor.

13. The processing system for processing sensor data according to claim 11 , wherein the latency time is measured by including a first timestamp indicating when the message was transmitted to the first processing apparatus, and measuring the difference in time between the first timestamp and a second timestamp indicating when the message is retransmitted by the first processing apparatus.

14. The processing system for processing sensor data according to claim 10 , wherein the at least one processor of the second processing apparatus comprises a first central processing portion including a first main processing core and a first checker processing core, and a second central processing portion including a second main processing core and a second checker core,

wherein data is processed on both the first main processing core and the first checker processing core and processing from each of the first main processing core and the first checker core are compared by the second health status detection module, and

in response to the second health status detection module determining that the comparison results are not consistent, the first main processing core is disabled and the data processing is changed to be processed the second main processing core and the second checker core.

15. The processing system according to claim 10 , the first processing apparatus further comprising:

a sensor processing and calibration module that processes sensor data and performs sensor calibration based on the processed sensor data,

wherein the sensor processing and calibration module processes sensor data at a frequency lower than a normal frequency of operation and determines whether the low frequency processed data indicates a harsh condition representing external impacts on the sensors prevent useful data, and

in response to determining the harsh condition exists, the sensor processing and calibration module requests that the sensor be self-calibrated to adjust sensor characteristics at the sensor.

16. The processing system according to claim 15 , wherein in response to the sensor processing and calibration module determining that the low frequency data provides useful data, the sensor processing and calibration module determines whether sensor data obtained from the sensors exhibits anomalies representing harsh conditions exist for each sensor, and

in response to anomalies being detected in the sensor data from a corresponding sensor, the sensor processing and calibration module instructs the sensor processing and calibration module to perform post-processing on the sensor data received from the corresponding sensor.

17. The processing system according to claim 16 , wherein the post-processing detects portions of the sensor data from a corresponding sensors data that corresponds to anomalies to identify anomalous portions, and transmits the other portions of the sensor data to the second processing apparatus, or transmits the sensor data with annotated portions identifying the portions of the sensor data that correspond to the anomalies to the second processing apparatus.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED ON REEL 056062 FRAME 0088. ASSIGNOR(S) HEREBY CONFIRMS THE EXECUTED ASSIGNMENT. Recorded Apr 29, 2021
From: YEHLIU, KUEN
To: MITSUBISHI ELECTRIC AUTOMOTIVE AMERICA, INC.
Reel/Frame 056097/0887 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 1, 2021
From: MITSUBISHI ELECTRIC AUTOMOTIVE AMERICA, INC.
To: MITSUBISHI ELECTRIC CORPORATION
Reel/Frame 055795/0590 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 054688 FRAME: 0911. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 22, 2021
From: YEHLIU, KUEN
To: MITSUBISHI ELECTRIC AUTOMOTIVE AMERICAN, INC.
Reel/Frame 056062/0088 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2020
From: YEHLIU, KUEN
To: MITSUBISHI ELECTRIC AUTOMOTIVE AMERICA, INC.
Reel/Frame 054688/0911 →
Continuity (1)
Related Publication 20220182855A1 · Jun 9, 2022