IP Library Granted Patent US 11,531,339
Granted Patent B2
US 11,531,339 · App. 16/791,870 · Granted Dec 20, 2022

Monitoring of drive by wire sensors in vehicles

Inventors: Robert Richard Noel Bielby (Placerville, CA); Poorna Kale (Folsom, CA)
Assignee: Micron Technology, Inc.
G05D1/0088B60W10/18B60W10/20G05D1/0221G06N3/08G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,531,339
App. No.
16/791,870
Granted
Dec 20, 2022
Kind
B2
Abstract

The subject application relates to monitoring of drive by wire sensors in vehicles. Systems, methods and apparatuses of predictive maintenance of drive by wire systems of vehicles are provided. For example, a data storage device is configured in a vehicle to receive and store sensor data from at least one drive by wire system of the vehicle. During a period in which the vehicle is assumed to be operating normally, the sensor data stored in the data storage device is used to train an artificial neural network to recognize the normal patterns in the sensor data. Subsequently, using the trained artificial neural network, the data storage device determines whether operations of the at least one drive by wire system are abnormal based on the sensor data. A maintenance alert can be generated for the vehicle in response to a determination that the operations of the at least one drive by wire system are abnormal according to the artificial neural network.

Claims (48)

1. A method, comprising:

receiving, in a data storage device configured in a vehicle, sensor data from at least one drive by wire system of the vehicle;

determining operating parameters of the vehicle associated with the sensor data of the at least one drive by wire system;

determining, by the data storage device using an artificial neural network and based on the sensor data of the at least one drive by wire system and the operating parameters, whether operations of the at least one drive by wire system are abnormal, wherein the operations of the at least one drive by wire system are determined to be abnormal based on the sensor data deviating from a recognized pattern associated with the sensor data, wherein the artificial neural network is trained to identify the recognized pattern during a period of time in which a probability of the operations being normal is above a threshold; and

generating a maintenance alert for the vehicle in response to a determination that the operations of the at least one drive by wire system are abnormal according to the artificial neural network.

2. The method of claim 1 , wherein the at least one drive by wire system includes a steering by wire system, a brake by wire system, or a throttle by wire system, or any combination therein.

3. The method of claim 2 , wherein the artificial neural network includes a spiking neural network.

4. The method of claim 2 , wherein the sensor data includes input data generate by at least one drive by wire sensor of the at least one drive by wire system.

5. The method of claim 4 , wherein the sensor data further includes control signals generated from the input data by at least one control unit in the at least one drive by wire system.

6. The method of claim 5 , wherein the sensor data is received in the data storage device from the at least one control unit.

7. The method of claim 6 , wherein the at least one control unit includes a steer control unit, a brake control unit, or a throttle control unit, or any combination therein.

8. The method of claim 7 , further comprising:

storing the sensor data in an input region of the data storage device; and

in response to the artificial neural network classifying the sensor data of the at least one drive by wire system and the operating parameters as abnormal, storing the sensor data in an output region of the data storage device.

9. The method of claim 8 , further comprising:

checking, by a computer system of the vehicle, the output region of the data storage device for presence of data classified as abnormal by the artificial neural network; and

presenting by the computer system of the vehicle, an alert for a maintenance visit in response to detecting the presence of the data classified as abnormal by the artificial neural network.

10. The method of claim 9 , further comprising:

transmitting, to a server from the output region of the data storage device and via a wireless connection from the vehicle, the data classified as abnormal by the artificial neural network, wherein the transmitted data is analyzed in the server to generate a diagnosis result; and

training the artificial neural network in the data storage device to predict the diagnosis result based on the data stored in the output region and classified as abnormal by the artificial neural network.

11. The method of claim 8 , wherein sensor data received from the at least one drive by wire system is stored in the input region of the data storage device cyclically, where an oldest portion of sensor data stored in the input region is erased to store an incoming portion of sensor data from the at least one drive by wire system.

12. The method of claim 1 , wherein the operating parameters include an environmental temperature of the autonomous vehicle, a speed of the autonomous vehicle, a location of the autonomous vehicle, an input from a sensor configured on the autonomous vehicle, a status of a vehicle control, a status of an infotainment system of the autonomous vehicle, or a status of an advanced driver assistance system of the autonomous vehicle, or any combination thereof.

13. The method of claim 1 , further comprising:

storing sensor data received from the at least one drive by wire system in an input region of the data storage device cyclically;

configuring the data storage device in a first mode;

training the artificial neural network to classify sensor data stored in the input region as normal in the first mode;

configuring the data storage device in a second mode; and

determining, in the second mode and using the artificial neural network trained in the first mode, whether sensor data stored in the input region is normal or abnormal.

14. A data storage device configured in a vehicle, the data storage device comprising:

a host interface configured to receive sensor data from at least one drive by wire system of the vehicle;

one or more memory components configured to store the sensor data in an input region cyclically;

a controller configured to determine operating parameters of the vehicle associated with the sensor data of the at least one drive by wire system; and

an inference engine configured to determine, using an artificial neural network and based on the sensor data of the at least one drive by wire system and the operating parameters, whether operations of the at least one drive by wire system are abnormal, wherein the operations of the at least one drive by wire system are determined to be abnormal based on the sensor data deviating from a recognized pattern associated with the sensor data, wherein the artificial neural network is trained to identify the recognized pattern during a period of time in which a probability of the operations being normal is above a threshold, wherein a maintenance alert is generated for the vehicle in response to a determination that the operations of the at least one drive by wire system are abnormal according to the artificial neural network.

15. The data storage device of claim 14 , wherein the artificial neural network includes a spiking neural network.

16. The data storage device of claim 15 , wherein the controller is configured to copy abnormal sensor data from the input region to an output region in the one or more memory components.

17. The data storage device of claim 15 , wherein the inference engine includes a neural network accelerator configured to perform matrix arithmetic computations more efficiently than the controller.

18. A vehicle, comprising:

at least one drive by wire system, the drive by wire system having:

a sensor configured to measure user interactions with a control element to generate input signals; and

a control unit configured to generate control signals to drive the vehicle based on the input signals that are transmitted electronically from the sensor to the control unit;

a computer system configured to provide operating parameters of the vehicle; and

a data storage device having:

a communication interface configured to receive sensor data from at least one drive by wire system of the vehicle;

one or more memory components configured to store the sensor data; and

an inference engine configured to determine, using an artificial neural network and based on the sensor data of the at least one drive by wire system and the operating parameters, whether operations of the at least one drive by wire system are abnormal, wherein the operations of the at least one drive by wire system are determined to be abnormal based on the sensor data deviating from a recognized pattern associated with the sensor data, wherein the artificial neural network is trained to identify the recognized pattern during a period of time in which a probability of the operations being normal is above a threshold;

wherein the computer system is configured to generate a maintenance alert in response to a determination that the operations of the at least one drive by wire system are abnormal according to the artificial neural network.

19. The vehicle of claim 18 , wherein the artificial neural network includes a spiking neural network; and the data storage device is further configured to generate training data and train the spiking neural network to determine whether the operations of the at least one drive by wire system are abnormal.

20. The vehicle of claim 19 , wherein the drive by wire system is a steering by wire system, a brake by wire system, or a throttle by wire system.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2020
From: BIELBY, ROBERT RICHARD NOEL; KALE, POORNA
To: MICRON TECHNOLOGY, INC.
Reel/Frame 051935/0697 →
Continuity (1)
Related Publication 20210255620A1 · Aug 19, 2021
Cited By (6)
US 12,210,401 US 12,249,189 US 12,443,387 US 12,497,055 US 12,518,570 US 12,559,135