IP Library › Granted Patent US 11,340,624
Granted Patent B2
US 11,340,624 · App. 16/555,362 · Granted May 24, 2022

Vehicle neural network

Inventors: Francois Charette (Tracy, CA); Jose Enrique Solomon (Belmont, CA)
Assignee: Ford Global Technologies, LLC
G05D1/0221G06F17/15G06N3/08G06V20/58G06V20/584G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,340,624
App. No.
16/555,362
Granted
May 24, 2022
Kind
B2
Abstract

A computer, including a processor and a memory, the memory including instructions to be executed by the processor to generate a first color image of a road environment, determine one or more value decompositions of one or more of the red, green, and blue channels of the first color image, obtain one or more modified singular value decompositions by modifying respective ones of the singular value decompositions by a non-linear equation and reconstruct a second color image based on the modified one or more singular value decompositions. The instructions can include further instructions to train a deep neural network based on the second color image and operate a vehicle based on the deep neural network.

Claims (33)

1. A computer, comprising a processor; and

a memory, the memory including instructions to be executed by the processor to:

generate a first color image of a road environment;

determine one or more singular value decompositions of one or more of a red, green, and blue channel of the first color image;

obtain one or more modified singular value decompositions by modifying respective ones of the singular value decompositions by a non-linear equation corresponding to modified lighting conditions;

reconstruct a second color image based on the modified one or more singular value decompositions;

train a deep neural network based on the second color image; and

operate a vehicle based on the deep neural network.

2. The computer of claim 1 , the instructions including further instructions to generate the first color image by one or more of acquiring vehicle sensor data and generating synthetic image data.

3. The computer of claim 1 , wherein the modified lighting conditions correspond to darker lighting conditions including overcast lighting, evening or night.

4. The computer of claim 1 , the instructions including further instructions to determine the one or more singular value decompositions including with a matrix equation X=UΣV T , wherein X is an array that includes one of a red, green, or blue color channel of the first color image, U and V are left and right eigenvectors of array X, and Σ is a diagonal matrix of singular values.

5. The computer of claim 1 , the instructions including further instructions to modify the one or more singular value decompositions based on a non-linear equation S={s−n for s≥S, else 0}, wherein s is a singular value, n is an integer and S is a set of singular values.

6. The computer of claim 1 , wherein the deep neural network is convolutional neural network.

7. The computer of claim 1 , the instructions including further instructions to train the deep neural network to identify objects in the road environment including a roadway, another vehicle, and a pedestrian.

8. The computer of claim 1 , the instructions including further instructions to operate the vehicle based on the deep neural network by inputting vehicle sensor data to the deep neural network which outputs data regarding objects in the road environment.

9. The computer of claim 8 , the instructions including further instructions to operate the vehicle based on determining a vehicle path based on the data regarding objects in the road environment.

10. The computer of claim 9 , the instructions including further instructions to operate the vehicle along the vehicle path by controlling vehicle powertrain, steering and brakes.

11. A method, comprising:

generating a first color image of a road environment;

determining one or more singular value decompositions of one or more of a red, green, and blue channel of the first color image;

obtaining one or more of modified singular value decompositions by modifying respective ones of the singular value decompositions by a non-linear equation corresponding to modified lighting conditions;

reconstructing a second color image based on the modified one or more singular value decompositions;

training a deep neural network based on the second color image; and

operating a vehicle based on the deep neural network.

12. The method of claim 11 , further comprising generating the first color image by one or more of acquiring vehicle sensor data and generating synthetic image data.

13. The method of claim 11 , wherein the modified lighting conditions correspond to darker lighting conditions including overcast lighting, evening or night.

14. The method of claim 11 , further comprising determining one or more singular value decompositions including with a matrix equation X=UΣV T , wherein X is an array that includes one of a red, green, or blue color channel of the first color image, U and V are left and right eigenvectors of an array X, and Σ is a diagonal matrix of singular values.

15. The method of claim 11 , further comprising modifying the one or more singular value decompositions based on a non-linear equation S={s−n for s≥S, else 0}, wherein s is a singular value, n is an integer and S is a set of singular values.

16. The method of claim 11 , wherein the deep neural network is convolutional neural network.

17. The method of claim 11 , further comprising training the deep neural network to identify objects in the road environment including a roadway, another vehicle, and a pedestrian.

18. The method of claim 11 , further comprising operating the vehicle based on the deep neural network by inputting vehicle sensor data to the deep neural network which outputs object data regarding objects in the road environment.

19. The method of claim 18 , further comprising determining a vehicle path based on the object data regarding objects in the road environment.

20. The method of claim 19 , further comprising operating the vehicle along the vehicle path by controlling vehicle powertrain, steering and brakes.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2019
From: CHARETTE, FRANCOIS; SOLOMON, JOSE ENRIQUE
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 050213/0505 →
Continuity (1)
Related Publication 20210064047A1 · Mar 4, 2021
Cited By (1)
US 12,256,179