IP Library Granted Patent US 10,083,361
Granted Patent B2
US 10,083,361 · App. 14/852,704 · Granted Sep 25, 2018

Path sensing using structured lighting

Inventor: Gregory Gerhard Schamp (South Lyon, MI)
Assignee: Joyson Safety Systems Acquisition LLC
G06K9/00825B60G17/019B60K31/0008B60R1/00G01B11/2513G01B11/2545G06K9/00791G06K9/00805G06K9/2036B60G2400/82B60Q2400/50B60R2300/10B60W2520/16B60W2550/141B60W2550/147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,083,361
App. No.
14/852,704
Granted
Sep 25, 2018
Kind
B2
Abstract

A structured light pattern is projected onto the path of a vehicle so as to generate a plurality of light spots, and an image thereof is captured from the vehicle. A world-space elevation of at least a portion of the light spots is responsive to a pitch angle of the vehicle determined responsive to image-space locations of down-range-separated light spots.

Claims (65)

1. A method of sensing a physical feature of or along a path of a vehicle, comprising:

a. projecting a structured light pattern from a vehicle onto a path upon which said vehicle may travel, so as to generate a plurality of light spots on said path, wherein at least two of said plurality of light spots are at different world-space down-range locations relative to said vehicle;

b. capturing from said vehicle at least one image of said plurality of light spots using at least one camera;

c. determining at least two image-space locations in said at least one image of a corresponding said at least two of said plurality of lights spots at said different world space down-range locations;

d. determining a pitch angle of said vehicle responsive to said at least two of said plurality of light spots;

e. determining corresponding world-space elevations of at least a portion of said plurality of light spots responsive to said pitch angle of said vehicle; and

f. determining said pitch angle of said vehicle responsive to at least one image-space separation of at least one pair of said plurality of light spots.

2. A method of sensing a physical feature of or along a path of a vehicle, comprising:

projecting a structured light pattern from a vehicle onto a path upon which said vehicle may travel, so as to generate a plurality of light spots on said path, wherein at least two of said plurality of light spots are at different world-space down-range locations relative to said vehicle;

capturing from said vehicle at least one image of said plurality of light spots using at least one camera;

determining at least two image-space locations in said at least one image of a corresponding said at least two of said plurality of lights spots at said different world space down-range locations;

determining a pitch angle of said vehicle responsive to said at least two of said plurality of light spots; and

determining corresponding world-space elevations of at least a portion of said plurality of light spots responsive to said pitch angle of said vehicle; and

transforming said at least one image to a plurality of elevation tiles, wherein each elevation tile of said plurality of elevation tiles comprises a plurality of corners that are defined by corresponding image-space locations of associated light spots of said plurality of light spots, and a world-space elevation of each said elevation tile is determined as a median or average of corresponding world-space elevation coordinates of a set of image pixels associated with said elevation tile within a boundary defined by said plurality of corners.

3. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 2 , further comprising selecting as a pitch/roll analysis patch a subset of elevation tiles associated with a region along said path that is relatively close in down-range to said vehicle and substantially laterally centered with respect to said vehicle.

4. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 3 , further comprising determining said pitch angle of said vehicle responsive to at least one curve fit of world-space elevation values with respect to world-space down-range values of a subset of said subset of said elevation tiles associated with a substantially common cross-range location with respect to said vehicle, wherein said subset of said subset of said elevation tiles are associated with said pitch/roll analysis patch.

5. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 3 , further comprising determining a roll angle of said vehicle responsive to at least one curve fit of world-space elevation values with respect to world-space cross-range values of a subset of said subset of said elevation tiles associated with a substantially common down-range location with respect to said vehicle, wherein said subset of said subset of said elevation tiles are associated with said pitch/roll analysis patch.

6. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 2 , further comprising:

a. selecting at least one subset of said elevation tiles along at least one prospective tire track of at least one tire of said vehicle, and

b. communicating a plurality of measures responsive to at least one world-space elevation profile with respect to travel distance along a corresponding said at least one subset of said elevation tiles so as to provide for controlling a suspension system of said vehicle responsive thereto.

7. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 6 , wherein said plurality of measures comprises at least one differential elevation profile associated with at least one corresponding said at least one prospective tire track, and each value of said at least one differential elevation profile is responsive to a difference of world-space elevation values of different elevation tiles of said at least one subset of said elevation tiles along said at least one corresponding said at least one prospective tire track.

8. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 6 , wherein each value of said at least one world-space elevation profile is responsive to a weighted average of world space elevation values of a plurality of said elevation tiles for different cross-range locations within said at least one prospective tire track at a substantially common said travel distance along said at least one prospective tire track.

9. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 6 , further comprising predicting a corresponding at least one location of said at least one prospective tire track responsive to measures of yaw rate and velocity of said vehicle.

10. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 1 , wherein each light spot of said at least one pair of said plurality of light spots are at said different world-space down-range locations and are at a substantially common world-space cross-range location.

11. A method of sensing a physical feature of or along a path of a vehicle, comprising:

projecting a structured light pattern from a vehicle onto a path upon which said vehicle may travel, so as to generate a plurality of light spots on said path, wherein at least two of said plurality of light spots are at different world-space down-range locations relative to said vehicle;

capturing from said vehicle at least one image of said plurality of light spots using at least one camera;

determining at least two image-space locations in said at least one image of a corresponding said at least two of said plurality of lights spots at said different world space down-range locations;

determining a pitch angle of said vehicle responsive to said at least two of said plurality of light spots; and

determining corresponding world-space elevations of at least a portion of said plurality of light spots responsive to said pitch angle of said vehicle; and

wherein said at least one image is captured using a stereo-vision imaging system that provides for generating first and second stereo image components, each from a different perspective along a baseline separated by a baseline distance, further comprising:

a. determining a corresponding down-range location of each light spot of said plurality of light spots responsive to a disparity of image-space locations of said corresponding light spot in said first and second stereo image components, and

b. determining a corresponding world-space elevation of said light spot responsive to said corresponding down-range location of said light spot.

12. A method of sensing a physical feature of or along a path of a vehicle, comprising:

projecting a structured light pattern from a vehicle onto a path upon which said vehicle may travel, so as to generate a plurality of light spots on said path, wherein at least two of said plurality of light spots are at different world-space down-range locations relative to said vehicle;

capturing from said vehicle at least one image of said plurality of light spots using at least one camera;

determining at least two image-space locations in said at least one image of a corresponding said at least two of said plurality of lights spots at said different world space down-range locations;

determining a pitch angle of said vehicle responsive to said at least two of said plurality of light spots; and

determining corresponding world-space elevations of at least a portion of said plurality of light spots responsive to said pitch angle of said vehicle; and

wherein said at least one image is captured using a mono-vision imaging system, and for each light spot of said plurality of light spots, further comprising:

a. associating said light spot with a stored pre-calibrated world-space down-range location of said light spot, wherein said pre-calibrated world-space down-range location of said light spot is determined when said vehicle is on a nominal relatively flat surface, and

b. determining a world-space elevation of said light spot responsive to said pre-calibrated world-space down-range location of said light spot.

13. A method of sensing a physical feature of or along a path of a vehicle, comprising:

projecting a structured light pattern from a vehicle onto a path upon which said vehicle may travel, so as to generate a plurality of light spots on said path, wherein at least two of said plurality of light spots are at different world-space down-range locations relative to said vehicle;

capturing from said vehicle at least one image of said plurality of light spots using at least one camera;

determining at least two image-space locations in said at least one image of a corresponding said at least two of said plurality of lights spots at said different world space down-range locations;

determining a pitch angle of said vehicle responsive to said at least two of said plurality of light spots; and

determining corresponding world-space elevations of at least a portion of said plurality of light spots responsive to said pitch angle of said vehicle; and

further comprising, for each light spot of said at least said portion of said plurality of light spots:

a. locating a plurality of pixels associated with said light spot in said at least one image; and

b. determining a location of said light spot in said at least one image responsive to said plurality of pixels associated with said light spot in said at least one image.

14. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 13 , wherein the operation of locating said plurality of pixels associated with said light spot in said at least one image for each light spot of said at least said portion of said plurality of light spots comprises searching a predetermined region-of-interest in said at least one image associated with said light spot.

15. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 14 , wherein a location of said predetermined region-of-interest in said at least one image is modified responsive to said pitch angle.

16. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 13 , wherein the operation of determining said location of said light spot in said at least one image comprises successively calculating centroids of each of a plurality of subsets of said plurality of pixels associated with said light spot in said at least one image so as to generate a single centroid that is representative of said location of said light spot in said at least one image.

17. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 13 , wherein the operation of determining said location of said light spot in said at least one image comprises:

a. using an adaptive gradient method to determine an edge profile of said light spot from said plurality of pixels associated with said light spot in said at least one image; and

b. calculating said location of said light spot in said at least one image responsive to said edge profile.

18. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 17 , wherein the operation of calculating said location of said light spot in said at least one image comprises calculating either a centroid of said edge profile, an average pixel location of said edge profile, or a median pixel location of said edge profile.

19. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 17 , further comprising:

a. determining a best-fit ellipse associated with said edge profile; and

b. determining at least one of said pitch angle or a roll angle of said vehicle responsive to an orientation of said best-fit ellipse in said at least one image.

20. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 13 , wherein the operation of determining said location of said light spot in said at least one image comprises successively binning said plurality of pixels associated with said light spot in said at least one image so as to generate a single binned pixel that is representative of said location of said light spot in said at least one image.

21. A method of sensing a physical feature of or along a path of a vehicle as recited in claim 13 , wherein the operation of determining said location of said light spot in said at least one image comprises:

a. locating a plurality of edge points of said light spot, wherein each edge point of said plurality of edge points is located responsive to a Savitzky-Golay filtering process of said plurality of pixels along a corresponding polar direction relative to a nominal center of said plurality of pixels, and

b. determining said location of said light spot in said at least one image from a centroid of said plurality of edge points.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded May 8, 2026
From: DEUTSCHE BANK TRUST COMPANY AMERICAS
To: JOYSON SAFETY SYSTEMS ACQUISITION LLC
Reel/Frame 075554/0717 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT Recorded Jan 13, 2026
From: JOYSON SAFETY SYSTEMS ACQUISITION LLC; JOYSON SAFETY SYSTEMS JAPAN G.K.; KEY SAFETY SYSTEMS, INC.; JOYSON SAFETY SYSTEMS GERMANY GMBH
To: DEUTSCHE BANK TRUST COMPANY AMERICAS
Reel/Frame 074331/0585 →
RELEASE OF SECURITY INTEREST Recorded Oct 12, 2021
From: DEUTSCHE BANK TRUST COMPANY AMERICAS, AS SECURITY AGENT FOR THE SECURED PARTIES
To: JOYSON SAFETY SYSTEMS ACQUISITION LLC
Reel/Frame 057775/0726 →
SECURITY INTEREST Recorded Oct 12, 2021
From: JOYSON SAFETY SYSTEMS ACQUISITION LLC
To: DEUTSCHE BANK TRUST COMPANY AMERICAS, AS SECURITY AGENT FOR THE SECURED PARTIES
Reel/Frame 057828/0411 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2018
From: TK HOLDINGS INC.
To: JOYSON SAFETY SYSTEMS ACQUISITION LLC
Reel/Frame 046173/0180 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT Recorded Apr 17, 2018
From: JOYSON SAFETY SYSTEMS ACQUISITION LLC
To: DEUTSCHE BANK TRUST COMPANY AMERICAS
Reel/Frame 045959/0305 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2015
From: SCHAMP, GREGORY GERHARD
To: TK HOLDINGS INC.
Reel/Frame 036692/0156 →
Continuity (3)
Continuation PCTUS2014027376 · Mar 14, 2014
Provisional Application 61799376 · Mar 15, 2013
Related Publication 20160034771A1 · Feb 4, 2016
Cited By (2)
US 12,317,389 US 12,637,027