IP Library Granted Patent US 10,032,083
Granted Patent B2
US 10,032,083 · App. 14/964,995 · Granted Jul 24, 2018

Driver assistance for a vehicle

Inventors: Chuho Yi (Seoul, KR); Woonki Park (Seoul, KR); Yeonchool Park (Seoul, KR)
Assignee: LG Electronics Inc.
G06K9/00798G06K9/00791H04N13/0203H04N13/0271H04N13/0296
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,032,083
App. No.
14/964,995
Granted
Jul 24, 2018
Kind
B2
Abstract

Systems and techniques are disclosed for providing driver assistance in a vehicle. A driver assistance apparatus includes a stereo camera configured to acquire stereo images of a view ahead of a vehicle, and a processor configured to generate a depth map based on the acquired stereo images and to determine whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map.

Claims (160)

1. A driver assistance apparatus comprising:

a stereo camera configured to acquire stereo images of a view ahead of a vehicle;

a stereo camera drive unit configured to drive the stereo camera; and

at least one processor configured to:

generate a depth map based on the acquired stereo images; and

determine whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map,

wherein the at least one processor is configured to control the stereo camera drive unit based on a determination of whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment by:

determining that the road segment that is within the view ahead of the vehicle is an uphill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is an uphill road segment:

determining a travelling speed of the vehicle; and

controlling the stereo camera to be tilted upward at a speed that is proportional to the determined traveling speed of the vehicle,

wherein the at least one processor is further configured to, based on a determination that the road segment that is within the view ahead of the vehicle is an uphill road segment:

determine a tilt of the uphill road segment;

determine that a distance between the vehicle and an inflection point of the uphill road segment is at least a predetermined distance; and

based on a determination that the distance between the vehicle and the inflection point of the uphill road segment is at least the predetermined distance, control the stereo camera to be tilted upward according to the determined tilt of the uphill road segment.

2. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

detect a road surface based on the generated depth map; and

determine whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the detected road surface.

3. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

detect a lane from the stereo images;

determine a shape of the detected lane; and

determine whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the determined shape of the detected lane.

4. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

determine a disappearance point indicated in the depth map; and

determine whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the determined disappearance point indicated in the depth map.

5. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

detect a plurality of fixed objects around the road segment indicated in the generated depth map; and

determine whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the detected plurality of fixed objects around the road segment indicated in the generated depth map.

6. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

determine that the vehicle has entered the uphill road segment; and

based on a determination that the vehicle has entered the uphill road segment, control the stereo camera to return to a state in which the stereo camera is not tilted.

7. The driver assistance apparatus according to claim 6 , wherein the at least one processor is configured to:

determine a travelling speed of the vehicle; and

control, in proportion to the determined traveling speed of the vehicle and according to the determined tilt of the uphill road segment, a speed of the stereo camera to return to the state in which the stereo camera is not tilted.

8. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

determine that the road segment that is within the view ahead of the vehicle is a downhill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is a downhill road segment:

determine a tilt of the downhill road segment; and

control the stereo camera to be tilted downward according to the determined tilt of the downhill road segment.

9. The driver assistance apparatus according to claim 8 , wherein the at least one processor is configured to:

determine that a distance between the vehicle and an inflection point of the downhill road segment is at least a predetermined distance; and

based on a determination that the distance between the vehicle and an inflection point of the downhill road segment is at least a predetermined distance, control the stereo camera to be tilted downward according to the determined tilt of the downhill road segment.

10. The driver assistance apparatus according to claim 8 , wherein:

the at least one processor is configured to determine that the vehicle has entered the downhill road segment, and

the stereo camera drive unit is configured to, based on a determination that the vehicle has entered the downhill road segment, drive the stereo camera to return to a state in which the stereo camera is not tilted.

11. The driver assistance apparatus according to claim 10 , wherein the at least one processor is configured to:

determine a travelling speed of the vehicle; and

control, according to the determined tilt of the downhill road segment or in proportion to the determined traveling speed of the vehicle, the stereo camera to return to the state in which the stereo camera is not tilted.

12. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

determine that the road segment that is within the view ahead of the vehicle is a downhill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is a downhill road segment:

determine a travelling speed of the vehicle; and

control the stereo camera to be tilted downward at a speed that is proportional to the determined traveling speed of the vehicle.

13. The driver assistance apparatus according to claim 1 , wherein the at least one processor is further configured to:

detect lines in the acquired stereo images; and

generate, based on the detected lines, information regarding a curve of the road segment that is within the view ahead of the vehicle.

14. The driver assistance apparatus according to claim 1 , further comprising a display unit configured to output information regarding the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle.

15. The driver assistance apparatus according to claim 1 , further comprising:

a first memory, wherein

the at least one processor is further configured to cumulatively store, in the first memory, information regarding a state of the vehicle based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle.

16. A vehicle comprising:

a first memory;

a power source drive unit configured to control an output of an engine of the vehicle, wherein the output of the engine is configured to limit a speed of the vehicle;

a brake drive unit configured to drive a brake of the vehicle;

a stereo camera configured to acquire stereo images for a view ahead of the vehicle;

at least one processor configured to:

generate a depth map based on the acquired stereo images;

determine, based on the generated depth map, whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment;

generate road surface information based on the acquired stereo images; and

output a control signal configured to control the brake drive unit based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle and based on the generated road surface information; and

a slip determination unit configured to determine a slip degree of the vehicle,

wherein the at least one processor is further configured to output the control signal to control the brake drive unit of the vehicle based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle and based on the determined slip degree of the vehicle,

wherein the at least one processor is further configured to:

cumulatively store, in the first memory, information regarding the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle;

apply a Kalman filter to the cumulatively stored information regarding the determined uphill road segment or the determined downhill road segment;

estimate the slip degree of the vehicle based on an application of the Kalman filter to the cumulatively stored information regarding the determined uphill road segment or the determined downhill road segment; and

output the control signal to control, based on the estimated slip degree of the vehicle, the power source drive unit or the brake drive unit to drive the brake of the vehicle at a level equivalent to 90% or less than a full brake level.

17. The vehicle according to claim 16 , further comprising:

a navigation system configured to provide a map and information regarding a position of the vehicle on the map, wherein

the at least one processor is configured to:

receive, from the navigation system, information regarding the road segment on which the vehicle is traveling excluding a region of the road segment displayed in the stereo images; and

estimate, based on the received information regarding the road segment on which the vehicle is traveling excluding a region of the road segment displayed in the stereo images, a state of the road segment.

18. A driver assistance method comprising:

acquiring, by a stereo camera of a vehicle, stereo images of a view ahead of the vehicle;

generating a depth map based on the acquired stereo images;

determining whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map; and

controlling a stereo camera drive unit to drive the stereo camera, based on a determination of whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment,

wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

based on a determination that the road segment that is within the view ahead of the vehicle is an uphill road segment:

determining that a distance between the vehicle and an inflection point of the uphill road segment is at least a predetermined distance; and

based on a determination that the distance between the vehicle and the inflection point of the uphill road segment is at least the predetermined distance, driving the stereo camera to be tilted upward according to a tilt of the uphill road segment.

19. The driver assistance method according to claim 18 , wherein determining whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map comprises:

detecting a road surface from the generated depth map; and

determining whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the detected road surface.

20. The driver assistance method according to claim 18 , wherein determining whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map comprises:

detecting a lane from the stereo images;

determining a shape of the detected lane; and

determining whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the determined shape of the detected lane.

21. The driver assistance method according to claim 18 , wherein determining whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map comprises:

determining a disappearance point indicated in the depth map; and

determining whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the determined disappearance point indicated in the depth map.

22. The driver assistance method according to claim 18 , wherein determining whether a road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the generated depth map comprises:

detecting a plurality of fixed objects around the road segment indicated in the depth map; and

determining whether the road segment that is within the view ahead of the vehicle is an uphill road segment or a downhill road segment based on the detected plurality of fixed objects around the road segment indicated in the depth map.

23. The driver assistance method according to claim 18 , further comprising:

determining that the road segment that is within the view ahead of the vehicle is an uphill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is an uphill road segment, determining a tilt of the uphill road segment, wherein

controlling the stereo camera drive unit to drive the stereo camera comprises driving the stereo camera to be tilted upward according to the determined tilt of the uphill road segment.

24. The driver assistance method according to claim 23 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining that the vehicle has entered the uphill road segment; and

based on a determination that the vehicle has entered the uphill road segment, driving the stereo camera to return to a state in which the stereo camera is not tilted.

25. The driver assistance method according to claim 24 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining a travelling speed of the vehicle; and

driving, in proportion to the determined traveling speed of the vehicle and according to the determined tilt of the uphill road segment, the stereo camera to return to the state in which the stereo camera is not tilted.

26. The driver assistance method according to claim 18 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining that the road segment that is within the view ahead of the vehicle is an uphill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is an uphill road segment:

determining a travelling speed of the vehicle; and

driving the stereo camera to be tilted upward at a speed that is proportional to the determined traveling speed of the vehicle.

27. The driver assistance method according to claim 18 , further comprising:

determining that the road segment that is within the view ahead of the vehicle is a downhill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is a downhill road segment, determining a tilt of the downhill road segment, wherein

controlling the stereo camera drive unit to drive the stereo camera comprises driving the stereo camera to be tilted downward according to the determined tilt of the downhill road segment.

28. The driver assistance method according to claim 27 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining that a distance between the vehicle and an inflection point of the downhill road segment is at least a predetermined distance; and

based on a determination that the distance between the vehicle and an inflection point of the downhill road segment is at least a predetermined distance, driving the stereo camera to be tilted downward according to the determined tilt of the downhill road segment.

29. The driver assistance method according to claim 27 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining that the vehicle has entered the downhill road segment; and

based on a determination that the vehicle has entered the downhill road segment, driving the stereo camera to return to a state in which the stereo camera is not tilted.

30. The driver assistance method according to claim 29 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining a travelling speed of the vehicle; and

driving, according to the determined tilt of the downhill road segment or in proportion to the determined traveling speed of the vehicle, the stereo camera to return to the state in which the stereo camera is not tilted.

31. The driver assistance method according to claim 18 , wherein controlling the stereo camera drive unit to drive the stereo camera comprises:

determining that the road segment that is within the view ahead of the vehicle is a downhill road segment; and

based on a determination that the road segment that is within the view ahead of the vehicle is a downhill road segment:

determining a travelling speed of the vehicle; and

driving the stereo camera to be tilted downward at a speed that is proportional to the determined traveling speed of the vehicle.

32. The driver assistance method according to claim 18 , further comprising:

detecting lines in the acquired stereo images; and

generating, based on the detected lines, information regarding a curve of the road segment that is within the view ahead of the vehicle.

33. The driver assistance method according to claim 18 , further comprising outputting, to a display unit, information regarding the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle.

34. The driver assistance method according to claim 18 , further comprising cumulatively storing, in a first memory, information regarding a state of the vehicle based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle.

35. The driver assistance method according to claim 18 , further comprising:

controlling a brake drive unit to drive a brake of the vehicle;

generating road surface information based on the acquired stereo images; and

outputting a control signal configured to control the brake drive unit based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle and based on the generated road surface information.

36. The driver assistance method according to claim 35 , further comprising:

receiving, from a navigation system that is configured to provide a map and information regarding a position of the vehicle on the map, information regarding the road segment on which the vehicle is traveling excluding a region of the road segment displayed in the stereo images from a navigation system; and

estimating, based on the received information regarding the road segment on which the vehicle is traveling excluding a region of the road segment displayed in the stereo images, a state of the road segment.

37. The driver assistance method according to claim 35 , further comprising:

determining a slip degree of the vehicle, wherein

outputting a control signal configured to control the brake drive unit based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle and based on the generated road surface information comprises:

outputting the control signal to control the brake drive unit of the vehicle based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle and based on the determined slip degree of the vehicle.

38. The driver assistance method according to claim 37 , further comprising:

controlling an output of an engine of the vehicle through a power source drive unit of the vehicle, wherein the output of the engine is configured to limit a speed of the vehicle;

cumulatively storing, in a first memory, information regarding the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle;

applying a Kalman filter to the cumulatively stored information regarding the determined uphill road segment or the determined downhill road segment; and

estimating the slip degree of the vehicle based on an application of the Kalman filter to the cumulatively stored information regarding the determined uphill road segment or the determined downhill road segment, wherein

outputting a control signal configured to control the brake drive unit based on the determined uphill road segment or the determined downhill road segment that is within the view ahead of the vehicle and based on the generated road surface information comprises:

outputting the control signal to control, based on the estimated slip degree of the vehicle, the power source drive unit or the brake drive unit to drive the brake of the vehicle at a level equivalent to 90% or less than a full brake level.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 26, 2018
From: YI, CHUHO; PARK, WOONKI; PARK, YEONCHOOL
To: LG ELECTRONICS INC.
Reel/Frame 046202/0842 →
Priority Claims (1)
KR 10-2014-0177580 · Dec 10, 2014 · national
Continuity (1)
Related Publication 20160171315A1 · Jun 16, 2016
Cited By (1)
US 12,585,276