IP Library Granted Patent US 11,292,700
Granted Patent B2
US 11,292,700 · App. 16/500,547 · Granted Apr 5, 2022

Driver assistance system and a method

Inventors: Hans Lyngbäck (Hudiksvall, SE); Per Gustafsson (Hudiksvall, SE); Marcus Rösth (Hudiksvall, SE)
Assignee: Hiab AB
B66C13/46B66C15/06G06T19/006B66C2700/08G06T2200/08G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,292,700
App. No.
16/500,547
Granted
Apr 5, 2022
Kind
B2
Abstract

A driver assistance system ( 2 ) for a vehicle ( 4 ) provided with a crane ( 6 ) for loading or unloading objects ( 8 ), the system comprises: a control unit ( 10 ) configured to generate a presentation signal ( 12 ) to be applied to a display unit ( 14 ), an object detecting device ( 16 ) configured to capture images of the vehicle and/or of the environment around the vehicle, and configured to generate an image data signal ( 18 ) comprising data representing captured images, and to apply said signal ( 18 ) to said control unit ( 10 ). The control unit ( 10 ) is configured to: —determine a three dimensional, 3D, representation of at least one object, being a part of the vehicle and/or being at least one object ( 8 ) in the environment around the vehicle, based upon images captured by said object detecting device ( 16 ), —determine a 3D representation of the environment around the vehicle, —determine a predetermined alert condition measure related to said determined 3D representations, wherein if said alert condition measure fulfils at least one of a set of predetermined alert criteria, the control unit is configured initiate at least one responsive activity of a set of predetermined responsive activities.

Claims (33)

1. A driver assistance system for a vehicle provided with a crane for loading or unloading objects, the system comprises:

a control unit configured to generate a presentation signal to be applied to a display unit,

an object detecting device configured to capture images of the vehicle and/or of the environment around the vehicle, and configured to generate an image data signal comprising data representing captured images, and to apply said signal to said control unit, and that the control unit is configured to:

determine a three dimensional, 3D, representation of at least one object, being a part of the vehicle and/or being at least one object in the environment around the vehicle, based upon images captured by said object detecting device,

determine a 3D representation of the environment around the vehicle, and

determine a predetermined alert condition measure related to said determined 3D representations, wherein if said alert condition measure fulfils at least one of a set of predetermined alert criteria, the control unit is configured to initiate at least one responsive activity of a set of predetermined responsive activities, and

wherein said set of responsive activities comprises that the control unit is configured to determine a dynamic augmented reality, AR, image, including highlighted portions, of at least a part of said determined 3D representations, wherein the control unit is configured to determine, generate and include presentation instructions in said presentation signal, and wherein said presentation instructions comprises to present said AR image on said display unit such that said at least a part of said 3D representations in the AR image overlays, corresponds and adapts to the object(s) and environment visible at the display unit;

wherein said presentation instructions comprises image locking parameters that match, relate and lock at least one object in the AR image to the corresponding at least one object visible at the display unit;

wherein said presentation instructions includes adapting the AR image to crane movements and/or vehicle movements.

2. The driver assistance system according to claim 1 , wherein the control unit is configured to determine an alert value related to an object in said presented AR image and being a part of said presentation instructions, and wherein the alert value is determined to a value ranging from a low alert value to a high alert value, and wherein said object in the AR image is highlighted in dependence of said determined alert value, such that the highlighting is more intense the higher the alert value is.

3. The driver assistance system according to claim 1 , wherein said set of responsive activities comprises that the control unit is configured to determine and generate control signals for controlling movements of the crane and/or the vehicle.

4. The driver assistance system according to claim 3 , wherein said control unit is configured to determine control signals for controlling movements of the crane and/or vehicle such that said alert condition measure no longer fulfils any of said predetermined alert criteria.

5. The driver assistance system according to claim 1 , wherein said set of responsive activities comprises that the control unit is configured to determine and generate control signals for controlling movements of the crane, and wherein said control signals include a stop signal command to stop the movement of the crane.

6. The driver assistance system according to claim 1 , wherein said set of responsive activities comprises that the control unit is configured to determine and generate alert signals being one or many of an audible, visible, and tactile signal.

7. The driver assistance system according to claim 1 , wherein said predetermined alert condition measure is related to a distance between a part of said crane and/or vehicle and an object, and wherein said set of predetermined alert criteria includes different distances between the part of said crane and/or vehicle, and the object.

8. The driver assistance system according to claim 1 , wherein said control unit is configured to determine said 3D representation of the environment around the vehicle based upon images captured by said object detecting device, and/or based upon image information received from an external source.

9. The driver assistance system according to claim 1 , wherein said object detecting device is a camera system comprising two cameras that have essentially overlapping field of views.

10. The driver assistance system according to claim 1 , wherein said display unit is a pair of glasses structured to present the AR image such that the AR image is overlaid on the transparent glasses through which a user observes the object and/or the environment.

11. The driver assistance system according to claim 1 , wherein said display unit is a pair of virtual reality goggles.

12. A method in a driver assistance system for a vehicle provided with a crane for loading or unloading objects, the method comprises:

generating a presentation signal, by a control unit,

applying the presentation signal to a display unit,

capturing images, by an object detecting device, of the vehicle and/or of the environment around the vehicle,

generating an image data signal comprising data representing captured images,

applying said signal to said control unit,

determining a three dimensional, 3D, representation of at least one object, being a part of the vehicle and/or being at least one object in the environment around the vehicle, based upon images captured by said object detecting device,

determining a 3D representation of the environment around the vehicle,

determining a predetermined alert condition measure related to said determined 3D representations, wherein if said alert condition measure fulfils at least one of a set of predetermined alert criteria, initiating at least one responsive activity of a set of predetermined responsive activities, and wherein said set of responsive activities comprises that the method further comprises:

determining a dynamic augmented reality, AR, image, including highlighted portions, of at least a part of said determined 3D representations,

determining, generating and including presentation instructions in said presentation signal, and wherein said presentation instructions comprises presenting said AR image on said display unit such that said at least a part of said 3D representations in the AR image overlays, corresponds and adapts to the object(s) and environment visible at the display unit;

wherein said presentation instructions comprises image locking parameters that match, relate and lock at least one object in the AR image to the corresponding at least one object visible at the display unit;

wherein said presentation instructions includes adapting the AR image to crane movements and/or vehicle movements.

13. The method according to claim 12 , wherein said set of responsive activities comprises that the method comprises determining and generating control signals for controlling movements of the crane and/or the vehicle.

Assignments (3)
CHANGE OF ADDRESS Recorded Jul 10, 2024
From: HIAB AB
To: HIAB AB
Reel/Frame 068262/0119 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: CARGOTEC PATENTER AB
To: HIAB AB
Reel/Frame 056095/0843 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2019
From: LYNGBACK, HANS; GUSTAFSSON, PER; ROSTH, MARCUS
To: CARGOTEC PATENTER AB
Reel/Frame 050649/0254 →
Priority Claims (1)
SE 1750410-1 · Apr 3, 2017 · national
Continuity (1)
Related Publication 20200048052A1 · Feb 13, 2020
Cited By (1)
US 12,227,145