IP Library Granted Patent US 10,031,624
Granted Patent B2
US 10,031,624 · App. 15/368,741 · Granted Jul 24, 2018

System and method for monitoring 3D space in front of an output unit for the control of the output unit

Inventors: Alexander Barth (Wuppertal, DE); Piotr Bando (Cracow, PL); Roland Hollemann (Seesen, DE)
Assignee: Delphi Technologies, Inc.
G06F3/0425G06F3/011G06F3/017G06F3/0416G06K9/00375H04N13/0203
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,031,624
App. No.
15/368,741
Granted
Jul 24, 2018
Kind
B2
Abstract

A system for monitoring 3D space in front of an output unit for the control of the output unit comprises a 3D imaging unit, a processing and control unit coupled to the 3D imaging unit and an output unit comprising a display area and coupled to the processing and control unit. The 3D imaging unit is configured to monitor an interaction zone in front of the display area of the output unit. The processing and control unit is configured to predict where a finger of a hand that approaches the display area of the output unit and is monitored by the 3D imaging unit will touch the display area and to control the output unit to modify content displayed on the display area based on the predicted touch position.

Claims (26)

1. A system for monitoring 3D space in front of an output unit for the control of the output unit, said system comprising:

a 3D imaging unit;

a processing and control unit coupled to the 3D imaging unit; and

an output unit comprising a display area, said output unit coupled to the processing and control unit, wherein

the 3D imaging unit is configured to monitor an interaction zone in front of the display area of the output unit,

the processing and control unit is configured to predict where a finger of a hand that approaches the display area of the output unit and is monitored by the three-dimensional imaging unit (3D imaging unit) will touch the display area, and to control the output unit to modify content displayed on the display area based on the predicted touch position,

the processing and control unit is configured to localize a finger-tip and a finger base of the finger and to fit a line through the finger-tip and the finger base,

the processing and control unit is configured to compute an intersection point of the line through the finger-tip and the finger base with the display area,

the processing and control unit comprises a two-dimensional map (2D map) representing the display area and comprising a plurality of cells, and the processing and control unit is configured to compute the intersection point of the line with the display area in consecutive time steps, wherein for each of the time steps at least one of the cells is identified corresponding to the computed intersection point and a numeric value assigned to the at least one identified cell is incremented, and

the processing and control unit is configured to find peaks in the 2D map for each of the time steps in order to identify the predicted touch position of the finger.

2. A system as claimed in claim 1 , wherein the output unit is configured to display a plurality of elements on the display area, wherein at least one of the elements is highlighted and/or enlarged if the processing and control unit predicts that the finger of the hand will touch the at least one of the elements.

3. A system as claimed in claim 1 , wherein the processing and control unit is configured to identify a hand with an exposed index finger in an image received from the 3D imaging unit.

4. A system as claimed in claim 1 , wherein the processing and control unit is configured to compute an intersection line of a plane through the line and a camera origin of the 3D imaging unit with the display area, wherein, the predicted touch position of the finger lies on the intersection line ( 59 ).

5. A system as claimed in claim 1 , wherein the processing and control unit is configured to compute a distance between the finger-tip and the predicted touch position, a velocity of the finger-tip, and the time when the finger-tip will touch the display area.

6. A system as claimed in claim 1 , wherein the processing and control unit is configured to increment numeric values assigned to cells located within a certain distance around the at least one identified cell for each of the time steps, wherein the distance depends on an uncertainty of the computed intersection point.

7. A vehicle comprising a system as claimed claim 1 , wherein the output unit comprises a human machine interface.

8. A method for monitoring three-dimensional space (3D space) in front of an output unit for the control of the output unit, said method comprising:

monitoring an interaction zone in front of a display area of the output unit;

predicting where a finger of a hand that approaches the display area of the output unit and is monitored will touch the display area; and

controlling the output unit to modify content displayed on the display area based on the predicted touch position, wherein

a finger-tip and a finger base of the finger are localized and a line is fitted through the finger-tip and the finger base,

an intersection point of the line through the finger-tip and the finger base with the display area is computed,

a two-dimensional map (2D map) representing the display area and comprising a plurality of cells is provided, and the intersection point of the line with the display area is computed in consecutive time steps, wherein for each of the time steps at least one of the cells is identified corresponding to the computed intersection point and a numeric value assigned to the at least one identified cell is incremented, and

peaks are identified in the 2D map for each of the time steps in order to identify the predicted touch position of the finger.

9. The method as claimed in claim 8 , wherein the output unit displays a plurality of elements on the display area and at least one of the elements is highlighted and/or enlarged if it is predicted that the finger of the hand, will touch the at least one of the elements.

10. The method as claimed in claim 8 , wherein an intersection line of a plane through the line and a camera origin of the 3D imaging unit with the display area is computed, wherein, in particular, the predicted touch position of the finger lies on the intersection line.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2024
From: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
To: APTIV TECHNOLOGIES AG
Reel/Frame 066551/0219 →
MERGER Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES (2) S.À R.L.
To: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
Reel/Frame 066566/0173 →
ENTITY CONVERSION Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES LIMITED
To: APTIV TECHNOLOGIES (2) S.À R.L.
Reel/Frame 066746/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2018
From: DELPHI TECHNOLOGIES INC.
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 047143/0874 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2016
From: BARTH, ALEXANDER; BANDO, PIOTR; HOLLERMANN, ROLAND
To: DELPHI TECHNOLOGIES, INC.
Reel/Frame 040514/0949 →
Priority Claims (1)
EP 15201193 · Dec 18, 2015 · regional
Continuity (1)
Related Publication 20170192617A1 · Jul 6, 2017