IP Library Granted Patent US 10,890,521
Granted Patent B2
US 10,890,521 · App. 16/008,326 · Granted Jan 12, 2021

Method of tracking a plurality of point-shaped objects in a three-dimensional space

Inventors: Bernhard Wieneke (Goettingen, DE); Andreas Schröder (Goettingen, DE); Arne Henning (Goettingen, DE); Matteo Novara (Goettingen, DE); Daniel Schanz (Goettingen, DE)
Assignee: LAVISION GMBH
G01N15/1463G01B11/002G01N15/1429G01N15/1456G01N15/1459G01N15/1475G01P5/001G01P5/20G06K9/6203G06T7/70G01N2015/144G01N2021/1791G01N2201/0696G01N2201/0697G01N2201/0698G06T7/292G06T2207/10052G06T2207/10152G06T2207/30241H04N5/04H04N5/2354H04N5/247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,890,521
App. No.
16/008,326
Granted
Jan 12, 2021
Kind
B2
Abstract

For tracking a plurality of objects in a three-dimensional space two-dimensional pictures objects are recorded with two black and white cameras out of two different imaging directions. Both first pictures and second pictures of the two cameras are simultaneously exposed at two points in time in equal ways, a point in time at which the second pictures are exposed for a first time following to a point in time at which the first pictures are exposed for a last time at a much shorter interval than the two points in time of exposure of both the first and second pictures. First and second distributions of real positions of the individual objects are determined from their images in the first and second pictures, respectively; and temporally resolved trajectories of the individual objects in the three-dimensional space are determined from the first and second distributions of real positions.

Claims (59)

1. A method of tracking a plurality of point-shaped objects in a three-dimensional space, the method comprising

recording two-dimensional pictures of the plurality of objects by means of at least two black and white cameras out of at least two different imaging directions;

wherein at least first and second pictures of the at least two black and white cameras are each exposed at at least two points in time in equal ways in that the objects are illuminated with short light pulses at the at least two points in time,

wherein the first pictures are exposed simultaneously for a first time at a first point in time of a first light pulse of the short light pulses and for a last time at a second point in time of a second light pulse of the short light pulses following to the first point in time of the first light pulse of the short light pulses at a first inner-picture interval of time,

wherein the second pictures are exposed simultaneously for a first time at a third point in time of a third light pulse of the short light pulses and for a last time at a fourth point in time of a fourth light pulse of the short light pulses following to the third point in time of the third light pulse of the short light pulses at a second inner-picture interval of time, and

wherein the third point in time of the third light pulse of the short light pulses follows to the second point in time of the second light pulse of the short light pulses at an inner-picture interval of time of not more than 50% of the first inner-picture interval of time and of not more than 50% of the second inner-picture interval of time;

determining a first distribution and a second distribution of real positions of the individual objects of the plurality of objects in the three-dimensional space from their images in the first pictures and from their images in the second pictures, respectively,

wherein at least the first or the second distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space is determined from the images of the individual objects of the plurality of objects in the first or second pictures, respectively, in that an estimated distribution of the real positions of the individual objects in the three-dimensional space is adapted to their images in the first or second pictures, respectively; and

determining temporally resolved trajectories of the individual objects of the plurality of objects in the three-dimensional space from the determined first and second distributions of the real positions of the individual objects,

wherein different real positions of the same individual objects of the plurality of objects in the first distribution are interrelated,

wherein different real positions of the same individual objects of the plurality of objects in the second distribution are interrelated, and

wherein different real positions of the same individual objects of the plurality of objects in the first and second distributions are interrelated.

2. The method of claim 1 , wherein the inter-picture interval of time at which the third point in time follows to the second point in time is not more than 20% of the first inner-picture interval of time and of not more than 20% of the second inner-picture interval of time.

3. The method of claim 2 , wherein the inter-picture interval of time at which the third point in time follows to the second point in time is not more than 10% of the first inner-picture interval of time and of not more than 10% of the second inner-picture interval of time.

4. The method of claim 1 , wherein the different real positions of the same individual objects of the plurality of objects in the first distribution and in the second distribution are interrelated between the first and second distributions, before the different real positions of the same individual objects of the plurality of objects in the first distribution are interrelated within the first distribution and the different real positions of the same individual objects of the plurality of objects in the second distribution are interrelated within the second distribution.

5. The method of claim 1 , wherein third pictures of the at least two black and white cameras are each exposed at at least two points in time in equal ways, wherein the third pictures are exposed simultaneously for a first time at a fifth point in time and for a last time at a sixth point in time following to the fifth point in time at a third inner-picture interval of time, wherein the fifth point in time follows to the fourth point in time at the inter-picture interval of time of not more than 50% of the first inner-picture interval of time, of not more than 50% of the second inner-picture interval of time and of not more than 50% of the third inner-picture interval of time.

6. The method of claim 1 , wherein at least one of

velocities of the individual objects of the plurality of objects, and

accelerations of the individual objects of the plurality of objects are determined from the temporally resolved trajectories of the individual objects of the plurality of objects.

7. The method of claim 1 , wherein at least one of

a velocity field of a flow with which the individual objects of the plurality of objects move,

an acceleration field of a flow with which the individual objects of the plurality of objects move, and

a pressure field of a flow with which the individual objects of the plurality of objects move

are determined from the temporally resolved trajectories of the individual objects of the plurality of objects.

8. The method of claim 1 , wherein the pictures of the black and white cameras are exposed at the at least two points in time in that the objects of the plurality of objects are illuminated by light pulses.

9. The method of claim 1 , wherein the estimated distribution of the real positions of the individual objects in the three-dimensional space is an object distribution of all of the individual objects in the three-dimensional space for which trajectories in the three-dimensional space have been determined before.

10. The method of claim 1 , wherein the estimated distribution of the real positions of the individual objects in the three-dimensional space is estimated by extrapolating a starting distribution of the individual objects in the three-dimensional space along the trajectories of the individual objects in the three-dimensional space for the at least two points in time at which the respective first or second pictures are exposed.

11. The method of claim 10 , wherein the starting distribution is

a distribution which has been determined from images of the individual objects of the plurality of objects in other pictures of the black and white cameras which have been exposed at least two other points in time, or

a momentary distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space which has been determined from images of the individual objects of the plurality of objects in other pictures of the black and white cameras which have been exposed at one other point in time.

12. The method of claim 1 , wherein an auto-correlation function of at least the first or the second distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space is calculated to interrelate the different real positions of the same individual objects of the plurality of objects in the first or the second distribution, respectively.

13. The method of claim 1 , wherein a cross-correlation function of the first and the second distributions of the real positions of the individual objects of the plurality of objects in the three-dimensional space is calculated to interrelate the different real positions of the same individual objects of the plurality of objects in the first and second distributions.

14. A method of tracking a plurality of point-shaped objects in a three-dimensional space, the method comprising

recording two-dimensional pictures of the plurality of objects by means of at least two black and white cameras out of at least two different imaging directions;

wherein at least first and second pictures of the at least two black and white cameras are each exposed at at least two points in time in equal ways in that the objects are illuminated with short light pulses at the at least two points in time,

wherein the first pictures are exposed simultaneously for a first time at a first point in time of a first light pulse of the short light pulses and for a last time at a second point in time of a second light pulse of the short light pulses following to the first point in time of the first light pulse of the short light pulses at a first inner-picture interval of time,

wherein the second pictures are exposed simultaneously for a first time at a third point in time of a third light pulse of the short light pulses and for a last time at a fourth point in time of a fourth light pulse of the short light pulses following to the third point in time of the third light pulse of the short light pulses at a second inner-picture interval of time, and

wherein the third point in time of the third light pulse of the short light pulses follows to the second point in time of the second light pulse of the short light pulses at an inner-picture interval of time of not more than 50% of the first inner-picture interval of time and of not more than 50% of the second inner-picture interval of time;

determining a first distribution and a second distribution of real positions of the individual objects of the plurality of objects in the three-dimensional space from their images in the first pictures and from their images in the second pictures, respectively,

wherein at least the first or the second distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space is determined from the images of the individual objects of the plurality of objects in the first or second pictures, respectively, in that an estimated distribution of the real positions of the individual objects in the three-dimensional space is adapted to their images in the first or second pictures, respectively, and

wherein the estimated distribution of the real positions of the individual objects in the three-dimensional space is an object distribution of all of the individual objects in the three-dimensional space for which trajectories in the three-dimensional space have been determined before; and

determining temporally resolved trajectories of the individual objects of the plurality of objects in the three-dimensional space from the determined first and second distributions of the real positions of the individual objects,

wherein different real positions of the same individual objects of the plurality of objects in the first distribution are interrelated,

wherein different real positions of the same individual objects of the plurality of objects in the second distribution are interrelated, and

wherein different real positions of the same individual objects of the plurality of objects in the first and second distributions are interrelated.

15. A method of tracking a plurality of point-shaped objects in a three-dimensional space, the method comprising

recording two-dimensional pictures of the plurality of objects by means of at least two black and white cameras out of at least two different imaging directions;

wherein at least first and second pictures of the at least two black and white cameras are each exposed at at least two points in time in equal ways in that the objects are illuminated with short light pulses at the at least two points in time,

wherein the first pictures are exposed simultaneously for a first time at a first point in time of a first light pulse of the short light pulses and for a last time at a second point in time of a second light pulse of the short light pulses following to the first point in time of the first light pulse of the short light pulses at a first inner-picture interval of time,

wherein the second pictures are exposed simultaneously for a first time at a third point in time of a third light pulse of the short light pulses and for a last time at a fourth point in time of a fourth light pulse of the short light pulses following to the third point in time of the third light pulse of the short light pulses at a second inner-picture interval of time, and

wherein the third point in time of the third light pulse of the short light pulses follows to the second point in time of the second light pulse of the short light pulses at an inner-picture interval of time of not more than 20% of the first inner-picture interval of time and of not more than 20% of the second inner-picture interval of time;

determining a first distribution and a second distribution of real positions of the individual objects of the plurality of objects in the three-dimensional space from their images in the first pictures and from their images in the second pictures, respectively; and

determining temporally resolved trajectories of the individual objects of the plurality of objects in the three-dimensional space from the determined first and second distributions of the real positions of the individual objects,

wherein different real positions of the same individual objects of the plurality of objects in the first distribution are interrelated,

wherein different real positions of the same individual objects of the plurality of objects in the second distribution are interrelated, and

wherein different real positions of the same individual objects of the plurality of objects in the first and second distributions are interrelated.

16. The method of claim 15 , wherein at least the first or the second distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space is determined by triangulation from the images of the individual objects of the plurality of objects in the first or second pictures, respectively.

17. The method of claim 15 , wherein at least the first or the second distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space is tomographically determined from the images of the individual objects of the plurality of objects in the first or second pictures, respectively.

18. The method of claim 15 , wherein at least the first or the second distribution of the real positions of the individual objects of the plurality of objects in the three-dimensional space is determined from the images of the individual objects of the plurality of objects in the first or second pictures, respectively, in that an estimated distribution of the real positions of the individual objects in the three-dimensional space is adapted to their images in the first or second pictures, respectively.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 13, 2018
From: WIENEKE, BERNHARD; SCHROEDER, ANDREAS, DR.; HENNING, ARNE, DR.; NOVARA, MATTEO, DR.; SCHANZ, DANIEL, DR.
To: LAVISION GMBH
Reel/Frame 046628/0519 →
Priority Claims (1)
DE 10 2017 113 194 · Jun 14, 2017 · national
Continuity (1)
Related Publication 20190139235A1 · May 9, 2019