System and method for tracking three dimensional objects
Embodiments of the invention are directed to improved systems and methods for three dimensional (3D) image reconstruction. The systems and methods are directed to extracting, digitally reconstructing and optionally tracking 3D objects from multiple two dimensional (2D) video camera sources. The systems and methods are directed to reconstructing a 3D scene via 2D cameras and then re-projecting this data back onto 2D surfaces. This system and method can greatly simplify the image processing required to analyze the 3D model by moving the analysis techniques back into the 2D domain.
1. A method for creating a 3D representation of a target object in a 3D digital domain, the method comprising:
(i) placing a target object within a background space;
(ii) directing at least two video cameras at said target object;
(iii) acquiring a total image from each of said video cameras directed at the target object;
(iv) subtracting said background space from each of said total images to produce a target image of the target object;
(v) performing internal calibration of each of said video cameras;
(vi) performing relative external calibration of two or more of said video cameras; and
(vii) reconstructing a 3D image of said target object via projections of one or more of said target images into said 3D digital domain, wherein said image reconstruction comprises:
(a) conceptualizing one or more virtual projectors in said 3D digital domain in a configuration analogous to the direction, and from the position, of said video cameras, and with the lens properties of said video cameras;
(b) inserting one or more virtual 2D slices into said 3D digital domain;
(c) comparing each of said virtual 2D slices to each of said projected images;
(d) incrementing a counter each time one of said projected images intersects one of said virtual 2D slices;
(e) comparing said counter to the number of said video cameras; and
(f) deciding whether said target object is considered to exist at the location of said virtual 2D slices in said 3D digital domain.
2. The method of claim 1 , wherein said image reconstruction further comprises dynamically configuring the characteristics of each of said virtual 2D slices to best fit the characteristics of said target object.
3. The method of claim 1 , wherein each of said virtual 2D slices are inserted into said 3D digital domain such that each of said virtual 2D slices is parallel to every other of said virtual 2D slices.
4. A method for creating a 3D representation of a target object in a 3D digital domain, the method comprising:
(i) placing a target object within a background space;
(ii) directing at least two video cameras at said target object;
(iii) acquiring a total image from each of said video cameras directed at the target object;
(iv) subtracting said background space from each of said total images to produce a target image of the target object;
(v) performing internal calibration of each of said video cameras;
(vi) performing relative external calibration of two or more of said video cameras;
(vii) reconstructing a 3D image of said target object via projections of one or more of said target images into said 3D digital domain, wherein said image reconstruction via projections of one or more of said target images into said 3D digital domain comprises projecting said target images acquired from each of said video cameras into said 3D digital domain in the direction, and from the position, of said video camera, and with the lens properties of said video camera; and
(viii) tracking said target object in said 3D digital domain, wherein said tracking of said target object comprises:
(1) performing 2D image analysis on each of said acquired images projected into said 3D digital domain; wherein the 2D image analysis comprises:
(a) assessing each virtual 2D slice for one or more image representation constructs;
(b) calculating the outline of each of said image representation constructs;
(c) fitting an ellipse to each of said image representation constructs;
(d) calculating the centerpoint, x- and y-radii, and an angle of rotation for each of said image representation constructs; and
(e) applying the known properties of each virtual 2D slice to each of said image representation constructs located within said virtual 2D slice to generate real-world coordinates of each of said image representation constructs;
(2) detecting the presence of said target object based on the known characteristics of said target object; and
(3) creating a rag doll virtual representation of said target object in said 3D digital domain.
5. A method for creating a 3D representation of a target object in a 3D digital domain, the method comprising:
(i) placing a target object within a background space;
(ii) directing at least two video cameras at said target object;
(iii) acquiring a total image from each of said video cameras directed at the target object;
(iv) subtracting said background space from each of said total images to produce a target image of the target object;
(v) performing internal calibration of each of said video cameras;
(vi) performing relative external calibration of two or more of said video cameras;
(vii) reconstructing a 3D image of said target object via projections of one or more of said target images into said 3D digital domain, wherein said image reconstruction via projections of one or more of said target images into said 3D digital domain comprises projecting said target images acquired from each of said video cameras into said 3D digital domain in the direction, and from the position, of said video camera, and with the lens properties of said video camera; and
(viii) tracking said target object in said 3D digital domain, wherein said tracking of said target object comprises:
(1) performing 2D image analysis on each of said acquired images projected into said 3D digital domain;
(2) detecting the presence of said target object based on the known characteristics of said target object, wherein detection of the target object comprises:
(a) conceptualizing one or more virtual projectors in said 3D digital domain in a configuration analogous to the configuration of said one or more video cameras;
(b) inserting one or more virtual 2D slices into said 3D digital domain at one or more locations predetermined based on the known characteristics of said target object;
(c) comparing each of said virtual 2D slices to each of said projected images;
(d) incrementing a counter each time one of said projected images intersects one of said virtual 2D slices;
(e) comparing said counter to the number of said one or more video cameras;
(f) deciding whether said target object is considered to exist at the location of said virtual 2D slices in said 3D digital domain;
(g) assessing each virtual 2D slice for one or more image representation constructs;
(h) calculating the outline of each of said image representation constructs;
(i) fitting an ellipse to each of said image representation constructs;
(j) calculating the centerpoint, x- and y-radii, and an angle of rotation for each of said image representation constructs; and
(k) applying the known properties of each virtual 2D slice to each of said image representation constructs located within said virtual 2D slice to generate real-world coordinates of each of said image representation constructs; and
(3) creating a rag doll virtual representation of said target object in said 3D digital domain.
6. The method of claim 5 , wherein said target object is a human, and the method further comprises at least two of:
(a) inserting a virtual 2D slice parallel to the floor and at floor height;
(b) inserting a virtual 2D slice parallel to the floor and at ankle height;
(c) inserting a virtual 2D slice parallel to the floor and at knee height;
(d) inserting a virtual 2D slice parallel to the floor and at hip height;
(e) inserting a virtual 2D slice parallel to the floor and at chest height; and
(f) inserting one or more virtual 2D slices parallel to the floor and at head height.
7. The method of claim 6 , further comprising determining the orientation of said human by analyzing the relationship between one or more image representation constructs located within said virtual 2D slice parallel to the floor and at floor height and one or more image representation constructs located within said 2D slice parallel to the floor and at ankle height.
8. The method of claim 7 , wherein the determination of the orientation of said human further comprises analyzing the relationship between one or more image representation constructs located within said virtual 2D slice parallel to the floor and at hip height and one or more image representation constructs located within said 2D slice parallel to the floor and at chest height.
9. The method of claim 8 , further comprising determining the height of said human.
10. The method of claim 9 , further comprising inserting one or more virtual 2D slices as end caps, such that the edge of said target object can be determined.