Mapping three-dimensional depth map data onto two-dimensional images
A 3D distance sensor projects a plurality of points of light onto a target. A 2D camera having a fixed position relative to the 3D distance sensor acquires a first image of the points on the target. A light receiving system of the 3D distance sensor acquires a second image of the points on the target. A set of 2D coordinates of a first point is associated with a distance coordinate of the first point to form a single set of coordinates for the first point. The set of 2D coordinates is obtained from the first image, and the distance coordinate is obtained from the second image. A relational expression is derived from the single set of coordinates and describes a relationship between a distance of the light receiving system to the target and a position of the first point on an image sensor of the 2D camera.
1. A method, comprising:
controlling, by a processing system, a three-dimensional distance sensor to project a pattern of light onto a target surface, wherein the pattern of light comprises a plurality of points of light;
controlling, by the processing system, a two-dimensional camera to acquire a first image of the pattern of light on the target surface, wherein the two-dimensional camera has a fixed position relative to the three-dimensional distance sensor;
controlling, by the processing system, a light receiving system of the three-dimensional distance sensor to acquire a second image of the pattern of light on the target surface;
associating, by the processing system, a set of two-dimensional coordinates of a first point of light of the plurality of points of light with a distance coordinate of a set of three-dimensional coordinates of the first point of light to form a single set of coordinates for the first point of light, wherein the set of two-dimensional coordinates is obtained from the first image, and wherein the set of three-dimensional coordinates is obtained from the second image; and
deriving, by the processing system, a relational expression from the single set of coordinates, wherein the relational expression describes a relationship between a distance of the light receiving system to the target surface and a position of the first point of light on an image sensor of the two-dimensional camera.
2. The method of claim 1 , wherein the three-dimensional distance sensor is calibrated, prior to controlling the distance sensor to project the pattern of light, using a three-dimensional target.
3. The method of claim 1 , wherein the pattern of light comprises infrared light, and the light receiving system includes an infrared bandpass filter.
4. The method of claim 3 , wherein the two-dimensional camera includes an infrared cut-off filter.
5. The method of claim 4 , wherein the target surface comprises a material that emits fluorescence when the material is excited by infrared light.
6. The method of claim 5 , wherein the target surface has a planar shape.
7. The method of claim 1 , wherein the first image and the second image are acquired simultaneously by the two-dimensional camera and the three-dimensional distance sensor, respectively.
8. The method of claim 1 , wherein the set of two-dimensional coordinates of a first point of light describe a position of the first point of light on an image sensor of the two-dimensional camera.
9. The method of claim 1 , wherein the relational expression is derived from single sets of coordinates that are formed for a subset of the plurality of points of light, wherein the subset includes that first point of light and at least one additional point of light.
10. The method of claim 1 , wherein the associating comprises identifying, by the processing system, the first point of light in the first image and in the second image based on a position of the first point of light relative to other points of light of the plurality of points of light.
11. The method of claim 1 , further comprising, after the associating but prior to the deriving:
repeating, by the processing system, the controlling the three-dimensional distance sensor to project a pattern of light, the controlling the two-dimensional camera, the controlling the light receiving system of the three-dimensional distance sensor, and the associating, wherein the repeating is performed after the target surface has been moved to alter a distance between the target surface and the three-dimensional distance sensor and a distance between the target surface and the two-dimensional camera.
12. The method of claim 1 , further comprising, after the deriving:
controlling, by the processing system, the two-dimensional camera to acquire a third image of the pattern of light on an object other than the target surface;
controlling, by the processing system, the light receiving system of the three-dimensional distance sensor to acquire a fourth image of the pattern of light on the object, wherein the third image and the fourth image are acquired simultaneously; and
identifying, by the processing system, a set of two-dimensional coordinates for a second point of light of the plurality of points of light, based on a three-dimensional distance coordinate of the second point of light measured by the three-dimensional distance sensor and on the relational expression.
13. The method of claim 1 , wherein a lens of the two-dimensional camera is set to a fixed value that controls a zoom or focus of the lens.
14. The method of claim 13 , wherein the associating includes associating the fixed value with the single set of coordinates.
15. The method of claim 14 , further comprising, after the associating but prior to the deriving:
repeating, by the processing system, the controlling the three-dimensional distance sensor to project a pattern of light, the controlling the two-dimensional camera, the controlling the light receiving system of the three-dimensional distance sensor, and the associating, wherein the repeating is performed after the fixed value has been set to a new fixed value.
16. A non-transitory machine-readable storage medium encoded with instructions executable by a processor, wherein, when executed, the instructions cause the processor to perform operations, the operations comprising:
controlling a three-dimensional distance sensor to project a pattern of light onto a target surface, wherein the pattern of light comprises a plurality of points of light;
controlling a two-dimensional camera to acquire a first image of the pattern of light on the target surface, wherein the two-dimensional camera has a fixed position relative to the three-dimensional distance sensor;
controlling a light receiving system of the three-dimensional distance sensor to acquire a second image of the pattern of light on the target surface;
associating a set of two-dimensional coordinates of a first point of light of the plurality of points of light with a distance coordinate of a set of three-dimensional coordinates of the first point of light to form a single set of coordinates for the first point of light, wherein the set of two-dimensional coordinates is obtained from the first image, and wherein the set of three-dimensional coordinates is obtained from the second image; and
deriving a relational expression from the single set of coordinates, wherein the relational expression describes a relationship between a distance of the light receiving system to the target surface and a position of the first point of light on an image sensor of the two-dimensional camera.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the pattern of light comprises infrared light, wherein the light receiving system includes an infrared bandpass filter, and wherein the two-dimensional camera includes an infrared cut-off filter.
18. The non-transitory machine-readable storage medium of claim 17 , wherein the target surface comprises a material that emits fluorescence when the material is excited by infrared light.
19. The non-transitory machine-readable storage medium of claim 16 , further comprising, after the deriving:
controlling the two-dimensional camera to acquire a third image of the pattern of light on an object other than the target surface;
controlling the light receiving system of the three-dimensional distance sensor to acquire a fourth image of the pattern of light on the object, wherein the third image and the fourth image are acquired simultaneously; and
identifying a set of two-dimensional coordinates for a second point of light of the plurality of points of light, based on a three-dimensional distance coordinate of the second point of light measured by the three-dimensional distance sensor and on the relational expression.
20. An apparatus, comprising:
a processor; and
a non-transitory machine-readable storage medium encoded with instructions executable by a processor, wherein, when executed, the instructions cause the processor to perform operations, the operations comprising:
controlling a three-dimensional distance sensor to project a pattern of light onto a target surface, wherein the pattern of light comprises a plurality of points of light;
controlling a two-dimensional camera to acquire a first image of the pattern of light on the target surface, wherein the two-dimensional camera has a fixed position relative to the three-dimensional distance sensor;
controlling a light receiving system of the three-dimensional distance sensor to acquire a second image of the pattern of light on the target surface;
associating a set of two-dimensional coordinates of a first point of light of the plurality of points of light with a distance coordinate of a set of three-dimensional coordinates of the first point of light to form a single set of coordinates for the first point of light, wherein the set of two-dimensional coordinates is obtained from the first image, and wherein the set of three-dimensional coordinates is obtained from the second image; and
deriving a relational expression from the single set of coordinates, wherein the relational expression describes a relationship between a distance of the light receiving system to the target surface and a position of the first point of light on an image sensor of the two-dimensional camera.