Projector-camera misalignment correction for structured light systems

1. A method of misalignment correction in a structured light device, the method comprising:

extracting features from a first captured image of a scene, wherein the first captured image is captured by an imaging sensor component of the structured light device, and wherein the first captured image comprises a pattern projected into the scene by a projector component of the structured light device;

matching the features of the first captured image to predetermined features of a pattern image corresponding to the projected pattern to generate a dataset of matching features;

determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features; and

applying the image alignment transformation model to a second captured image using the determined alignment correction parameter values.

2. The method of claim 1 , wherein the first captured image and the second captured image are a same captured image.

3. The method of claim 1 , wherein the image alignment transformation model is an affine transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

4. The method of claim 1 , wherein the image alignment transformation model is a perspective transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

5. The method of claim 1 , wherein the predetermined features of the pattern image are a subset of a total number of features of the pattern image.

6. The method of claim 1 , wherein the extracting, matching, and determining are performed at intervals during operation of the structured light device.

7. The method of claim 1 , wherein the pattern is a two-dimensional high frequency binary pattern.

8. A structured light device comprising:

an imaging sensor component configured to capture images of a scene;

a projector component configured to project a pattern into the scene;

a memory configured to store predetermined features of a pattern image corresponding to the pattern;

means for extracting features from a first image of the scene captured by the imaging sensor component, wherein the first image comprises the pattern projected into the scene by the projector component;

means for matching the features of the first image to the predetermined features to generate a dataset of matching features;

means for determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features; and

means for applying the image alignment transformation model with the determined alignment correction parameter values to a second image of the scene captured by the imaging sensor component.

9. The structured light device of claim 8 , wherein the first image and the second image are a same image.

10. The structured light device of claim 8 , wherein the image alignment transformation model is an affine transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

11. The structured light device of claim 8 , wherein the image alignment transformation model is a perspective transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

12. The structured light device of claim 8 , wherein the predetermined features of the pattern image are a subset of a total number of features of the pattern image.

13. The structured light device of claim 8 , wherein the extracting, matching, and determining are performed at intervals during operation of the structured light device.

14. The structured light device of claim 8 , wherein the pattern is a two-dimensional high frequency binary pattern.

15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor in a structured light device, cause a method of misalignment correction to be performed, the method comprising:

extracting features from a first captured image of a scene, wherein the first captured image is captured by an imaging sensor component of the structured light device, and wherein the first captured image comprises a pattern projected into the scene by a projector component of the structured light device;

matching the features of the first captured image to predetermined features of a pattern image corresponding to the projected pattern to generate a dataset of matching features;

determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features; and

applying the image alignment transformation model to a second captured image using the determined alignment correction parameter values.

16. The computer-readable medium of claim 15 , wherein the first captured image and the second captured image are a same captured image.

17. The computer-readable medium of claim 15 , wherein the image alignment transformation model is an affine transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

18. The computer-readable medium of claim 15 , wherein the image alignment transformation model is a perspective transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

19. The computer-readable medium of claim 15 , wherein the predetermined features of the pattern image are a subset of a total number of features of the pattern image.

20. The computer-readable medium of claim 15 , wherein the extracting, matching, and determining are performed at intervals during operation of the structured light device.