Methods and apparatus for capturing images and using images for depth determination and/or other purposes
Methods and apparatus for illuminating scene areas to facilitate depth determination are described. A sequence of frames is displayed, e.g., projected, onto a scene area to illuminate the area for image capture. The sequence of frames includes a patterned frame followed by a concealing frame. The patterned frame and concealing frame are each displayed for no more than 1/60 of a second. The concealing frame maybe and sometimes does display a complementary pattern to the preceding patterned frame. When viewed sequentially by a human viewer the human sees a non-distracting uniformly illuminated scene area as a result of the effect of the concealing frame. One or more cameras are used to capture images of the illuminated area during a time in which the pattern is displayed and the scene area is effectively painted with a pattern that can facilitate depth determination based on one or more captured images.
1 . A method of capturing images of an area illuminated with an illumination sequence of frames including a first illumination frame corresponding to a first illumination frame period having a first light pattern and a first frame duration followed by a first concealing frame having a second light pattern and a second frame duration, the second light pattern concealing the first light pattern when an illuminated area is observed over a duration corresponding to a combined period having a combined duration equal to a sum of the first and second frame durations, the method comprising: operating a first camera to capture a first image of the illuminated area using an exposure time which is different from said combined duration, the captured first image including a light pattern due to the illumination of the illuminated area; operating a second camera to capture a second image of the illuminated area using said exposure time, said second camera being operated to capture the second image at the same time the first camera is being operated to capture the first image, said exposure time being a non-even multiple of the first frame duration and being different from said combined duration; and making a stereoscopic depth determination based on both the first and second images image.
2 . The method of claim 1 , wherein said first frame and the first concealing frame have the same duration and wherein the exposure time is aligned to begin with the start of one of the first frame or the start of the first concealing frame.
3 . The method of claim 1 , further comprising:
producing an additional image of the illuminated area which does not include said first light pattern, producing the additional image including at least one of:
i) operating an additional camera to capture the additional image of the illuminated area, using a second exposure time which is an integer multiple of the combined duration;
or ii) operating the first camera to capture the additional image of the illuminated area using the second exposure time which is an integer multiple of the combined duration.
4 . The method of claim 3 , further comprising:
storing the additional image; and
performing a model training operation, said step of performing a model training operation including using the additional image to train a model to recognize an object in the additional image.
5 . The method of claim 1 , further comprising:
synchronizing, based on captured images, the first camera with an illumination device used to project the first frame and the first concealing frame.
6 . The method of claim 1 , wherein making the stereoscopic depth determination is based on a captured light pattern included in the first and second images.
7 . The method of claim 1 ,
wherein the first frame duration and second frame duration are the same.
8 . The method of claim 7 , wherein the non-even multiple of the frame duration is one, said exposure time being equal to the duration of the first and second frames.
9 . The method of claim 1 , wherein the first exposure time is less than or equal to the first frame duration.
10 . The method of claim 1 ,
wherein the exposure time is longer than the first frame duration.
11 . The method of claim 1 , further comprising:
controlling a start of an exposure used in the capture of the first image to align with the start of one of said first and second frames.
12 . The method of claim 11 , further comprising:
prior to controlling the start of the exposure used in the capture of the first image, electronically or wirelessly synchronizing the first camera with an illumination device used to project said first frame and the first concealing frame.
13 . The method of claim 11 , further comprising:
prior to controlling the start of the exposure used in the capture of the first image, synchronizing the first camera to the illumination device based on processing of images captured by the first camera.
14 . The method of claim 1 , further comprising:
adjusting an image capture start time of the first camera to optimize the contrast of images captured by the first camera.
15 . A system for use in an area illuminated with an illumination sequence of frames including a first frame having a first light pattern and a first frame duration followed by a first concealing frame having a second light pattern and a second frame duration, the second light pattern concealing the first light pattern when an illuminated area is observed over a duration corresponding to a combined period having a combined duration equal to the sum of the first and second frame durations, the system comprising:
a first camera;
a second camera; and
a processor, the processor being configured to:
control the first camera to capture a first image of the illuminated area using an exposure time which is different from said combined duration, the captured first image including a light pattern due to the illumination of the illuminated area;
control the second camera to capture a second image of the illuminated area using said exposure time, said second camera being operated to capture the second image at the same time the first camera is being operated to capture the first image, said exposure time being a non-even multiple of the first frame duration and being different from said combined duration; and
make a stereoscopic depth determination based on both the first and the second images.
16 . The system of claim 15 , wherein the processor is further configured to:
produce an additional image of the illuminated area which does not include said first light pattern by: i) controlling an additional camera to capture the additional image of the illuminated area, using a second exposure time which is an integer multiple of the combined duration;
or ii) controlling the first camera to capture the additional image of the illuminated area using the second exposure time which is an integer multiple of the combined duration.
17 . The system of claim 16 , further comprising:
a storage device that stores the additional image; and
wherein the processor is further configured to:
use the additional image to train a model to recognize an object in the additional image.
18 . The system of claim 15 , wherein the non-even multiple of the frame duration is one, said exposure time being equal to the duration of the first frame.
19 . The system of claim 15 , wherein the exposure time is less than the first frame duration.
20 . A non-transitory machine readable medium including processor executable instructions which when executed by a processor control the capture and use of one or more images in an area illuminated with an illumination sequence of frames including a first frame having a first light pattern and a first frame duration followed by a first concealing frame having a second light pattern and a second frame duration, the second light pattern concealing the first light pattern when an illuminated area is observed over a duration corresponding to a combined period having a combined duration equal to a sum of the first and second frame durations, the processor, as part of controlling the capture and use of one or more images performing the steps of:
controlling a first camera to capture a first image of the illuminated area using an exposure time which is different from said combined duration, the captured first image including a light pattern due to the illumination of the illuminated area;
controlling the second camera to capture a second image of the illuminated area using said exposure time, said second camera being operated to capture the second image at the same time the first camera is being operated to capture the first image, said exposure time being a non-even multiple of the first frame duration and being different from said combined duration; and
making a stereoscopic depth determination based on the first and second images.