Apparatus and methods for distance estimation using stereo imagery
Frame sequences from multiple image sensors may be combined in order to form, for example, an interleaved frame sequence. Individual frames of the combined sequence may be configured a by combination (e.g., concatenation) of frames from one or more source sequences. The interleaved/concatenated frame sequence may be encoded using a motion estimation encoder. Output of the video encoder may be processed (e.g., parsed) in order to extract motion information present in the encoded video. The motion information may be utilized in order to determine a depth of visual scene, such as by using binocular disparity between two or more images by an adaptive controller in order to detect one or more objects salient to a given task. In one variant, depth information is utilized during control and operation of mobile robotic devices.
1. A method for motion detection and distance measurement of at least one target object, comprising:
receiving a first image frame from a first imaging camera and a second image frame from a second imaging camera, the first image frame being captured contemporaneous to the second image frame;
combining the first and second image frame to create a first combined frame;
receiving a third image frame from the first imaging camera and a fourth image frame from the second imaging camera, the third image frame being captured contemporaneous to the fourth image frame and immediately subsequent to the first and second image frames;
combining the third and fourth image frames to create a second combined frame;
generating an interleaved sequence of concatenated frames to evaluate distance and motion detection;
determining the distance measurement based on a pixel-wise disparity between image frames of the first or second combined frames; and
determining the motion of at least one target object based on a pixel-wise disparity between the first and second combined frames.
2. The method of claim 1 , further comprising:
extracting luminance data of pixels of the image frames within the interleaved sequence of concatenated frames; and
generating macroblocks comprising pixels of similar luminance within each of the image frames of the interleaved sequence of concatenated frames.
3. The method of claim 2 , wherein the determining of the distance measurement is based on an at least one pixel disparity between macroblocks of the first and second image frames of a first interleaved sequence of concatenated frames and a spatial separation of first and second imaging cameras.
4. The method of claim 2 , wherein the determining of motion of the at least one target object corresponds to assigning a motion vector to each of the macroblocks within the interleaved sequence of concatenated frames, the motion vectors being assigned based on an at least one pixel disparity between macroblocks of the first combined frame and the second combined frame.
5. The method of claim 4 , further comprising:
determining gestures of the at least one target object within the interleaved sequence of concatenated frames by performing at least one of:
(i) identifying background pixels within the interleaved sequence of concatenated frames based on spatially coherent motion or differential motion;
(ii) removing pixels corresponding to the identified background pixels; and
(iii) determining the gesture based on a resulting motion vector field of macroblocks within the first interleaved sequence of concatenated frames, the motion vector field being formed using the image processor by at least one remaining macroblock with an assigned motion vectors upon removal of the background pixels.
6. The method of claim 5 , wherein,
the resulting motion vector field of macroblocks is associated with a gesture based on an output from an adaptive predictor apparatus, and
the at least one target object comprises at least one of a human, inanimate object, portions of a human, or portions of an inanimate object.
7. The method of claim 1 , wherein,
the first and second imaging cameras are separated spatially by a nonzero distance.
8. A non-transitory computer readable medium comprising a plurality of instructions stored thereon, that when executed by at least one processor, configure the at least one processor to,
receive a first image frame from a first imaging camera and a second image frame from a second imaging camera, the first image frame being captured contemporaneous to the second image frame;
combine the first and second image frames to create a first combined frame;
receive a third image frame from the first imaging camera and a fourth image frame from the second imaging camera, the third image frame being captured contemporaneous to the fourth and subsequent to the first and second image frames;
combine the third and fourth image frames to create a second combined frame;
generate an interleaved sequence of concatenated frames to evaluate distance and motion detection;
determine distance measurement based on a pixel-wise disparity between image frames of the first or second combined frames; and
determine motion of at least one target object based on the pixel-wise disparity between the first and second combined frames.
9. The non-transitory computer readable medium of claim 8 , wherein the at least one processor is further configured to execute the computer readable instructions to,
extract luminance data of pixels of the image frames within the interleaved sequence of concatenated frames; and
generate macroblocks comprising pixels of similar luminance within each of the image frames of the interleaved sequence of concatenated frames.
10. The non-transitory computer readable medium of claim 9 , wherein the distance measurement determination is based on an at least one pixel disparity between macroblocks of the first and second image frames of a first interleaved sequence of concatenated frames and a spatial separation of first and second imaging cameras.
11. The non-transitory computer readable medium of claim 9 , wherein the motion of the at least one target object corresponds to assigning a motion vector to each of the macroblocks within the interleaved sequence of concatenated frames, the motion vectors being assigned based on an at least one pixel disparity between macroblocks of the first combined frame and the second combined frame.
12. The non-transitory computer readable medium of claim 11 , wherein the at least one processor is further configured to execute the computer readable instructions to,
determine gestures of the at least one target object within the interleaved sequence of concatenated frames by performing at least one of:
(i) identifying background pixels within the interleaved sequence of concatenated frames based on spatially coherent motion or differential motion;
(ii) removing pixels corresponding to the identified background pixels; and
(iii) determining the gesture based on a resulting motion vector field of macroblocks within the first interleaved sequence of concatenated frames, the motion vector field being formed using the image processor by at least one remaining macroblock with an assigned motion vectors upon removal of the background pixels.
13. The non-transitory computer readable medium of claim 12 , wherein,
the resulting motion vector field of macroblocks is associated with a gesture based on an output from an adaptive predictor apparatus, and
the at least one target object comprises at least one of a human, inanimate object, portions of a human, or portions of an inanimate object.
14. The non-transitory computer readable medium of claim 8 , wherein the first and second imaging cameras are separated spatially by a nonzero distance.
15. A system for motion detection and distance measurement of at least one target object, comprising:
a memory having computer readable instructions thereon; and
at least one processor configured to execute the computer readable instructions to,
receive a first image frame from a first imaging camera and a second image frame from a second imaging camera, the first image frame being captured contemporaneous to the second image frame;
combine the first and second image frames to create a first combined frame;
receive a third image frame from the first imaging camera and a fourth image frame from the second imaging camera, the third image frame being captured contemporaneous to the fourth and subsequent to the first and second image fames;
generate an interleaved sequence of concatenated frames to evaluate distance and motion detection;
combine the third and fourth image frames to create a second combined frame;
determine the distance measurement based on a pixel-wise disparity between image frames of the first or second combined frames; and
determine the motion of the at least one target object based on the pixel-wise disparity between the first and second combined frames.
16. The system of claim 15 , wherein the at least one processor is further configured to execute the computer readable instructions to,
extract luminance data of pixels of the image frames within the interleaved sequence of concatenated frames; and
generate macroblocks comprising pixels of similar luminance within each of the image frames of the interleaved sequence of concatenated frames.
17. The system of claim 16 , wherein the distance measurement determination is based on an at least one pixel disparity between macroblocks of the first and second image frames of a first interleaved sequence of concatenated frames and a spatial separation of first and second imaging cameras.
18. The system of claim 16 , wherein the motion of the at least one target object corresponds to assigning a motion vector to each of the macroblocks within the interleaved sequence of concatenated frames, the motion vectors being assigned based on an at least one pixel disparity between macroblocks of the first combined frame and the second combined frame.
19. The system of claim 18 , wherein the at least one processor is further configured to execute the computer readable instructions to,
determine gestures of the at least one target object within the interleaved sequence of concatenated frames by performing at least one of:
(i) identifying background pixels within the interleaved sequence of concatenated frames based on spatially coherent motion or differential motion;
(ii) removing pixels corresponding to the identified background pixels; and
(iii) determining the gesture based on a resulting motion vector field of macroblocks within the first interleaved sequence of concatenated frames, the motion vector field being formed using the image processor by at least one remaining macroblock with an assigned motion vectors upon removal of the background pixels.
20. The system of claim 19 , wherein,
the resulting motion vector field of macroblocks is associated with a gesture based on an output from an adaptive predictor apparatus, and
the at least one target object comprises at least one of a human, inanimate object, portions of a human, or portions of an inanimate object.
21. The system of claim 15 , wherein the first and second imaging cameras are separated spatially by a nonzero distance.