Depth data measurement head, depth data computing device, and corresponding method
A depth data computing device and a corresponding method are provided. The computing device includes a projection device for scanning and projecting a set of structured light with different patterns onto a shooting area, and first and second image sensors for capturing the area to obtain image frames under illumination of the structured light for single depth calculation. Each image sensor includes at least a first sub-image sensor and a second sub-image sensor sharing at least part of an optical path, the sub-image sensors successively imaging the structured light of different patterns projected by the projection device. Continuous imaging is performed using multiple binocular configurations, enabling high-frame-rate depth data acquisition.
1 . A depth data computing device, comprising:
a projection device for scanning and projecting a set of structured light having different patterns to a shooting area, and the set of structured lights includes at least two structured lights of different patterns;
first and second image sensors having a predetermined relative positional relationship for capturing the shooting area to obtain a set of image frame pairs illuminated by the set of structured light; and
a processor connected to the projection device and the first and second image sensors, configured to determine the depth data of an object in the shooting area according to the set of image frame pairs obtained by imaging the structured light,
wherein, each of the first and second image sensors comprises at least a first sub-image sensor and a second sub-image sensor that share at least part of an optical path, the first sub-image sensors of the first and second image sensors form a first binocular sub-image sensor pair used to image the structured light of a first pattern among the different patterns sequentially projected by the projection device, and the second sub-image sensors of the first and second image sensors form a second binocular sub-image sensor pair used to image the structured light of a second pattern among the different patterns sequentially projected by the projection device after the first pattern.
2 . The depth data computing device according to claim 1 , comprising:
synchronization device for making the first binocular sub-image sensor pair and the second binocular sub-image sensor pair to sequentially image the at least two structured light of different patterns, wherein the first binocular sub-image sensor pair images simultaneously, the second binocular sub-image sensor pair images simultaneously, with a first interval therebetween, the first interval being smaller thana frame imaging interval of the sub-image sensor, while the projection device projects the structured light of the first pattern and the second pattern with the first interval therebetween, and making each of the first and second sub-image sensor perform its next frame imaging at a second interval not smaller than the frame imaging interval of the sub-image sensor, and is synchronized with the projection of the projection device.
3 . The depth data computing device according to claim 1 , wherein each of the first and second image sensors comprises:
lens assembly for receiving incident return structured light;
a beam splitting device for splitting the incident return structured light into at least a first beam and a second beam,
wherein the first sub-image sensor images the first light beam corresponding to the returning structured light with the first pattern, and the second sub-image sensor images the second light beams corresponding to the returning structured light with the second pattern.
4 . The depth data computing device according to claim 1 , wherein each of the first and second image sensors comprises:
lens assembly for receiving incident return structured light;
an optical path conversion device for delivering the incident return structured light to at least a first sub-path and a second sub-path,
wherein the first sub-image sensor images the returning structured light corresponding to the first pattern on the first sub-path, and the second sub-image sensor images the returned structured light corresponding to the second different pattern on the second sub-path.
5 . The depth data computing device according to claim 1 , wherein in each of the first and second image sensors, the first sub-image sensor and the second sub-image sensor that share at least part of the optical path have the same optical path length.
6 . The depth data computing device according to claim 5 , wherein in each of the first and second image sensors, the first sub-image sensor and the second sub-image sensor that share at least part of the optical path are aligned at the pixel level.
7 . The depth data computing device according to claim 1 , wherein each sub-image sensor is an infrared light sensor.
8 . The depth data computing device according to claim 1 , wherein the set of structured lights with different patterns projected by the projection device is a set of structured lights with different coded stripes.
9 . The depth data computing device according to claim 1 , wherein the projection device comprises:
a laser generator for generating line-shaped and/or point laser light, and the laser generator performs high-speed switching to scan and project light and dark structured light corresponding to the stripe code.
10 . The depth data computing device according to claim 9 , wherein the projection device comprises:
a light emitting device for generating line-shaped light; and
a reflecting device for reflecting the line-shaped light to project the line-shaped light moving in a direction perpendicular to the stripe direction to the shooting area.
11 . The depth data computing device according to claim 9 , wherein each sub-image sensor is a global image sensor.
12 . The depth data computing device according to claim 11 , wherein in each of the first and second image sensors, the first sub-image sensor and the second sub-image sensor that share at least part of the optical path are installed upside down from each other.
13 . The depth data computing device according to claim 9 , wherein each sub-image sensor is a rolling shutter image sensor, and the depth data computing device further comprises:
a column synchronization device, for synchronously enabling the pixel column in the stripe direction corresponding to the current scanning position in the sub-image sensor currently used for imaging to perform imaging based on the scanning position of the projection device.
14 . The depth data computing device according to claim 9 , wherein, each of the first and second image sensors comprises a beam splitting device, and the one sub-image sensor of the first binocular sub-image sensor pair and the one sub-image sensor of the second binocular sub-image sensor pair share the optical path until a beam splitting surface of the beam splitting device in the first image sensor, and the other sub-image sensor of the first binocular sub-image sensor pair and the other sub-image sensor of the second binocular sub-image sensor pair share the optical path until a beam splitting surface of the beam splitting device in the second image sensor.
15 . The depth data computing device according to claim 1 , wherein, each of the first and second image sensors further comprises a beam splitting device, the first sub-image sensor and the second sub-image sensor of each of the first and second image sensors share the optical path until a beam splitting surface of the beam splitting device.
16 . A method for measuring depth data, comprising:
scanning and projecting first structured light to a shooting area;
capturing the shooting area to obtain a first image frame pair illuminated by first structured light of a first pattern by using a first binocular sub-image sensor pair with a predetermined relative positional relationship;
after the projection of the first structured light, scanning and projecting second structured light of a second pattern to the shooting area, the second pattern is different from the first pattern;
capturing the shooting area to obtain a second image frame pair under the illumination of the second structured light by using a second binocular sub-image sensor pair with a predetermined relative positional relationship, wherein one sub-image sensor of the first binocular sub-image sensor pair and one sub-image sensor of the second binocular sub-image sensor pair share at least part of an optical path and form a first image sensor, and the other sub-image sensor of the first binocular sub-image sensor pair and the other sub-image sensor of the second binocular sub-image sensor pair share at least part of an optical path and form a second image sensor; and
determining the depth data of the object to be measured in the shooting area according to the first and second image frame pairs.
17 . The method according to claim 16 , wherein scanning and projecting second structured light of the second pattern to the shooting area comprises:
projecting the second structured light after the projection of the first structured light, with a first interval therebetween, the first interval being smaller than a frame imaging interval of the sub-image sensor, and
time interval between capturing the first image frame pair by the first binocular sub-image sensor pair and capturing the second image frame pair by the second binocular sub-image sensor pair is smaller than the frame imaging interval of the sub-image sensors.
18 . The method according to claim 17 , further comprising:
projecting a third structured light of a third pattern to the shooting area after the projection of the second structured light, with a second interval therebetween, the second interval being not smaller than the frame imaging interval of the sub-image sensors, the third pattern is different from the first pattern and the second pattern;
capturing the shooting area to obtain a third image frame pair under the illumination of the third structured light by using the first binocular sub-image sensor pair, wherein the third image frame pair is used to determine the depth data of the shooting area.
19 . The method according to claim 17 , further comprising:
projecting a third structured light of a third pattern to the shooting area after the projection of the second structured light, with the first interval therebetween, the third pattern is different from the first pattern and the second pattern; and
capturing the shooting area to obtain a third image frame pair under the illumination of the third structured light by using a third sub-image sensor pair, wherein one sub-image sensor of the third sub-image sensor pair belongs to the first image sensor and shares at least part of the optical path with other sub-image sensors of the first image sensor, the other sub-image sensor of the third sub-image sensor pair belongs to the second image sensor and shares at least part of the optical path with other sub-image sensors of the second image sensor, the third image frame pair is used to determine the depth data of the shooting area.
20 . The method according to claim 16 , wherein in each of the first and second image sensors, the first sub-image sensor and the second sub-image sensor respectively acquire the split beams of the first structured light and the second structured light, and selectively turning on one of the first sub-image sensor and the second sub-image sensor for capturing.
21 . The method according to claim 16 , wherein the optical path of the incident light is controlled so that in each of the first and second image sensors, only the first sub-image sensor acquires and captures the first structured light, and only the second sub-image sensor acquires and captures the second structured light.