Image sensors and sensing methods to obtain Time-of-Flight and phase detection information

1. An image sensor pixel, comprising:

a plurality of sub-pixels;

a microlens covering the plurality of sub-pixels; and

a read-out circuit (ROC) for extracting time-of-flight (ToF) signals of each sub-pixel individually, wherein the image sensor pixel includes a switch, wherein in one state the switch is closed so that the plurality of sub-pixels form together one pixel and the ROC reads out the one pixel for generating a ToF depth map, and wherein in another state the switch is opened so that the ROC reads out each sub-pixel individually for generating a stereo depth map.

2. The image sensor pixel of claim 1 , wherein the plurality of sub-pixels includes 2 sub-pixels.

3. The image sensor pixel of claim 1 , wherein the plurality of sub-pixels includes 4 sub-pixels.

4. The image sensor pixel of claim 1 , wherein the image sensor pixel is a direct ToF (d-ToF) image sensor pixel.

5. The image sensor pixel of claim 1 , wherein the image sensor pixel is an indirect ToF (i-ToF) image sensor pixel.

6. The image sensor pixel of claim 1 , included in an image sensor of a camera having a focal length fin the range of 1.5 mm-10 mm.

7. The image sensor pixel of claim 1 , included in an image sensor of a stereo camera having a baseline B in the range of 0.5 mm-50 mm.

8. The image sensor pixel of claim 1 , included in an image sensor of a stereo camera having a baseline B in the range of 0.5 mm-25 mm.

9. The image sensor pixel of claim 1 , included in an image sensor of a stereo camera having a baseline B in the range of 0.5 mm-10 mm.

10. The image sensor pixel of claim 1 , wherein each sub-pixel has a size of 1 μm-10 μm.

11. The image sensor pixel of claim 1 , included in an image sensor of a stereo camera that has a vertical or a horizontal baseline.

12. The image sensor pixel of claim 1 , wherein the image sensor pixel is included in a image sensor of a stereo camera, and wherein the stereo camera is included in a mobile device.

13. The image sensor pixel of claim 12 , wherein the mobile device is a smartphone.

14. A method, comprising:

providing a time-of-flight (ToF) image sensor pixel that includes a plurality of sub-pixels, wherein the plurality of sub-pixels is covered by a microlens;

using a read-out circuit (ROC) to read out ToF signals of each sub-pixel individually, by:

obtaining a stereo depth map and an ToF map calculated from the ToF signals;

analyzing the stereo depth map and the ToF depth map for assigning stereo scores to segments of the stereo depth map and for assigning ToF scores to the segments of the ToF depth map; and

generating a fused depth map by using stereo depth map data for segments that have high stereo scores and using ToF depth map data for segments that have high ToF scores.

15. The method of claim 14 , wherein the using a ROC to read out ToF signals of each sub-pixel individually includes using the ROC to read out fewer than all ToF signals generated by the plurality of the sub-pixels.

16. The method of claim 14 , wherein the image sensor pixel is a direct ToF (d-ToF) image sensor pixel.

17. The method of claim 14 , wherein the image sensor pixel is an indirect ToF (i-ToF) image sensor pixel.

18. The method of claim 14 , wherein the 1-shot depth map is used to generate a high fps depth map stream having a fps≥35.

19. The method of claim 14 , wherein the image sensor pixel is included in a image sensor of a stereo camera, and wherein the stereo camera is included in a mobile device.

20. The method of claim 19 , wherein the mobile device is a smartphone.