Granted Patent B2

US 12,470,834 · App. 18/733,890 · Granted Nov 11, 2025

Inventors: Tzu-Chia Liu (Taipei, TW); Chi-Min Weng (Taipei, TW); Jiun-Shiung Chen (Taipei, TW)

Assignee: LITE-ON TECHNOLOGY CORPORATION

H04N23/81H04N5/2628H04N7/18H04N23/88H04N23/90

Abstract

Claims (76)

1 . A ranging system, comprising:

a pinhole camera having a first field-of-view (FoV) and configured to capture a pinhole image of a scene;

a fisheye camera having a second field-of-view and configured to capture a fisheye image of the scene, wherein the second field-of-view is larger than the first field-of-view; and

a processor configured to:

perform an undistorting process on the fisheye image to obtain a corresponding undistorted fisheye image;

perform a size-converting process on the pinhole image to obtain a corresponding size-converted pinhole image, wherein the corresponding size-converted pinhole image is consistent with the corresponding undistorted fisheye image in imaging size;

obtain a transformation relation between a pinhole image plane of the pinhole camera and a fisheye image plane of the fisheye camera;

obtain a corresponding point of the corresponding undistorted fisheye image corresponding to a target point of the corresponding size-converted pinhole image based on the transformation relation, wherein the target point and the corresponding point correspond to a physical point; and

obtain a distance between the ranging system and the physical point based on the transformation relation, the target point and the corresponding point;

wherein the undistorted fisheye image has a FoV overlapping region;

wherein the FoV overlapping region is a preset region; the processor is further configured to:

obtain an epipolar line of the corresponding undistorted fisheye image corresponding to the target point of the corresponding size-converted pinhole image based on the transformation relation, wherein the epipolar line comprises a plurality of pixel points.

2 . The ranging system according to claim 1 , wherein the pinhole camera and the fisheye camera are disposed on a moving vehicle, and the pinhole camera and the fisheye camera are staggered in a length direction of the moving vehicle.

3 . The ranging system according to claim 1 , wherein the processor is further configured to:

perform a color tone alignment processing on the pinhole image and the fisheye image to make the pinhole image and the fisheye image consistent in tone.

4 . The ranging system according to claim 1 , wherein the processor is further configured to:

obtain the corresponding point of the FoV overlapping region of the corresponding undistorted fisheye image corresponding to the target point of the size-converted pinhole image according to the transformation relation.

5 . The ranging system according to claim 4 , wherein the corresponding undistorted fisheye image further has a non-FoV overlapping region; the non-FoV overlapping region is connected adjacent to a side edge and a lower edge of the FoV overlapping region; the processor is further configured to:

obtain the corresponding point of the pixel points of the epipolar line that is closest to the target point in a characteristic value.

6 . The ranging system of claim 1 , wherein the processor is further configured to:

obtain a ratio of a focal length of the fisheye camera to a focal length of the pinhole camera; and

reduce a size of the pinhole image by the ratio.

7 . A ranging method, comprising:

capturing, by a pinhole camera, a pinhole image of a scene, wherein the pinhole camera has a first field-of-view;

capturing, by a fisheye camera, a fisheye image of the scene, wherein the fisheye camera has a second field-of-view, wherein the second field-of-view is larger than the first field-of-view;

performing, by a processor, an undistorting process on the fisheye image to obtain a corresponding undistorted fisheye image;

performing, by the processor, a size-converting process on the pinhole image to obtain a corresponding size-converted pinhole image, wherein the corresponding size-converted pinhole image is consistent with the corresponding undistorted fisheye image in imaging size;

obtaining, by the processor, a transformation relation between a pinhole image plane of the pinhole camera and a fisheye image plane of the fisheye camera;

obtaining, by the processor, a corresponding point of the corresponding undistorted fisheye image corresponding to a target point of the corresponding size-converted pinhole image based on the transformation relation, wherein the target point and the corresponding point correspond to a physical point; and

obtaining, by the processor, a distance between the ranging system and the physical point based on the transformation relation, the target point and the corresponding point;

wherein the corresponding undistorted fisheye image has a FoV overlapping region;

wherein the FoV overlapping region is a preset region, and the ranging method further comprises:

obtaining, by the processor, an epipolar line of the corresponding undistorted fisheye image corresponding to the target point of the corresponding size-converted pinhole image based on the transformation relation, wherein the epipolar line comprises a plurality of pixel points.

8 . The ranging method according to claim 7 , wherein the pinhole camera and the fisheye camera are disposed on a moving vehicle, and the pinhole camera and the fisheye camera are staggered in a length direction of the moving vehicle.

9 . The ranging method according to claim 7 , further comprising:

performing, by the processor, a color tone alignment processing on the pinhole image and the fisheye image to make the pinhole image and the fisheye image consistent in tone.

10 . The ranging method according to claim 7 , wherein the ranging method further comprises:

obtaining, by the processor, the corresponding point of the FoV overlapping region of the corresponding undistorted fisheye image corresponding to the target point of the corresponding size-converted pinhole image based on the transformation relation.

11 . The ranging method according to claim 10 , wherein the corresponding undistorted fisheye image further has a non-FoV overlapping region; the non-FoV overlapping region is connected adjacent to a side edge and a lower edge of the FoV overlapping region; the ranging method further comprises:

obtaining the corresponding point of the pixel points of the epipolar line that is closest to the target point in a characteristic value.

12 . The ranging method according to claim 7 , further comprising:

obtaining, by the processor, a ratio of a focal length of the fisheye camera to a focal length of the pinhole camera; and

reducing, by the processor, a size of the pinhole image by the ratio.

13 . A ranging system, adapted for a mobile vehicle, and comprising:

a first camera having a first field-of-view and configured to capture a first image of a scene;

a second camera having a second field-of-view and configured to capture a second image of the scene, wherein the second field-of-view is larger than the first field-of-view; and

a processor configured to:

perform an undistorting process on one of the first image and the second image to obtain a corresponding undistorted image;

calculate a size conversion proportion based on camera parameters of the first camera and the second camera and perform a size-converting process on the other of the first image and the second image based on the size conversion proportion to obtain a corresponding size-converted image, wherein the corresponding size-converted image is consistent with the corresponding undistorted image in imaging size;

obtain a transformation relation between a first image plane of the first camera and a second image plane of the second camera;

obtain a corresponding point of the corresponding undistorted image corresponding to a target point of the corresponding size-converted image based on the transformation relation, wherein the target point and the corresponding point correspond to a physical point;

obtain a distance between the ranging system and the physical point based on the transformation relation, the target point and the corresponding point;

wherein the undistorted image has a FoV overlapping region;

wherein the FoV overlapping region is a preset region; the processor is further configured to:

obtain an epipolar line of the corresponding undistorted image corresponding to the target point of the corresponding size-converted image based on the transformation relation, wherein the epipolar line comprises a plurality of pixel points.

14 . The ranging system according to claim 13 , wherein the first camera is a pinhole camera and the second camera is a fisheye camera.

15 . The ranging system according to claim 13 , wherein the processor is further configured to:

obtain the corresponding point of the pixel points of the epipolar line that is closest to the target point in a characteristic value.

16 . The ranging system according to claim 13 , wherein the processor is further configured to:

obtain the corresponding point of the FoV overlapping region of the corresponding undistorted image corresponding to the target point of the size-converted image based on the transformation relation.

17 . A ranging method, comprising:

capturing, by a first camera, a first image of a scene, wherein the first camera has a first field-of-view;

capturing, by a second camera, a second image of the scene, wherein the second camera has a second field-of-view, and the second field-of-view is larger than the first field-of-view;

performing, by a processor, an undistorting process on one of the first image and the second image to obtain a corresponding undistorted image;

calculate, by the processor, a size conversion proportion by camera parameters of the first camera and the second camera and performing a size-converting process on another of the first image and the second image based on the size conversion proportion to obtain a corresponding size-converted image, wherein the corresponding size-converted image is consistent with the corresponding undistorted image in imaging size;

obtaining, by the processor, a transformation relation between a first image plane of the first camera and a second image plane of the second camera;

obtain, by the processor, a corresponding point of the corresponding undistorted image corresponding to a target point of the corresponding size-converted image based on the transformation relation, wherein the target point and the corresponding point correspond to a physical point; and

obtain, by the processor, a distance between the ranging system and the physical point based on the transformation relation, the target point and the corresponding point;

wherein the corresponding undistorted image has a FoV overlapping region;

wherein the FoV overlapping region is a preset region, and the ranging method further comprises:

obtaining, by the processor, an epipolar line of the corresponding undistorted image corresponding to the target point of the corresponding size-converted image based on the transformation relation, wherein the epipolar line comprises a plurality of pixel points.

18 . The ranging method according to claim 17 , wherein the first camera is a pinhole camera and the second camera is a fisheye camera.

19 . The ranging method according to claim 17 , further comprising:

obtaining the corresponding point of the pixel points of the epipolar line that is closest to the target point in a characteristic value.

20 . The ranging method according to claim 17 , wherein the ranging method further comprising:

obtaining, by the processor, the corresponding point of the FoV overlapping region of the corresponding undistorted image corresponding to the target point of the corresponding size-converted image based on the transformation relation.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (12)