Granted Patent B2

US 12,437,409 · App. 17/896,842 · Granted Oct 7, 2025

Inventors: Jung-Hao Yang (New Taipei, TW); Chin-Pin Kuo (New Taipei, TW); Chih-Te Lu (New Taipei, TW)

Assignee: HON HAI PRECISION INDUSTRY CO., LTD.

G06T7/11H04N13/271

Abstract

Claims (91)

1. A method for processing images implemented in an electronic device comprising:

obtaining images when a vehicle is moving, the images comprising at least one left image and at least one right image;

obtaining instance segmentation images by performing an instance segmentation process on the images, the instance segmentation images comprising left images of instance segmentation corresponding to the left images and right images of instance segmentation corresponding to the right images;

obtaining a predicted disparity map by reconstructing the left images based on a pre-established autoencoder;

generating a first error value of the autoencoder for processing the images according to the at least one left image, the predicted disparity map, and the at least one right image, and generating a second error value of the autoencoder for processing the instance segmentation images according to the left images of instance segmentation, the predicted disparity map, and the right images of instance segmentation;

establishing an autoencoder model by adjusting the autoencoder according to the first error value and the second error value;

obtaining a test monocular image as the vehicle is moving, and obtaining a target disparity map by reconstructing the test monocular image according to the autoencoder model; and

obtaining a depth image corresponding to the test monocular image by converting the target disparity map.

2. The method according to claim 1 , wherein obtaining images when a vehicle is moving comprises:

obtaining the left images and the right images by controlling a binocular camera device to capture images of a view as the vehicle is moving; or

obtaining images corresponding to the vehicle from a first preset database as the left images, obtaining capture time of each of the left images from the first preset database, and obtaining the images corresponding to the vehicle from a second preset database as the right images, and obtaining capture time of each of the right images from the second preset database.

3. The method according to claim 1 , wherein obtaining instance segmentation images by performing an instance segmentation process on the images comprises:

inputting the images as the vehicle is moving into a pre-trained instance segmentation model, and obtaining output images of the pre-trained instance segmentation model;

detecting a segmentation accuracy of the instance segmentation model for segmenting the images based on the output images; and

in response that the segmentation accuracy is greater than a preset threshold, determining the output images to be the instance segmentation images.

4. The method according to claim 1 , wherein obtaining a predicted disparity map by reconstructing the left images based on a pre-established autoencoder comprises:

obtaining a feature vector of the left images by encoding the left images based on an encoder of the autoencoder; and

obtaining the predicted disparity map by mapping the feature vector based on a decoder of the autoencoder.

5. The method according to claim 1 , wherein generating a first error value of the autoencoder for processing the images according to the at least one left image, the predicted disparity map, and the at least one right image comprises:

obtaining a predicted disparity value corresponding to each pixel in the left image from the predicted disparity map;

locating a first pixel position of each pixel in the left image;

determining a second pixel position based on the predicted disparity value and the first pixel position;

obtaining an RGB pixel value of the pixel corresponding to the first pixel position from the left image;

generating a predicted right image corresponding to the left image by the autoencoder according to the RGB pixel value of the pixel corresponding to the first pixel position and the second pixel position; and

calculating the first error value according to the right image and the predicted right image.

6. The method according to claim 5 , wherein the first error value is calculated according to a formula of:

MSE =( y−ŷ ) 2 ;

in which, MSE representing the first error value, y representing the RGB pixel value of the right image, and ŷ representing the RGB pixel value of the predicted right image.

7. The method according to claim 1 , wherein establishing an autoencoder model by adjusting the autoencoder according to the first error value and the second error value comprises:

obtaining a total training error value of the autoencoder by calculating a sum of the first error value and the second error value; and

adjusting configuration parameters of the autoencoder, until the total training error values do not decrease, and the autoencoder model being established.

8. The method according to claim 1 , wherein obtaining a depth image corresponding to the test monocular image by converting the target disparity map comprises:

obtaining a focal length value of a capturing device of the test monocular image, and obtaining a distance between lenses of the capturing device;

obtaining a target disparity value of each pixel in the test monocular image from the target disparity map;

calculating a depth pixel value corresponding to each pixel in the test monocular image according to the focal length value, the distance, and the target disparity value; and

generating the depth image according to the pixels and the depth pixel value of each of the pixels in the test monocular image.

9. An electronic device comprising:

at least one processor; and

a storage device coupled to the at least one processor and storing instructions for execution by the at least one processor to cause the at least one processor to:

obtain images when a vehicle is moving, wherein the images comprise at least one left image and at least one right image;

obtain instance segmentation images by performing an instance segmentation process on the images, wherein the instance segmentation images comprise left images of instance segmentation corresponding to the left images and right images of instance segmentation corresponding to the right images;

obtain a predicted disparity map by reconstructing the left images based on a pre-established autoencoder;

generate a first error value of the autoencoder for processing the images according to the at least one left image, the predicted disparity map, and the at least one right image, and generating a second error value of the autoencoder for processing the instance segmentation images according to the left images of instance segmentation, the predicted disparity map, and the right images of instance segmentation;

establish an autoencoder model by adjusting the autoencoder according to the first error value and the second error value;

obtain a test monocular image during moving of the vehicle, and obtain a target disparity map by reconstructing the test monocular image according to the autoencoder model; and

obtain a depth image corresponding to the test monocular image by converting the target disparity map.

10. The electronic device according to claim 9 , wherein the at least one processor is further caused to:

obtaining the left images and the right images by controlling the binocular camera device to capture images of a view as the vehicle is moving; or

obtain images corresponding to the vehicle from the first preset database as the left images, obtain the capture time of each of the left images from the first preset database, and obtain the images corresponding to the vehicle from the second preset database as the right images, and obtain the capture time of each of the right images from the second preset database.

11. The electronic device according to claim 9 , wherein the at least one processor is further caused to:

input the images as the vehicle is moving into a pre-trained instance segmentation model, and obtain output images of the pre-trained instance segmentation model;

detect a segmentation accuracy of the instance segmentation model for segmenting the images based on the output images; and

in response that the segmentation accuracy is greater than a preset threshold, determine the output image to be the instance segmentation images.

12. The electronic device according to claim 9 , wherein the at least one processor is further caused to:

obtain a feature vector of the left image by encoding the left images based on an encoder of the autoencoder; and

obtain the predicted disparity map by mapping the feature vector based on a decoder of the autoencoder.

13. The electronic device according to claim 9 , wherein the at least one processor is further caused to:

obtain the predicted disparity value corresponding to each pixel in the left image from the predicted disparity map;

locate the first pixel position of each pixel in the left image; determining a second pixel position based on the predicted disparity value and the first pixel position;

obtain an RGB pixel value of the pixel corresponding to the first pixel position from the left image;

generate the predicted right image corresponding to the left image by the autoencoder according to the RGB pixel value of the pixel corresponding to the first pixel position and the second pixel position; and

calculate the first error value according to the right image and the predicted right image.

14. The electronic device according to claim 9 , wherein the first error value is calculated according to a formula of:

MSE =( y−ŷ ) 2 ;

in which, MSE representing the first error value, y representing the RGB pixel value of the right image, and ŷ representing the RGB pixel value of the predicted right image.

15. The electronic device according to claim 9 , wherein the at least one processor is further caused to:

obtain the total training error value of the autoencoder by calculating the sum of the first error value and the second error value; and

adjust configuration parameters of the autoencoder, until the total training error values do not decrease, and establish the autoencoder model.

16. The electronic device according to claim 9 , wherein the at least one processor is further caused to:

obtain a focal length value of a capturing device of the test monocular image, and obtain the distance between the lenses in the capturing device;

obtain a target disparity value of each pixel in the test monocular image from the target disparity map;

calculate the depth pixel value corresponding to each pixel in the test monocular image according to the focal length value, the distance, and the target disparity value; and

generate the depth image according to the pixels and the depth pixel value of each of the pixels in the test monocular image.

17. A non-transitory computer-readable storage medium having instructions stored thereon, when the instructions are executed by a processor of an electronic device, the processor is configured to perform a method for processing images, wherein the method comprises:

obtaining images when a vehicle is moving, wherein the images comprise at least one left image and at least one right image;

obtaining instance segmentation images by performing an instance segmentation process the images, wherein the instance segmentation images comprise left images of instance segmentation corresponding to the left images and right images of instance segmentation corresponding to the right images;

obtaining a predicted disparity map by reconstructing the left images based on a pre-established autoencoder;

generating a first error value of the autoencoder for processing the images as the vehicle is moving according to the at least one left image, the predicted disparity map, and the at least one right image, and generating a second error value of the autoencoder for processing the instance segmentation images according to the left images of instance segmentation, the predicted disparity map, and the right images of instance segmentation;

establishing an autoencoder model by adjusting the autoencoder according to the first error value and the second error value;

obtaining a test monocular image as the vehicle is moving, and obtaining a target disparity map by reconstructing the test monocular image according to the autoencoder model; and

obtaining a depth image corresponding to the test monocular image by converting the target disparity map.

18. The non-transitory computer-readable storage medium according to claim 17 , wherein obtaining images when a vehicle is moving comprises:

obtaining the left images and the right images by controlling the binocular camera device to capture images of a view as the vehicle is moving; or

obtaining images corresponding to the vehicle from the first preset database as the left images, obtaining the capture time of each of the left images from the first preset database, and obtaining the images corresponding to the vehicle from the second preset database as the right images, and obtaining the capture time of each of the right images from the second preset database.

19. The non-transitory computer-readable storage medium according to claim 17 , wherein obtaining instance segmentation images by performing an instance segmentation process on the images comprises:

inputting the images as the vehicle is moving into a pre-trained instance segmentation model, and obtaining output images of the pre-trained instance segmentation model;

detecting a segmentation accuracy of the instance segmentation model for segmenting the images based on the output images; and

in response that the segmentation accuracy is greater than a preset threshold, determine the output image to be the instance segmentation images.

20. The non-transitory computer-readable storage medium according to claim 17 , wherein obtaining a predicted disparity map by reconstructing the left images based on a pre-established autoencoder comprises:

obtaining a feature vector of the left image by encoding the left images based on an encoder of the autoencoder; and

obtaining the predicted disparity map by mapping the feature vector based on a decoder of the autoencoder.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (10)