Method and device for seeing through an obstruction based on smart glasses, and computer-readable storage medium

1. A method for seeing through an obstruction based on smart glasses, comprising:

acquiring a first image of a user's view by smart glasses and recognizing an obstruction image in the first image;

acquiring a second image of the user's view by at least one image acquisition device;

replacing the obstruction image in the first image with portion of the second image that corresponds to the obstruction image, including steps of:

extracting a plurality of first feature points and a plurality of second feature points respectively from image portions other than the obstruction image of the first image and the second image, the second feature points correspond to the first feature points in one-to-one correspondence:

calculating and obtaining an image transformation matrix for converting from an imaging viewing angle of the image acquisition device to the user's viewing angle from based on all of the first feature points and all of the second feature points:

performing image transformation on the second image using the image transformation matrix;

splicing the portion of the image transformed second image that corresponds to the obstruction image and the first image with the obstruction image removed, so as to generate an unobstruction image of the user's view.

2. The method for seeing through an obstruction based on smart glasses according to claim 1 , wherein, extracting a plurality of first feature points and a plurality of second feature points respectively from image portions other than the obstruction image of the first image and the second image, includes:

extracting a plurality of first feature points and a plurality of second feature points respectively from image portions other than the obstruction image of the first image and the second image by a feature extraction algorithm.

3. The method for seeing through an obstruction based on smart glasses according to claim 1 , wherein,

the setting pixel width area of an edge of the first image with the obstruction image removed includes part of the obstruction image.

4. The method for seeing through an obstruction based on smart glasses according to claim 1 , wherein, before generating the unobstruction image by splicing the portion of the image transformed second image that corresponds to the obstruction image and the first image with the obstruction image removed together, further comprising:

initially localizing the splicing position of the portion of the image-transformed second image that corresponds to the obstruction image with the first image with the obstruction image removed, based on image acquisition angle and spatial location information of the image acquisition device.

5. The method for seeing through an obstruction based on smart glasses according to claim 1 , wherein, recognizing the obstruction image in the first image by the smart glasses, includes:

recognizing the obstruction image in the first image based on a constant property of the obstruction or a graphic marker on the obstruction.

6. The method for seeing through an obstruction based on smart glasses according to claim 1 , further comprising:

when picture in the user's view changes, the smart glasses updates the first image, the image acquisition device reacquires the second image so as to update the second image, and regenerates the unobstruction image based on the updated first image and the updated second image.

7. The method for seeing through an obstruction based on smart glasses according to claim 1 , further comprising:

when the viewing angle of the user changes, the smart glasses determines whether or not it is necessary to update the first image based on detection data of the gyroscope on the smart glasses and detection data of a gyroscope at a setting position;

if it is necessary to update the first image, the first image can be updated by the smart glasses, and the unobstruction image can be regenerated based on the updated first image.

8. The method for seeing through an obstruction based on smart glasses according to claim 1 , further comprising:

after the viewing angle of the user changes, reacquiring the second image by using another image acquisition device, and regenerating the unobstruction image based on the reacquired second image.

9. The method for seeing through an obstruction based on smart glasses according to claim 1 , wherein, difference between a direction angle of the obstruction with respect to the user and an imaging viewing angle of the image acquisition device is smaller than 90 degrees.

10. The method for seeing through an obstruction based on smart glasses according to claim 1 , wherein, the image acquisition devices may be in plural, acquiring the second image of the user's view by multiple image acquisition devices, includes:

acquiring third images of the user's view by each of the image acquisition device;

splicing the third images together to form the second image with no obstruction.

11. A non-transitory computer-readable storage medium including computer-readable instructions, wherein, when being executed, the computer-readable instructions cause a processor to perform at least operations according to claim 1 .

12. A device, comprising:

an image acquisition module for acquiring an image of user's view;

a storage for storing a program instruction;

a processor which is connected with the image acquisition module and the storage, executes the program instruction in the storage, and processes the image of the user's view in accordance with the steps according to claim 1 .