Granted Patent B2

US 11,688,143 · App. 17/179,988 · Granted Jun 27, 2023

Inventors: Danqing Sha (Shanghai, CN); Qiang Chen (Shanghai, CN); Zhen Jia (Shanghai, CN); Jinpeng Liu (Shanghai, CN)

Assignee: EMC IP Holdinq Company LLC

G06T19/006G06N3/045G06N3/088G06T3/4046G06V10/40

Abstract

Claims (85)

1. A method for processing an image, comprising:

acquiring an image about an augmented reality scene;

determining a target image part corresponding to a target object from the image;

using a machine learning model to augment information about the target object in the target image part to obtain an augmented target image part; and

displaying the augmented target image part;

wherein the machine learning model is configured to generate a complete image of a given object based on a partial image of the given object;

wherein the target image part comprises an unobscured part of an obscured image of the target object, the obscured image of the target object further comprising an obscured part that is obscured by at least a portion of an image of an additional object different than the target object; and

wherein using a machine learning model to augment information about the target object in the target image part comprises:

extracting the unobscured part of the target image part of the image; and

inputting the unobscured part of the target image part to the machine learning model to obtain a complete unobscured image of the target object as the augmented target image part.

2. The method according to claim 1 , wherein determining the target image part comprises:

extracting an image part corresponding to the target object from the image;

zooming the image part to a predetermined display size of the target object;

determining a ratio of the number of feature points of the target object in the zoomed image part to the number of standard feature points of the target object; and

determining the zoomed image part as the target image part in response to the ratio being less than a threshold.

3. The method according to claim 2 , wherein zooming the image part to a predetermined display size of the target object comprises:

zooming in the image part to the predetermined display size of the target object if it is determined that a size of the image part is smaller than the predetermined display size of the target object; and

zooming out the image part to the predetermined display size of the target object if it is determined that the size of the image part is larger than the predetermined display size of the target object.

4. The method according to claim 1 , wherein using a machine learning model to augment information about the target object in the target image part comprises:

inputting the target image part to the machine learning model for increasing the resolution of the target image part, wherein the machine learning model is configured to increase the resolution of an image.

5. The method according to claim 1 , wherein the target image part is a 2-dimensional image;

wherein using a machine learning model to augment information about the target object in the target image part comprises:

inputting the target image part to the machine learning model to obtain a 3-dimensional image of the target object, wherein the machine learning model is configured to convert a 2-dimensional image into a 3-dimensional image.

6. The method according to claim 1 , further comprising:

acquiring an original image about a reality scene through a camera;

acquiring feature point information in the reality scene; and

adding information about a virtual scene to the original image based on the feature point information to obtain the image about the augmented reality scene.

7. The method according to claim 1 , wherein the machine learning model is a generative adversarial network model.

8. An electronic device, comprising:

at least one processor; and

a memory coupled to the at least one processor and having instructions stored thereon, wherein the instructions, when executed by the at least one processor, cause the electronic device to execute actions comprising:

acquiring an image about an augmented reality scene;

determining a target image part corresponding to a target object from the image;

using a machine learning model to augment information about the target object in the target image part to obtain an augmented target image part; and

displaying the augmented target image part;

wherein the machine learning model is configured to generate a complete image of a given object based on a partial image of the given object;

wherein the target image part comprises an unobscured part of an obscured image of the target object, the obscured image of the target object further comprising an obscured part that is obscured by at least a portion of an image of an additional object different than the target object; and

wherein using a machine learning model to augment information about the target object in the target image part comprises:

extracting the unobscured part of the target image part of the image; and

inputting the unobscured part of the target image part to the machine learning model to obtain a complete unobscured image of the target object as the augmented target image part.

9. The electronic device according to claim 8 , wherein determining the target image part comprises:

extracting an image part corresponding to the target object from the image;

zooming the image part to a predetermined display size of the target object;

determining a ratio of the number of feature points of the target object in the zoomed image part to the number of standard feature points of the target object; and

determining the zoomed image part as the target image part in response to the ratio being less than a threshold.

10. The electronic device according to claim 9 , wherein zooming the image part to a predetermined display size of the target object comprises:

zooming in the image part to the predetermined display size of the target object if it is determined that a size of the image part is smaller than the predetermined display size of the target object; and

zooming out the image part to the predetermined display size of the target object if it is determined that the size of the image part is larger than the predetermined display size of the target object.

11. The electronic device according to claim 8 , wherein using a machine learning model to augment information about the target object in the target image part comprises:

inputting the target image part to the machine learning model for increasing the resolution of the target image part, wherein the machine learning model is configured to increase the resolution of an image.

12. The electronic device according to claim 8 , wherein the target image part is a 2-dimensional image;

wherein using a machine learning model to augment information about the target object in the target image part comprises:

inputting the target image part to the machine learning model to obtain a 3-dimensional image of the target object, wherein the machine learning model is configured to convert a 2-dimensional image into a 3-dimensional image.

13. The electronic device according to claim 8 , wherein the actions further comprise:

acquiring an original image about a reality scene through a camera;

acquiring feature point information in the reality scene; and

adding information about a virtual scene to the original image based on the feature point information to obtain the image about the augmented reality scene.

14. The electronic device according to claim 8 , wherein the machine learning model is a generative adversarial network model.

15. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions, wherein the machine-executable instructions, when executed by a machine, cause the machine to execute steps of a method for processing an image, comprising:

acquiring an image about an augmented reality scene;

determining a target image part corresponding to a target object from the image;

using a machine learning model to augment information about the target object in the target image part to obtain an augmented target image part; and

displaying the augmented target image part;

wherein the machine learning model is configured to generate a complete image of a given object based on a partial image of the given object;

wherein the target image part comprises an unobscured part of an obscured image of the target object, the obscured image of the target object further comprising an obscured part that is obscured by at least a portion of an image of an additional object different than the target object; and

wherein using a machine learning model to augment information about the target object in the target image part comprises:

extracting the unobscured part of the target image part of the image; and

inputting the unobscured part of the target image part to the machine learning model to obtain a complete unobscured image of the target object as the augmented target image part.

16. The computer program product according to claim 15 , wherein determining the target image part comprises:

extracting an image part corresponding to the target object from the image;

zooming the image part to a predetermined display size of the target object;

determining a ratio of the number of feature points of the target object in the zoomed image part to the number of standard feature points of the target object; and

determining the zoomed image part as the target image part in response to the ratio being less than a threshold.

17. The computer program product according to claim 16 , wherein zooming the image part to a predetermined display size of the target object comprises:

zooming in the image part to the predetermined display size of the target object if it is determined that a size of the image part is smaller than the predetermined display size of the target object; and

zooming out the image part to the predetermined display size of the target object if it is determined that the size of the image part is larger than the predetermined display size of the target object.

18. The computer program product according to claim 15 , wherein using a machine learning model to augment information about the target object in the target image part comprises:

inputting the target image part to the machine learning model for increasing the resolution of the target image part, wherein the machine learning model is configured to increase the resolution of an image.

19. The computer program product according to claim 15 , wherein the target image part is a 2-dimensional image;

wherein using a machine learning model to augment information about the target object in the target image part comprises:

inputting the target image part to the machine learning model to obtain a 3-dimensional image of the target object, wherein the machine learning model is configured to convert a 2-dimensional image into a 3-dimensional image.

20. The computer program product according to claim 15 , further comprising:

acquiring an original image about a reality scene through a camera;

acquiring feature point information in the reality scene; and

adding information about a virtual scene to the original image based on the feature point information to obtain the image about the augmented reality scene.

Assignments (10)

Priority Claims (1)

Continuity (1)