Photorealistic human holographic augmented reality communication with interactive control in real-time using a cluster of servers

1. A real-time holographic augmented reality image processing method, comprising:

receiving, at a cluster of servers, real-time image data from an image capturing component;

extracting one or more objects or a scene from the real-time image data based on results from real-time adaptive learning and one or more object/scene extraction parameters, wherein the real-time adaptive learning comprises object learning, object recognition, object segmentation, scene learning, scene recognition, scene segmentation, or a combination thereof;

extracting one or more human objects from the real-time image data based on results from real-time adaptive human learning and one or more human extraction parameters, wherein the real-time adaptive human learning comprises human characteristic learning, human recognition, human segmentation, human body movement tracking, or a combination thereof;

receiving augmented reality (AR) input data; and

creating holographic AR image data by projecting, for each image, the extracted object or scene, the extracted human object, and the AR input data using a multi-layered mechanism based on projection parameters.

2. The real-time image processing method of claim 1 , wherein the cluster of servers comprises a single server, a group of servers located a same location, or multiple group of servers located in multiple physical locations.

3. The real-time image processing method of claim 1 , wherein the method further comprise:

continuing to receive additional images constituting the real-time image data while applying real-time error compensation to the image capture component based on the real-time adaptive learning of the image data; and

performing the real-time adaptive learning of the additional image data to render updated real-time learning results.

4. The real-time image processing method of claim 1 , further comprising:

updating real-time learning results based on offline object data, offline human identification data, or real-time adaptive learning results from a previous time point.

5. The real-time image processing method of claim 1 , wherein the multi-layered mechanism comprises:

separating each pixel in a holographic AR image into multiple layers; and

determining each pixel of the holographic AR image based on corresponding pixels in the extracted object or scene, the extracted human object, and the augmented reality (AR) input data based on the projection parameters.

6. The real-time image processing method of claim 1 , wherein each pixel of the AR input data is separated into multiple layers.

7. The real-time processing method of claim 1 , wherein the one or more object/scene extraction parameters and the one or more human extraction parameters are the same.

8. The real-time image processing method of claim 1 , wherein the projection parameters comprise a depth relation, a transparency relation, a positional relation, a visual relation, human interactive control information interpreted from human gesture, a human body movement, a human facial expression, or a combination thereof.

9. A computer system comprising:

one or more processors; and

a non-transitory computer-readable medium including one or more sequences of instructions that, when executed by one or more processors, cause the processors to perform operations comprising:

receiving, at a cluster of servers, real-time image data from an image capturing component;

extracting one or more objects or a scene from the real-time image data based on results from real-time adaptive learning and one or more object/scene extraction parameters, wherein the real-time adaptive learning comprises object learning, object recognition, object segmentation, scene learning, scene recognition, scene segmentation, or a combination thereof;

extracting one or more human objects from the real-time image data based on results from real-time adaptive human learning and one or more human extraction parameters, wherein the real-time adaptive human learning comprises human characteristic learning, human recognition, human segmentation, human body movement tracking, or a combination thereof;

receiving augmented reality (AR) input data; and

creating holographic AR image data by projecting, for each image, the extracted object or scene, the extracted human object, and the AR input data using a multi-layered mechanism based on projection parameters.

10. The computer system of claim 9 , wherein the cluster of servers comprises a single server, a group of servers located a same location, or multiple group of servers located in multiple physical locations.

11. The computer system of claim 9 , wherein the operations further comprises:

continuing to receive additional images constituting the real-time image data while applying real-time error compensation to the image capture component based on the real-time adaptive learning of the image data; and

performing the real-time adaptive learning of the additional image data to render updated real-time learning results.

12. The computer system of claim 9 , wherein the operations further comprises:

updating real-time learning results based on offline object data, offline human identification data, or real-time adaptive learning results from a previous time point.

13. The computer system of claim 9 , wherein the multi-layered mechanism comprises:

separating each pixel in a holographic AR image into multiple layers; and

determining each pixel of the holographic AR image based on corresponding pixels in the extracted object or scene, the extracted human object, and the augmented reality (AR) input data based on the projection parameters.

14. The computer system of claim 9 , wherein each pixel of the AR input data is separated into multiple layers.

15. The computer system of claim 9 , wherein the projection parameters comprise a depth relation, a transparency relation, a positional relation, a visual relation, human interactive control information interpreted from human gesture, a human body movement, a human facial expression, or a combination thereof.

16. The real-time image processing method of claim 9 , wherein the projection parameters comprise a depth relation, a transparency relation, a positional relation, a visual relation, human interactive control information interpreted from human gesture, a human body movement, a human facial expression, or a combination thereof.

17. A non-transitory computer-readable medium including one or more sequences of instructions that, when executed by one or more processors, cause the processors to perform operations comprising:

receiving, at a cluster of servers, and from an image capturing component, real-time image data;

extracting one or more objects or a scene from the real-time image data based on results from real-time adaptive learning and one or more object/scene extraction parameters, wherein the real-time adaptive learning comprises object learning, object recognition, object segmentation, scene learning, scene recognition, scene segmentation, or a combination thereof;

extracting one or more human objects from the real-time image data based on results from real-time adaptive human learning and one or more human extraction parameters, wherein the real-time adaptive human learning comprises human characteristic learning, human recognition, human segmentation, human body movement tracking, or a combination thereof;

receiving augmented reality (AR) input data; and

creating holographic AR image data by projecting, for each image, the extracted object or scene, the extracted human object, and the AR input data using a multi-layered mechanism based on projection parameters.

18. The non-transitory computer-readable medium of claim 17 , wherein the cluster of servers comprises a single server, a group of servers located a same location, or multiple group of servers located in multiple physical locations.

19. The non-transitory computer-readable medium of claim 17 , wherein the operations further comprises:

continuing to receive additional images constituting the real-time image data while applying real-time error compensation to the image capture component based on the real-time adaptive learning of the image data; and

performing the real-time adaptive learning of the additional image data to render updated real-time learning results.

20. The non-transitory computer-readable medium of claim 17 , wherein the operations further comprises:

updating real-time learning results based on offline object data, offline human identification data, or real-time adaptive learning results from a previous time point.

21. The non-transitory computer-readable medium of claim 17 , wherein the multi-layered mechanism comprises:

separating each pixel in a holographic AR image into multiple layers; and

determining each pixel of the holographic AR image based on corresponding pixels in the extracted object or scene, the extracted human object, and the augmented reality (AR) input data based on the projection parameters.

22. The non-transitory computer-readable medium of claim 17 , wherein each pixel of the AR input data is separated into multiple layers.

23. The non-transitory computer-readable medium of claim 17 , wherein the one or more object/scene extraction parameters and the one or more human extraction parameters are the same.

24. The non-transitory computer-readable medium of claim 17 , wherein the projection parameters comprise a depth relation, a transparency relation, a positional relation, a visual relation, human interactive control information interpreted from human gesture, a human body movement, a human facial expression, or a combination thereof.