Granted Patent B2

US 12,361,514 · App. 17/894,083 · Granted Jul 15, 2025

Inventor: Cevat Yerli (Dubai, AE)

Assignee: TMRW Group IP

G06T3/4007G06T5/50

Abstract

Claims (52)

1. A system comprising:

a sending unit comprising a processor and memory, the sending unit configured to receive image frames of an object of interest, the sending unit comprising:

a frame selector configured to select image frames from the image frames of the object of interest at set intervals; and

an encoder configured to generate selected frame latent vectors from the selected frames, the selected frame latent vectors being a compressed representation of the selected image frames of the object of interest;

wherein the sending unit is further programmed to send the selected frame latent vectors to a receiving unit comprising a processor and memory; and

the receiving unit configured to receive the selected frame latent vectors, the receiving unit comprising:

an interpolator configured to reconstruct by interpolation a number of in-between latent vectors between two successive selected frame latent vectors;

wherein the interpolator is configured to send both the received selected frame latent vectors and reconstructed in-between latent vectors to a decoder; and

the decoder configured to decode both the received selected frame latent vectors and reconstructed in-between latent vectors and generate image frames of the object of interest from the selected frame latent vectors and the reconstructed in-between latent vectors, including image frames corresponding to image frames of the object of interest that were not selected by the frame selector.

2. The system of claim 1 , wherein the number of reconstructed in-between latent vectors is the same as the image frames not selected by the frame selector.

3. The system of claim 1 , wherein the number of reconstructed in-between latent vectors is not the same as the image frames not selected by the frame selector.

4. The system of claim 1 , wherein the receiving unit has an additional latent vector generator configured to generate additional replacement vectors to replace damaged or missing latent vectors not received by the receiving unit.

5. The system of claim 1 , comprising a plurality of sending units and receiving units, wherein the plurality of sending units send a corresponding plurality of selected frame latent vectors to the plurality of receiving units via a selective forwarding unit (SFU), the SFU being configured to:

receive the plurality of selected frame latent vectors;

select which of the receiving units to send the plurality of selected frame latent vectors;

and forward the corresponding latent frame latent vectors to the selected receiving units.

6. The system of claim 1 , wherein the sending unit further comprises an image enhancer configured to enhance an image of the image frames.

7. The system of claim 6 , wherein the image enhancer is further configured to enhance the image by:

detecting the object of interest in the image frames; and

upscaling the object of interest in size in the sending unit, wherein such upscaling comprises an increase in resolution of the object of interest in the image frames; and

wherein the receiving unit is further configured to receive the image frames with the upscaled object of interest and downscale the upscaled object of interest.

8. The system of claim 1 , wherein the sending unit further comprises an image reducer configured to reduce an image size of an image frames.

9. The system of claim 8 , wherein the image reducer is further configured to reduce the image size by removing background from the image frames.

10. The system of claim 8 , wherein the image reducer is further configured to reduce the image size by separating the object of interest and background in the frames; and

performing higher resolution compression on the object of interest in each image frame and performing lower resolution compression on the background in each image frame, thereby achieving a reduction in the image size.

11. A method performed by a computer system comprising one or more computing devices including a processor and memory, the method comprising:

receiving image frames of an object of interest;

selecting image frames from the image frames of the object of interest at set intervals;

generating, from the selected frames, selected frame latent vectors, the selected frame latent vectors being a compressed representation of the image frames of the object of interest;

reconstructing by interpolation a number of in-between latent vectors between two successive selected frame latent vectors;

decoding both the received selected frame latent vectors and reconstructed in-between latent vectors; and

generating image frames of the object of interest from the selected frame latent vectors and the in-between latent vectors, including image frames corresponding to image frames of the object of interest that were not selected in the selecting step.

12. The method of claim 11 , wherein the number of generated in-between latent vectors is the same as the frames not selected.

13. The method of claim 11 , further comprising reducing the image size of the image frames by removing background from the image frames.

14. The method of claim 11 , further comprising:

reducing the image size by separating the object of interest and background in the frames; and

performing higher resolution compression on the object of interest in each image frame and performing lower resolution compression on the background in each image frame, thereby achieving a reduction in image size.

15. The method of claim 11 , further comprising enhancing the image of the image frames by:

detecting an object of interest in the image frames;

upscaling the object of interest in size, wherein such upscaling comprises an increase in the resolution of the object of interest in the image frame; and

downscaling the upscaled object of interest.

16. The method of claim 11 , comprising implementing a plurality of sending units and receiving units, wherein the plurality of sending units send a corresponding plurality of selected frame latent vectors to the plurality of receiving units via a selective forwarding unit, the selective forwarding unit being configured to:

receive the plurality of selected frame latent vectors;

select which receiving units to send the plurality of selected frame latent vectors; and

forward the corresponding latent frame latent vectors to the selected receiving units.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a computer system, cause the computer system to perform operations comprising:

receiving image frames of an object of interest;

selecting image frames from the image frames of the object of interest at set intervals;

generating, from the selected frames, selected frame latent vectors, the selected frame latent vectors being a compressed representation of the image frames of the object of interest;

reconstructing by interpolation a number of in-between latent vectors between two successive selected frame latent vectors;

decoding both the received selected frame latent vectors and reconstructed in-between latent vectors; and

generating image frames of the object of interest from the selected frame latent vectors and the in-between latent vectors, including image frames corresponding to image frames of the object of interest that were not selected in the selecting operation.

Assignments (2)

Continuity (1)

References Cited (32)