IP Library Granted Patent US 9,137,488
Granted Patent B2
US 9,137,488 · App. 13/661,924 · Granted Sep 15, 2015

Video chat encoding pipeline

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,137,488
App. No.
13/661,924
Granted
Sep 15, 2015
Kind
B2
Abstract

Implementations relate to a system for video encoding and conversion including an image resolution conversion component operable to convert a resolution of a source image frame from a first resolution to a second resolution to produce a first intermediate image frame at the second resolution; an image conversion component operable to receive the first intermediate image frame and convert an image size of the first intermediate image frame to another image frame size to produce a first viewable image frame; an image viewer component operable to display the first viewable image on a first display; a color space conversion component comprising a luminance conversion component and a chrominance operable to receive the first viewable image frame and convent a first luminance value and a first chrominance value of the first viewable image frame to a second intermediate image frame having a second luminance value and a second chrominance value.

Claims (97)

1. A first device, having a first display and a first camera, operable to modify display information to be viewable on a second device having a second display comprising:

at least one processor; and

non-transitory computer readable medium comprising instructions that cause the at least one processor to perform a method comprising:

receiving video information including a source image frame from the first camera of the first device and storing the source image frame in a first buffer;

receiving information related to a video format for displaying the video information on the second display of the second device;

converting the video information to the video format by:

converting a resolution of the source image frame to produce a first intermediate image frame and storing the first intermediate image frame in a second buffer;

converting a size of the first intermediate image frame to produce a second intermediate image frame and storing the second intermediate image frame in a third buffer; and

converting a first color space format comprising a luminance conversion and a chrominance conversion by receiving the second intermediate image frame and converting a first set of luminance values and a first set of chrominance values of the second intermediate image frame to a second color space format having a second set of luminance values and a second set of chrominance values and storing one or more luminance values for the luminance conversion in a fourth input frame buffer and storing one or more chrominance values for the chrominance conversion in a fifth input frame buffer; and

transmitting video in the video format.

2. The device according to claim 1 , wherein the converting comprises:

converting the resolution of the source image frame obtained from the first camera from a first resolution to a second resolution to produce the first intermediate image frame at the second resolution;

converting a first image size of the first intermediate image frame to a second image size to produce the second intermediate image;

converting the first color space format of the second intermediate image to the second color space format to produce a third intermediate image; and

encoding the third intermediate image for transmission to the second device.

3. The device according to claim 2 , wherein the first color space format is RGB and the second color space format is YUV.

4. The device according to claim 1 , further comprising:

converting the resolution of the source image frame obtained from the first camera from a first resolution to a second resolution to produce the first intermediate image frame at the second resolution;

converting a first image size of the first intermediate image frame to a second image size to produce the second intermediate image; and

providing the second intermediate image to the first display for viewing on the first device.

5. The device according to claim 1 , wherein the video format is provided based on information related to one or more display capabilities of the second display.

6. The device according to claim 5 , wherein the information includes display resolution capabilities of the second display.

7. A non-transitory computer readable storage medium comprising instruction that cause one or more processors to perform a method for modify display information on a first device having a first display to be viewable on a second device having a second display comprising:

receiving video information including a source image frame from a first camera of the first device and storing the source image frame in a first buffer;

receiving information related to a video format for displaying the video information on the second display of the second device;

converting the video information to the video format by:

converting a resolution of the source image frame to produce a first intermediate image frame and storing the first intermediate image frame in a second buffer;

converting a size of the first intermediate image frame to produce a second intermediate image frame and storing the second intermediate image frame in a third buffer; and

converting a first color space format comprising a luminance conversion and a chrominance conversion by receiving the second intermediate image frame and converting a first set of luminance values and a first set of chrominance values of the second intermediate image frame to a second color space format having a second set of luminance values and a second set of chrominance values and storing one or more luminance values for the luminance conversion in a fourth input frame buffer and storing one or more chrominance values for the chrominance conversion in a fifth input frame buffer; and

transmitting video in the video format.

8. The non-transitory computer readable storage medium according to claim 7 , wherein the converting comprises:

converting the resolution of the source image frame obtained from the first camera from a first resolution to a second resolution to produce the first intermediate image frame at the second resolution;

converting a first image size of the first intermediate image frame to a second image size to produce the second intermediate image;

converting the first color space format of the second intermediate image to the second color space format to produce a third intermediate image; and

encoding the third intermediate image for transmission to the second device.

9. The non-transitory computer readable storage medium according to claim 8 , wherein the first color space format and the second color space format comprise formats selected from the group consisting of: RGB, YUV, YUV 4:2:2 planar, YUV 4:2:0 planar, and YUV 4:4:4.

10. The non-transitory computer readable storage medium according to claim 7 , further comprising:

converting the resolution of the source image frame obtained from the first camera from a first resolution to a second resolution to produce the first intermediate image frame at the second resolution;

converting a first image size of the first intermediate image frame to a second image size to produce the second intermediate image; and

providing the second intermediate image to the first display for viewing on the first device.

11. The non-transitory computer readable storage medium according to claim 7 , wherein the video format is provided based on information related to on one or more display capabilities of the second display.

12. The non-transitory computer readable storage medium according to claim 11 , wherein the information includes display resolution capabilities of the second display.

13. A system for modifying display information on a first device to be viewable on a second device, the system comprising:

memory operable to store computer-executable components;

a processor operable to execute the computer-executable components stored within the memory, the computer-executable components comprising:

converting a resolution of a source image frame from a first resolution to a second resolution to produce a first intermediate image frame at the second resolution;

receiving the first intermediate image frame;

converting a vertical or a horizontal image size of the first intermediate image frame to another vertical or another horizontal image frame size to produce a first viewable image frame;

receiving and displaying the first viewable image frame from the image conversion component on a first display;

performing a color space conversion comprising a luminance conversion and a chrominance conversion by receiving the first viewable image frame and converting a first set of luminance values and a first set of chrominance values of the first viewable image frame to a second intermediate image frame having a second set of luminance values and a second set of chrominance values; and

encoding the second intermediate image frame in a format to be displayed on a second display,

a first input buffer operable to store the source image frame;

a second input buffer operable to store the first intermediate image frame;

a third input frame buffer operable to store the first viewable image frame;

a fourth input frame buffer operable to store one or more luminance values for the luminance conversion; and

a fifth input frame buffer operable to store one or more chrominance values for the chrominance conversion.

14. The system according to claim 13 , wherein the processor comprises a graphics processing unit.

15. The system according to claim 13 , further comprising rotating the source image frame from a first orientation to a second orientation.

16. The system according to claim 13 , wherein the first set of luminance values are encoded in an RGB color space and the second set of luminance values are encoded in YUV color space with a pixel including four bands to store intensity values.

17. The system according to claim 13 , wherein the second display instructs the processor as to what format the encoder component encodes the second intermediate image frame.

18. The system according to claim 13 , further comprising receiving the source image frame from a camera coupled to the first device operating in preview mode.

19. A system comprising:

memory operable to store computer-executable components;

a processor operable to execute the computer-executable components stored within the memory, the computer-executable components comprising:

converting a vertical or a horizontal image size of a source image frame to another vertical or another horizontal image frame size to produce a first intermediate image frame;

receiving the first intermediate image frame;

converting a resolution of the first intermediate image frame from a first resolution to a second resolution to produce a first viewable image frame at the second resolution;

receiving and displaying the first viewable image frame from the image resolution conversion component on a first display;

performing a color space conversion comprising a luminance conversion and a chrominance conversion by receiving the first viewable image frame and converting a first set of luminance values and a first set of chrominance values of the first viewable image frame to a second intermediate image frame having a second set of luminance values and a second set of chrominance values; and

encoding the second intermediate image frame in a format to be displayed on a second display,

a first input buffer operable to store the source image frame;

a second input buffer operable to store the first intermediate image frame;

a third input frame buffer operable to store the first viewable image frame;

a fourth input frame buffer operable to store one or more luminance values for the luminance conversion; and

a fifth input frame buffer operable to store one or more chrominance values for the chrominance conversion.

20. The system according to claim 19 , wherein the processor comprises a graphics processing unit.

21. The system according to claim 19 , further comprising rotating the source image frame from a first orientation to a second orientation.

22. A computer-implemented method for modifying display information on a first device having a first display to be viewable on a second device having a second display, the method comprising:

receiving video including a source image frame from a first camera of the first device and storing the source image frame in a first buffer;

receiving information related to a video format for displaying the video information on the second display of the second device;

converting the video information to the video format by:

converting a resolution of the source image frame to produce a first intermediate image frame and storing the first intermediate image frame in a second buffer;

converting a size of the first intermediate image frame to produce a second intermediate image frame and storing the second intermediate image frame in a third buffer; and

converting a first color space format comprising a luminance conversion and a chrominance conversion by receiving the second intermediate image frame and converting a first set of luminance values and a first set of chrominance values of the second intermediate image frame to a second color space format having a second set of luminance values and a second set of chrominance values and storing one or more luminance values for the luminance conversion in a fourth input frame buffer and storing one or more chrominance values for the chrominance conversion in a fifth input frame buffer; and

providing video in the video format.

23. The computer-implemented method according to claim 22 , wherein the converting comprises:

converting the resolution of the source image frame obtained from the first camera from a first resolution to a second resolution to produce the first intermediate image frame at the second resolution;

converting a first image size of the first intermediate image frame to a second image size to produce the second intermediate image;

converting the first color space format of the second intermediate image to the second color space format to produce a third intermediate image; and

encoding the third intermediate image for transmission to the second device.

24. The computer-implemented method according to claim 23 , wherein the first color space format and the second color space format comprise formats selected from the group consisting of: RGB, YUV, YUV 4:2:2 planar, YUV 4:2:0 planar, and YUV 4:4:4.

25. The computer-implemented method according to claim 23 , further comprising:

converting the resolution of the source image frame obtained from the first camera from a first resolution to a second resolution to produce the first intermediate image frame at the second resolution;

converting a first image size of the first intermediate image frame to a second image size to produce the second intermediate image; and

providing the second intermediate image to the first display for viewing on the first device.

26. The computer-implemented method according to claim 22 , wherein the video format is provided based on information related to on one or more display capabilities of the second display.

27. The computer-implemented method according to claim 26 , wherein the information related to a video format includes display resolution capabilities of the second display.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044334/0466 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2012
From: DODD, MICHAEL; CARCERONI, RODRIGO
To: GOOGLE INC.
Reel/Frame 029201/0641 →