IP Library Granted Patent US 10,237,319
Granted Patent B2
US 10,237,319 · App. 15/294,143 · Granted Mar 19, 2019

Capture, recording, and streaming of media content

Inventors: Brian Schmidt (Sunnyvale, CA); George Leiming Xing (Mountain View, CA); Matt Snider (Sunnyvale, CA); Sunbir Gill (San Jose, CA)
Assignee: GOOGLE LLC
H04L65/602H04L41/509H04L41/5025H04L43/106H04L65/607H04L65/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,237,319
App. No.
15/294,143
Granted
Mar 19, 2019
Kind
B2
Abstract

An electronic device screencasts media content. The screencasting includes selecting a media content layers to include in an encoded stream. An output layer is created by capturing the selected media content layers without capturing unselected layers (although these unselected layers may still be presented locally by the electronic device). The output layer is encoded into a format compatible with a media hosting service to create the encoded stream, which is transmitted to the media hosting service for presentation at a remote device.

Claims (55)

1. A method comprising:

selecting at least one layer from a plurality of layers of media content available on a local device;

creating, by a processor, an output layer by capturing the at least one layer without capturing unselected layers of the plurality of layers, wherein the output layer comprises output frames captured at an initial frame rate corresponding to a display refresh rate of the local device;

determining a desired output frame rate for the output layer;

converting the initial frame rate corresponding to the display refresh rate to the desired output frame rate;

encoding the output layer at the desired output frame rate into a format compatible with a media hosting service to create an encoded stream; and

transmitting the encoded stream to the media hosting service for presentation at a remote device.

2. The method of claim 1 , wherein the plurality of layers are video content, and the method further comprises:

selecting at least one audio layer from a plurality of audio layers; and

multiplexing the at least one audio layer and the output layer to create a multiplexed media object.

3. The method of claim 1 , wherein the plurality of layers are audio content, a first layer of the plurality of layers being speaker output, and a second layer of the plurality of layers being microphone input, and wherein creating the output layer comprises mixing the speaker output and the microphone input.

4. The method of claim 1 , wherein presentation of the encoded stream occurs substantially in real time relative presenting the plurality of layers at the local device.

5. The method of claim 1 , wherein the plurality of layers are video content, the method further comprises:

monitoring one or more feedback measures to determine whether the output frame rate is equal to the desired output frame rate.

6. The method of claim 5 , wherein input frames are added to a frame buffer, the input frames being removed from the frame buffer after use to create an output frame, and the monitoring comprises reducing a quality of the output frames responsive to identifying that the frame buffer is full.

7. The method of claim 5 , further comprising:

associating a first timestamp with a packet of the encoded stream on creation of the packet; and

associating a second timestamp with the packet on completion of transmission of the packet, wherein the monitoring comprises reducing a quality of the output frames responsive to a difference between the first timestamp and the second timestamp exceeding a threshold.

8. A system comprising:

a memory; and

a processor, operatively coupled with the memory, to:

select at least one layer from a plurality of layers of media available on a local device;

create an output layer by capturing the at least one layer without capturing unselected layers of the plurality of layers, wherein the output layer comprises output frames captured at an initial frame rate corresponding to a display refresh rate of the local device;

determine a desired output frame rate for the output layer;

convert the initial frame rate corresponding to the display refresh rate to the desired output frame rate

encode the output layer at the desired output frame rate into a format compatible with a media hosting service to create an encoded stream; and

transmit the encoded stream to the media hosting service for presentation at a remote device.

9. The system of claim 8 , wherein the processor is further to:

select at least one audio layer from a plurality of audio layers; and

multiplex the at least one audio layer and the output layer to create a multiplexed media object.

10. The system of claim 8 , wherein the plurality of layers are audio content, a first layer of the plurality of layers being speaker output, and a second layer of the plurality of layers being microphone input, and wherein creating the output layer comprises mixing the speaker output and the microphone input.

11. The system of claim 8 , wherein the plurality of layers are video content, and the processor is further to:

monitor one or more feedback measures to determine whether the output frame rate is equal to the desired output frame rate.

12. The system of claim 11 , wherein input frames are added to a frame buffer, the input frames being removed from the frame buffer after use to create an output frame, and the monitoring comprises reducing a quality of the output frames responsive to identifying that the frame buffer is full.

13. The system of claim 11 , wherein the processor is further to:

associate a first timestamp with a packet of the encoded stream on creation of the packet; and

associate a second timestamp with the packet on completion of transmission of the packet, wherein the monitoring comprises reducing a quality of the output frames responsive to a difference between the first timestamp and the second timestamp exceeding a threshold.

14. A non-transitory computer-readable storage medium storing instructions, which when executed by a processor, cause the processor to perform operations comprising:

selecting at least one layer from a plurality of layers of media available on a local device;

creating an output layer by capturing the at least one layer without capturing unselected layers of the plurality of layers, wherein the output layer comprises output frames captured at an initial frame rate corresponding to a display refresh rate of the local device;

determining a desired output frame rate for the output layer;

converting the initial frame rate corresponding to the display refresh rate to the desired output frame rate;

encoding the output layer at the desired output frame rate into a format compatible with a media hosting service to create an encoded stream; and

transmitting the encoded stream to the media hosting service for presentation at a remote device.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the plurality of layers are video content, and wherein the operations further comprise:

selecting at least one audio layer from a plurality of audio layers; and

multiplexing the at least one audio layer and the output layer to create a multiplexed media object.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the plurality of layers are audio content, a first layer of the plurality of layers being speaker output, and a second layer of the plurality of layers being microphone input, and wherein creating the output layer comprises mixing the speaker output and the microphone input.

17. The non-transitory computer-readable storage medium of claim 14 , wherein presentation of the encoded stream occurs substantially in real time relative presenting the plurality of layers at the local device.

18. The non-transitory computer-readable storage medium of claim 14 , wherein the plurality of layers are video content, and the operations further comprise:

monitoring one or more feedback measures to determine whether the output frame rate is equal to the desired output frame rate.

19. The non-transitory computer-readable storage medium of claim 18 , wherein input frames are added to a frame buffer, the input frames being removed from the frame buffer after use to create an output frame, and the monitoring comprises reducing a quality of the output frames responsive to identifying that the frame buffer is full.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

associating a first timestamp with a packet of the encoded stream on creation of the packet; and

associating a second timestamp with the packet on completion of transmission of the packet, wherein the monitoring comprises reducing a quality of the output frames responsive to a difference between the first timestamp and the second timestamp exceeding a threshold.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2017
From: SCHMIDT, BRIAN; XING, GEORGE LEIMING; SNIDER, MATT; GILL, SUNBIR
To: GOOGLE INC.
Reel/Frame 041538/0109 →
Continuity (2)
Provisional Application 62241612 · Oct 14, 2015
Related Publication 20170111421A1 · Apr 20, 2017