IP Library Granted Patent US 9,609,179
Granted Patent B2
US 9,609,179 · App. 13/824,685 · Granted Mar 28, 2017

Methods for processing multimedia flows and corresponding devices

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,609,179
App. No.
13/824,685
Granted
Mar 28, 2017
Kind
B2
Abstract

A method for synchronizing of multimedia flows provided by different sources or transmitted via different transport protocols to a single receiving device or are designed to be rendered by different rendering devices. According to the invention, a reference datum representative of a datum of the first multimedia flow is provided in the second multimedia flow and associated with a datum of this second multimedia flow to synchronise the two flows. This reference datum constitutes a timing reference for the datum of the second multimedia flow with which it is associated. It is used to synchronize the two multimedia flows.

Claims (46)

1. A device for receiving multimedia data flows-comprising:

a first receiver that receives a first multimedia flow;

a second receiver that receives a second multimedia flow comprising video data,

said first multimedia flow comprising at least one first audio datum and/or video datum associated with at least one reference datum comprising a video frame and representative of a second video datum of said-second multimedia flow;

a synchronizer that synchronizes said first and said second multimedia flows by using said reference datum, said synchronizer synchronizing said first audio datum and/or video datum with a second video datum of said second multimedia flow represented by said reference datum associated with said; and

a renderer that renders said synchronized first and second multimedia flows.

2. The device according to claim 1 , wherein the synchronizing comprises comparing the video data of the second multimedia flow with the video frame comprised in said reference datum.

3. The device according to claim 1 , wherein the synchronizer calculates a signal-to-noise ratio between each of the video data of the second multimedia flow and the video frame comprised in said reference datum.

4. The device according to claim 1 , wherein said second video datum is a video frame and said video frame comprised in said reference datum is said second video datum or a video frame derived from said second video datum.

5. The device according to claim 4 , wherein said video frame derived from said second video datum comprises a frame corresponding to a low resolution image of said second video datum.

6. The device according to claim 1 , wherein said first and second multimedia flows come from different multimedia sources and/or are provided by different transport networks.

7. A device for receiving a first multimedia data flow comprising:

a capture interface that captures the rendering of a second multimedia flow;

a receiver that receives said first multimedia flow, said first multimedia flow comprising at least one first audio datum and/or video datum associated with at least one reference datum comprising a video frame and representative of a second video datum of said second multimedia flow;

a synchronizer that synchronizes the first multimedia flow with said second multimedia flow by using said reference datum, said synchronizer synchronizing said at least one first video datum and/or audio datum with a second video datum of said second multimedia flow represented by said reference datum associated with said first datum; and

a renderer that renders the synchronized first multimedia flow.

8. The device according to claim 7 , wherein the synchronizing comprises comparing the video data of the second multimedia flow with the video frame comprised in said reference datum.

9. The device according to claim 7 , wherein said second video datum is a video frame and said video frame and said video frame comprised in said reference datum is said second video datum or a video frame derived from said second video datum.

10. The device according to claim 9 , wherein said video frame derived from said second video datum is a frame corresponding to a low resolution image of said second video datum.

11. The device according to claim 7 , wherein said first and second multimedia flows come from different multimedia sources and/or are provided by different transport networks.

12. A transmitting device for generating a first multimedia data flow, said first multimedia flow comprising at least one first audio datum and/or video datum, said first multimedia flow to be synchronized with a received second multimedia flow comprising video data, the transmitting device comprising:

a receiver that receives said second multimedia,

a renderer that renders said second multimedia flow;

a first generator that generates said first multimedia flow, during said rendering of said second multimedia flow, said first datum being generated at the time of rendering of a second audio or video datum of said second multimedia flow;

a second generator that generates at least one reference datum comprising a video frame and being representative of said second rendered video datum and with which said first datum must be synchronized;

an inserter that inserts said reference datum in said first multimedia flow, said inserting comprising introducing said reference datum into said first multimedia flow and associating said reference datum with said first datum; and

a transmitter that transmits said first generated multimedia flow comprising said inserted reference datum.

13. The device according to claim 12 , wherein said second video datum is a video frame and said video frame comprised in said reference datum is said second video datum or a video frame derived from said second video datum.

14. The device according to claim 13 , wherein said video frame derived from said second video datum comprises a frame corresponding to a low resolution image of said second video datum.

15. A method for generating a first multimedia data flow in a processing device, said first multimedia flow having at least one first audio and/or video datum, said first multimedia flow being generated during a rendering of a second multimedia flow having video data, the method comprising:

generating said at least one first audio datum and/or video datum at the time of rendering of a second video datum of said second multimedia flow;

generating at least one reference datum having a video frame representative of said second rendered video datum, with which said at least one first audio datum and/or video datum must be synchronized; and

inserting said at least one reference datum in said first multimedia flow, said inserting comprising introducing said at least one reference datum into said first multimedia flow, and associating said at least one reference datum with said at least one first audio datum and/or video datum, wherein:

said second video datum of said second multimedia flow is a video frame, and said video frame comprised in said at least one reference datum is said second video datum of said second multimedia flow or a video frame derived from said second video datum of said second multimedia flow; and

said video frame derived from said second video datum of said second multimedia flow comprises a frame corresponding to a low resolution image of said second video datum of said second multimedia flow.

16. A device for rendering a first multimedia flow having at least one first audio datum and/or video datum associated with at least one reference datum, said at least one reference datum having a video frame representative of a second video datum of a second multimedia flow having video data, said device comprising:

a synchronizer that synchronizes said first and said second multimedia flows by using said at least one reference datum, said synchronizer synchronizing said at least one first audio and/or video datum of said first multimedia flow with a second video datum of said second multimedia flow represented by said video frame of said at least one reference datum associated with said at least one first audio datum and/or video datum of said first multimedia flow;

a first renderer that renders said synchronized first multimedia flow, a rendering of said at least one first audio datum and/or video datum of said first multimedia flow being synchronous with a rendering of said second video datum of said second multimedia flow; and

a converter that converts said video frame comprised in said at least one reference datum into a same resolution as the video data of the second multimedia flow.

17. A method, to be performed in a processing device, for rendering a first multimedia flow having at least one first audio and/or video datum, said method comprising:

synchronizing said at least one first audio datum and/or video datum of said first multimedia flow with a second video datum of a second multimedia flow having video data by using a reference datum of said first multimedia flow, said reference datum being associated with said at least one first audio datum and/or video datum of said first multimedia flow and comprising a video frame representative of said second video datum of said second multimedia flow, wherein the synchronizing comprises comparing the video data of the second multimedia flow with the video frame comprised in the reference datum;

rendering said synchronized first multimedia flow; and

converting said video frame comprised in said reference datum into a same resolution as the video data of the rendered second multimedia flow.

18. The method according to claim 17 , wherein said second video datum of said second multimedia flow is a video frame, and said video frame of said reference datum is said second video datum of said second multimedia flow or a video frame derived from said second video datum of said second multimedia flow.

19. The method according to claim 18 , wherein said video frame derived from said second video datum of said second multimedia flow comprises a frame corresponding to a low resolution image of said second video datum of said second multimedia flow.

20. The method according to claim 17 , wherein said first and second multimedia flows come from different multimedia sources and/or are provided by different transport networks.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 10, 2018
From: THOMSON LICENSING DTV
To: INTERDIGITAL MADISON PATENT HOLDINGS
Reel/Frame 046763/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2017
From: THOMSON LICENSING
To: THOMSON LICENSING DTV
Reel/Frame 041378/0630 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 13, 2017
From: THOMSON LICENSING
To: THOMSON LICENSING DTV
Reel/Frame 041370/0433 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 11, 2013
From: BICHOT, GUILLAUME; LAURENT, ANTHONY; LEGALLAIS, YVON
To: THOMSON LICENSING
Reel/Frame 030203/0811 →