IP Library Granted Patent US 10,659,721
Granted Patent B2
US 10,659,721 · App. 13/794,715 · Granted May 19, 2020

Method of processing a sequence of coded video frames

Inventors: Muralidharan Narayanan (Bangalore, IN); Christopher S. Del Sordo (Souderton, PA); Surya P. Maheswaram (Hydernager, IN); Krishna Prasad Panje (Bangalore, IN); Ernest G. Schmitt (Maple Glen, PA); Yaxi Zhang (Wayne, PA)
Assignee: ARRIS Enterprises LLC
H04N5/956H04N19/196H04N19/70H04N21/4147H04N21/4307
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,659,721
App. No.
13/794,715
Granted
May 19, 2020
Kind
B2
Abstract

A method of processing a sequence of coded video frames conveyed by a digital data stream, where each frame represents an image, includes receiving the sequence of coded video frames at a recording device, determining a frame interval between presentation of an ith coded frame of the sequence and an (i+1)th coded frame of the sequence, calculating a stream time stamp for the ith coded frame, and calculating a stream time stamp for the (i+1)th coded video frame based on the stream time stamp for the ith coded video frame and the previously determined frame interval.

Claims (59)

1. A method of processing a sequence of coded video frames conveyed by a digital data stream, where each frame represents an image, comprising:

a. receiving the sequence of coded video frames at a recording device, and recording the sequence of coded video frames,

b. determining a frame interval between presentation of an i th coded frame of the sequence and an (i+1) th coded frame of the sequence,

c. at the recording device, calculating a stream time stamp for the i th coded frame, wherein the calculated stream time stamp increases monotonically throughout the sequence of coded video frames as it is recorded, and

d. at the recording device, calculating a stream time stamp for the (i+1) th coded video frame based on the stream time stamp for the i th coded video frame and the frame interval determined in step b;

the method further comprising:

e. initializing a frame interval variable (Int_fr) with a frame interval value based on a nominal frame rate;

f. receiving a first frame and reading the presentation time stamp value of the frame, assigning the presentation time stamp value of the first frame to a presentation time stamp variable (PTS), assigning a value of PTS to a clock start variable (Clock_start) for representing the first frame's clock time based on presentation time stamp, assigning a value of PTS to a clock presentation time stamp variable (Clock_pts) for representing a frame's clock time based on presentation time stamp, and assigning a value of zero to a stream time stamp variable (ST*) for representing a stream time stamp of interest;

g. receiving a next frame and reading the presentation time stamp value of said next frame, assigning the presentation time stamp value of said next frame to PTS, assigning the updated value of PTS to the variable Clock_pts, and assigning a value (ST*+Int_fr) to the variable ST*; and

h. testing whether there is a discontinuity in presentation time stamp value between said next frame and the previous frame and, if so, assigning the updated value of ST* to a second stream time stamp variable (ST_dis*) for representing the value of the stream time stamp of interest at a point of discontinuity in presentation time stamp and assigning the value of PTS to the variable Clock_start and storing said next frame in a database using the value of ST* as an index.

2. A method according to claim 1 , comprising:

e. saving the (i+1) th coded video frame and the stream time stamp for the (i+1) th coded video frame in a relational database.

3. A method according to claim 2 , wherein each coded video frame includes a data field specifying a presentation time stamp for the frame, step b. comprises determining an improvised frame interval based on presentation time stamps for successive frames of the sequence, and step e. comprises saving the presentation time stamp and the stream time stamp in an index file and saving the coded video frame in a content file.

4. A method according to claim 3 , comprising reading a stream offset position from the index file based on a stream time stamp and a presentation time stamp and reading a coded video frame from the content file based on the stream offset position.

5. A method of modifying a digital data stream composed of a sequence of frames, where each frame represents an image and includes a data field specifying a presentation time stamp value for the frame, comprising:

a. initializing a frame interval variable (Int_fr) with a frame interval value based on a nominal frame rate;

b. receiving a first frame and reading the presentation time stamp value of the frame, assigning the presentation time stamp value of the first frame to a presentation time stamp variable (PTS), assigning a value of PTS to a clock start variable (Clock_start) for representing the first frame's clock time based on presentation time stamp, assigning a value of PTS to a clock presentation time stamp variable (Clock_pts) for representing a frame's clock time based on presentation time stamp, and assigning a value of zero to a stream time stamp variable (ST*) for representing a stream time stamp of interest;

c. receiving a next frame and reading the presentation time stamp value of said next frame, assigning the presentation time stamp value of said next frame to PTS, assigning the updated value of PTS to the variable Clock_pts, and assigning a value (ST*+Int_fr) to the variable ST*; and

d. testing whether there is a discontinuity in presentation time stamp value between said next frame and the previous frame and, if so, assigning the updated value of ST* to a second stream time stamp variable (ST_dis*) for representing the value of the stream time stamp of interest at a point of discontinuity in presentation time stamp and assigning the value of PTS to the variable Clock_start and storing said next frame in a database using the value of ST* as an index.

6. A method according to claim 5 , wherein the sequence of frames is composed of P frames, I frames and B frames,

wherein step a. further comprises initializing a minimum interval variable (Int_min) with a minimum frame interval value based on said nominal frame rate,

wherein step c. further comprises, prior to assigning the updated value of PTS to the variable Clock_pts, assigning a value (PTS−Clock_pts) to a difference variable (Diff), and

wherein the method further comprises, if there is no discontinuity in presentation time stamp value between said next frame and said previous frame,

testing whether said next frame is a P frame or an I frame and, if so, assigning a value (Clock_pts−Clock_start+ST_dis*) to the variable ST* if the value of Diff is greater than the value of Int_min.

7. A method according to claim 5 , wherein the sequence of frames is composed of P frames, I frames and B frames, and the method further comprises, if there is no discontinuity in presentation time stamp value between said next frame and said previous frame, testing whether said next frame is a B frame.

8. A method according to claim 7 , further comprising, if said next frame is not a B frame, resetting a B frame flag false.

9. A method according to claim 7 , further comprising, if said next frame is a B frame, testing whether a B frame flag is set true.

10. A method according to claim 9 , further comprising, if said B frame flag is not set true, setting said B frame flag true.

11. A method according to claim 9 ,

wherein step a. further comprises assigning a value to a tolerance variable (TOL),

wherein step c. further comprises, prior to assigning the updated value of PTS to the variable Clock_pts, assigning a value (PTS−Clock_pts) to a difference variable (Diff),

wherein step d. further comprises testing whether the absolute value of Diff exceeds the value of TOL, and

wherein the method further comprises, if said B frame flag is set true, assigning the value of Diff to Int_fr and assigning the value (3*Int_fr) to the variable TOL if the value of Diff is greater than the value of Int_min.

12. A video recording device for recording a sequence of coded video frames conveyed by a digital data stream, where each frame represents an image, the video recording device having an input terminal for receiving the digital data stream and comprising a computing machine programmed to perform the steps of:

a. determining a frame interval between presentation of an i th coded frame of the sequence and an (i+l) th coded frame of the sequence,

b. calculating a stream time stamp for the i th coded frame, wherein the calculated stream time stamp increases monotonically throughout the sequence of coded video frames as it is recorded, and

c. calculating a stream time stamp for the (i+1) th coded video frame based on the stream time stamp for the i th coded video frame and the frame interval determined in step b;

the steps further comprising:

d. initializing a frame interval variable (Int_fr) with a frame interval value based on a nominal frame rate;

e. receiving a first frame and reading the presentation time stamp value of the frame, assigning the presentation time stamp value of the first frame to a presentation time stamp variable (PTS), assigning a value of PTS to a clock start variable (Clock_start) for representing the first frame's clock time based on presentation time stamp, assigning a value of PTS to a clock presentation time stamp variable (Clock_pts) for representing a frame's clock time based on presentation time stamp, and assigning a value of zero to a stream time stamp variable (ST*) for representing a stream time stamp of interest;

f. receiving a next frame and reading the presentation time stamp value of said next frame, assigning the presentation time stamp value of said next frame to PTS, assigning the updated value of PTS to the variable Clock_pts, and assigning a value (ST*+Int_fr) to the variable ST*; and

g. testing whether there is a discontinuity in presentation time stamp value between said next frame and the previous frame and, if so, assigning the updated value of ST* to a second stream time stamp variable (ST_dis*) for representing the value of the stream time stamp of interest at a point of discontinuity in presentation time stamp and assigning the value of PTS to the variable Clock_start and storing said next frame in a database using the value of ST* as an index.

13. A video recording device according to claim 12 , wherein the computing machine is programmed to perform the further step of:

d. saving the (i+1) th coded video frame and the stream time stamp for the (i+1) th coded video frame in a relational database.

14. A video recording device according to claim 13 , wherein each coded video frame includes a data field specifying a presentation time stamp for the frame, step a. performed by the computing machine comprises determining an improvised frame interval based on presentation time stamps for successive frames of the sequence, and step d. comprises saving the presentation time stamp and the stream time stamp in an index file and saving the coded video frame in a content file.

15. A video recording device according to claim 14 , wherein the computing machine is programmed to read a stream offset position from the index file based on a stream time stamp and a presentation time stamp and to read a coded video frame from the content file based on the stream offset position.

16. A non-transitory computer-readable storage medium containing software which, when read and executed by a computing machine receiving a sequence of coded video frames conveyed by a digital data stream, where each frame represents an image, causes the computing machine to perform a method for recording the sequence, the method comprising:

a. determining a frame interval between presentation of an i th coded frame of the sequence and an (i+1) th coded frame of the sequence,

b. calculating a stream time stamp for the i th coded frame, wherein the calculated stream time stamp increases monotonically throughout the sequence of coded video frames as it is recorded, and

c. calculating a stream time stamp for the (i+1) th coded video frame based on the stream time stamp for the i th coded video frame and the frame interval determined in step a;

the method further comprising:

d. initializing a frame interval variable (Int_fr) with a frame interval value based on a nominal frame rate;

e. receiving a first frame and reading the presentation time stamp value of the frame, assigning the presentation time stamp value of the first frame to a presentation time stamp variable (PTS), assigning a value of PTS to a clock start variable (Clock_start) for representing the first frame's clock time based on presentation time stamp, assigning a value of PTS to a clock presentation time stamp variable (Clock_pts) for representing a frame's clock time based on presentation time stamp, and assigning a value of zero to a stream time stamp variable (ST*) for representing a stream time stamp of interest;

f. receiving a next frame and reading the presentation time stamp value of said next frame, assigning the presentation time stamp value of said next frame to PTS, assigning the updated value of PTS to the variable Clock_pts, and assigning a value (ST*+Int_fr) to the variable ST*; and

g. testing whether there is a discontinuity in presentation time stamp value between said next frame and the previous frame and, if so, assigning the updated value of ST* to a second stream time stamp variable (ST_dis*) for representing the value of the stream time stamp of interest at a point of discontinuity in presentation time stamp and assigning the value of PTS to the variable Clock_start and storing said next frame in a database using the value of ST* as an index.

17. A computer-readable storage medium according to claim 16 , wherein the software causes the computing machine is programmed to perform the further step of:

d. saving the (i+1) th coded video frame and the stream time stamp for the (i+1) th coded video frame in a relational database.

18. A computer-readable storage medium according to claim 17 , wherein each coded video frame includes a data field specifying a presentation time stamp for the frame, and the software causes the computing machine to perform a method in which step a. comprises determining an improvised frame interval based on presentation time stamps for successive frames of the sequence, and step d. comprises saving the presentation time stamp and the stream time stamp in an index file and saving the coded video frame in a content file.

19. A computer-readable storage medium according to claim 18 , wherein the software causes the computing machine to perform a method in which the computing machine reads a stream offset position from the index file based on a stream time stamp and a presentation time stamp and reads a coded video frame from the content file based on the stream offset position.

Assignments (12)
RELEASE OF SECURITY INTEREST AT REEL/FRAME 049905/0504 Recorded Dec 19, 2024
From: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
To: ARRIS ENTERPRISES LLC (F/K/A ARRIS ENTERPRISES, INC.); ARRIS TECHNOLOGY, INC.; ARRIS SOLUTIONS, INC.; COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; RUCKUS WIRELESS, LLC (F/K/A RUCKUS WIRELESS, INC.)
Reel/Frame 071477/0255 →
SECURITY INTEREST Recorded Dec 17, 2024
From: ARRIS ENTERPRISES LLC; COMMSCOPE TECHNOLOGIES LLC; COMMSCOPE INC., OF NORTH CAROLINA; OUTDOOR WIRELESS NETWORKS LLC; RUCKUS IP HOLDINGS LLC
To: APOLLO ADMINISTRATIVE AGENCY LLC
Reel/Frame 069889/0114 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2022
From: ARRIS TECHNOLOGY, INC.
To: ARRIS ENTERPRISES, INC.
Reel/Frame 060791/0583 →
SECURITY INTEREST Recorded Nov 19, 2021
From: ARRIS SOLUTIONS, INC.; ARRIS ENTERPRISES LLC; COMMSCOPE TECHNOLOGIES LLC; COMMSCOPE, INC. OF NORTH CAROLINA; RUCKUS WIRELESS, INC.
To: WILMINGTON TRUST
Reel/Frame 060752/0001 →
TERM LOAN SECURITY AGREEMENT Recorded Jul 3, 2019
From: COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; ARRIS ENTERPRISES LLC; ARRIS TECHNOLOGY, INC.; RUCKUS WIRELESS, INC.; ARRIS SOLUTIONS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 049905/0504 →
PATENT SECURITY AGREEMENT Recorded Jul 3, 2019
From: ARRIS ENTERPRISES LLC
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 049820/0495 →
ABL SECURITY AGREEMENT Recorded Jul 3, 2019
From: COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; ARRIS ENTERPRISES LLC; ARRIS TECHNOLOGY, INC.; RUCKUS WIRELESS, INC.; ARRIS SOLUTIONS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 049892/0396 →
CHANGE OF NAME Recorded Jul 2, 2019
From: ARRIS ENTERPRISES, INC.
To: ARRIS ENTERPRISES LLC
Reel/Frame 049649/0062 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Apr 8, 2019
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: ARRIS GROUP, INC.; ARRIS ENTERPRISES, INC.; ARRIS INTERNATIONAL LIMITED; ARRIS TECHNOLOGY, INC.; ARCHIE U.S. MERGER LLC; ARCHIE U.S. HOLDINGS LLC; ARRIS GLOBAL SERVICES, INC.; ARRIS HOLDINGS CORP. OF ILLINOIS, INC.; ARRIS SOLUTIONS, INC.; BIG BAND NETWORKS, INC.; TEXSCAN CORPORATION; POWER GUARD, INC.; JERROLD DC RADIO, INC.; NEXTLEVEL SYSTEMS (PUERTO RICO), INC.; GIC INTERNATIONAL HOLDCO LLC; GIC INTERNATIONAL CAPITAL LLC
Reel/Frame 050721/0401 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2015
From: ARRIS TECHNOLOGY, INC
To: ARRIS ENTERPRISES, INC.
Reel/Frame 037328/0341 →
SECURITY INTEREST Recorded Jun 26, 2015
From: ARRIS GROUP, INC.; ARRIS ENTERPRISES, INC.; ARRIS INTERNATIONAL LIMITED; ARRIS TECHNOLOGY, INC.; ARCHIE U.S. MERGER LLC; ARCHIE U.S. HOLDINGS LLC; ARRIS GLOBAL SERVICES, INC.; ARRIS HOLDINGS CORP. OF ILLINOIS, INC.; ARRIS SOLUTIONS, INC.; BIG BAND NETWORKS, INC.; TEXSCAN CORPORATION; POWER GUARD, INC.; JERROLD DC RADIO, INC.; NEXTLEVEL SYSTEMS (PUERTO RICO), INC.; GIC INTERNATIONAL HOLDCO LLC; GIC INTERNATIONAL CAPITAL LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 036020/0789 →
MERGER AND CHANGE OF NAME Recorded Mar 10, 2015
From: GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT CORPORATION
To: ARRIS TECHNOLOGY, INC.
Reel/Frame 035176/0620 →
Continuity (2)
Continuation 12973257 · Dec 20, 2010
Related Publication 20140099079A1 · Apr 10, 2014