IP Library Granted Patent US 11,044,437
Granted Patent B2
US 11,044,437 · App. 16/840,231 · Granted Jun 22, 2021

Method and system for combining multiple area-of-interest video codestreams into a combined video codestream

Inventors: Rudolf O. Ernst (Leesburg, VA); Rahul C. Thakkar (Leesburg, VA)
Assignee: PIXIA CORP.
H04N7/0117G06K9/00771G06T3/4038G06T11/60H04L65/602H04N5/23238H04N7/18H04N7/24H04N19/115H04N19/167H04N19/172H04N21/2365G06T2207/10016G09G2340/02G09G2340/04G09G2340/0435G09G2360/122
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,044,437
App. No.
16/840,231
Granted
Jun 22, 2021
Kind
B2
Abstract

A method and system of transmitting a plurality of area-of-interest video codestreams is described. A first video codestream and one or more second video codestreams are generated from a plurality of large format images that are captured. The first video codestream has a first plurality of areas-of-interest selected from the plurality of large format images and the one or more second video codestream have at least a second plurality of areas-of-interest from the same plurality of large format images. The first video codestream is generated at a first frame rate and each of the second video codestreams is generated at a second frame rate. The first and second video codestreams are combined to obtain a combined video codestream. The combined video codestream is then transmitted to a computer system that regenerates the first video codestream and the one or more second video codestreams at their respective frame rates.

Claims (29)

1. A method of providing a combined video codestream based on multiple areas-of-interest video codestreams from a plurality of image sources, the method comprising:

receiving, by a computer system, a plurality of requests from one or more client computer systems, each request including parameters indicating:

a collection of images that includes a corresponding area-of-interest of the areas-of-interest; and

an extent of the corresponding one of the areas-of-interest;

generating, by the computer system, a plurality of video codestreams, each video codestream corresponding to a respective one of the requests and comprising each corresponding area-of-interest over its respective extent wherein a first video codestream of the plurality of codestreams is generated from a plurality of image frames from a first image source of the plurality of image sources, the first video codestream comprising a first plurality of areas-of-interest selected from the plurality of image frames, each area-of-interest in the first plurality of areas-of-interest of the first video codestream being selected from a different image frame in the plurality of image frames, the first video codestream having a first frame rate and a second video codestream of the plurality of codestreams is generated from the plurality of image frames from a second, different, image source of the plurality of image sources, each of the one or more second video codestreams having a second frame rate and comprising a second plurality of areas-of-interest selected from the plurality of image frames, each area-of-interest in the second plurality of areas-of-interest of the respective second video codestream being selected from a different image frame in the plurality of image frames, the first video codestream and each of the one or more second video codestream being independent of each other such that information represented in the first video codestream is different from information represented in the second video codestream;

multiplexing the plurality of video codestreams to produce a multiplexed video codestream comprising a version of the first video codestream and a version of the second video codestream wherein each of each of the first plurality of areas-of-interest of the first video codestream and each of the second pluralities of areas-of-interest of the second video code stream have a size substantially equal to a size of a display device of each respective requesting client computer system; and

transmitting the multiplexed video codestream to the one or more client computer systems,

wherein each collection of images comprises a plurality of WAMI images and the WAMI images are large format images of at least 10,000 by 9,600 pixels, and each area-of-interest comprises a portion of the large format image smaller than the large format image.

2. A method as in claim 1 , wherein the extent of the areas-of-interest comprises a duration.

3. A method as in claim 1 , wherein the extent of the areas-of-interest comprises an image size having fewer pixels than images in the collection of images.

4. A method as in claim 1 , wherein the extent of the areas-of-interest comprises both an image size having fewer pixels than images in the collection of images and a start and end time for the images.

5. A method as in claim 1 , wherein one or more of the areas-of-interest is continuously updated, and wherein the generating, multiplexing, and transmitting includes updating a corresponding video codestream with continuously updated images from the one or more continuously updated areas-of-interest.

6. A method as in claim 5 , wherein a frame rate at which the continuously updated one or more of the areas-of-interest is continuously updated is different from a frame rate at which the corresponding video codestream is updated.

7. A method as in claim 1 , wherein the multiplexed video codestream includes metadata selected from the group consisting of: audio, close captioning, key length values, or combinations thereof.

8. A method as in claim 1 , wherein at least one video codestream of the plurality of video codestreams has a frame rate different from at least one other video code stream of the plurality of video codestreams.

9. A method as in claim 8 , wherein a total bandwidth of the multiplexed video codestream is smaller than a sum of bit rates of the plurality of video codestreams and further comprising:

modifying a bit rate of at least one of the video codestreams prior to the multiplexing based on the total bandwidth of the multiplexed video codestream.

10. A method as in claim 9 , wherein one or more of the plurality of requests includes priority information, and wherein the modifying comprises allocating a larger portion of the bandwidth to requests having a higher priority.

11. A method as in claim 1 , wherein transmitting the multiplexed video codestream comprises transmitting, via a network, the multiplexed video codestream to a demultiplexer at a location remote from a location of the computer system.

12. A method as in claim 1 , wherein the parameters include one or more spatial parameters related to the extent of a corresponding one or more of the areas-of-interest, and wherein the method further comprises adjusting, by the computer system, based on the one or more spatial parameters, a pixel location of the corresponding areas-of-interest during the transmission of the combined video codestream.

13. A method as in claim 12 , wherein the spatial parameters comprise parameters derived from a plurality of directional instructions available to a user on the one or more client computers.

14. A system for providing a combined video codestream based on multiple areas-of-interest video codestreams from a plurality of image sources, the method comprising:

one or more processors of a computer system programmed to execute one or more computer program instructions that, when executed, cause the one or more processors to:

receive, at the computer system, a plurality of requests from one or more client computer systems, each request including parameters indicating:

a collection of images that includes a corresponding area-of-interest of the areas-of-interest, wherein each collection of images comprises a plurality of WAMI images and the WAMI images are large format images of at least 10,000 by 9,600 pixels, and each area-of-interest comprises a portion of the large format image smaller than the large format image; and

an extent of the corresponding one of the areas-of-interest;

generate, by the computer system, a plurality of video codestreams, each video codestream corresponding to a respective one of the requests and comprising each corresponding area-of-interest over its respective extent wherein a first video codestream of the plurality of codestreams is generated from a plurality of image frames from a first image source of the plurality of image sources, the first video codestream comprising a first plurality of areas-of-interest selected from the plurality of image frames, each area-of-interest in the first plurality of areas-of-interest of the first video codestream being selected from a different image frame in the plurality of image frames, the first video codestream having a first frame rate and a second video codestream of the plurality of codestreams is generated from the plurality of image frames from a second, different, image source of the plurality of image sources, each of the one or more second video codestreams having a second frame rate and comprising a second plurality of areas-of-interest selected from the plurality of image frames, each area-of-interest in the second plurality of areas-of-interest of the respective second video codestream being selected from a different image frame in the plurality of image frames, the first video codestream and each of the one or more second video codestream being independent of each other such that information represented in the first video codestream is different from information represented in the second video codestream;

multiplex the plurality of video codestreams to produce a multiplexed video codestream comprising a version of the first video codestream and a version of the second video codestream wherein each of each of the first plurality of areas-of-interest of the first video codestream and each of the second pluralities of areas-of-interest of the second video code stream have a size substantially equal to a size of a display device of each respective requesting client computer system; and

transmit the multiplexed video codestream to the one or more client computer systems.

Assignments (9)
SUPERPRIORITY PATENT SECURITY AGREEMENT Recorded Oct 6, 2025
From: CUBIC CORPORATION; CUBIC DEFENSE APPLICATIONS INC.; CUBIC DIGITAL INTELLIGENCE INC.; CUBIC ITS, INC.; CUBIC SECURE COMMUNICATIONS, LLC; CUBIC TOTAL LEARNING PLATFORM, LLC; CUBIC TRANSPORTATION SYSTEMS, INC.; GATR TECHNOLOGIES INC.; NUVOTRONICS INC.
To: BARCLAYS BANK PLC
Reel/Frame 073008/0761 →
RELEASE OF SECURITY INTEREST Recorded Jul 30, 2025
From: ALTER DOMUS (US) LLC
To: CUBIC CORPORATION; CUBIC DEFENSE APPLICATIONS, INC.; CUBIC DIGITAL INTELLIGENCE, INC.
Reel/Frame 072278/0272 →
RELEASE OF SECURITY INTEREST Recorded Jul 30, 2025
From: ALTER DOMUS (US) LLC
To: CUBIC CORPORATION; CUBIC DIGITAL SOLUTIONS LLC; NUVOTRONICS, INC.
Reel/Frame 072281/0176 →
RELEASE OF SECURITY INTEREST AT REEL/FRAME 056393/0281 Recorded Jul 28, 2025
From: BARCLAYS BANK PLC, AS ADMINISTRATIVE AGENT
To: CUBIC CORPORATION; CUBIC DEFENSE APPLICATIONS, INC.; CUBIC DIGITAL SOLUTIONS LLC (FORMERLY PIXIA CORP.)
Reel/Frame 072282/0124 →
SECURITY INTEREST Recorded May 2, 2025
From: CUBIC DEFENSE APPLICATIONS, INC.; CUBIC DIGITAL INTELLIGENCE, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 071161/0299 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 18, 2024
From: PIXIA CORP.
To: CUBIC DIGITAL INTELLIGENCE, INC.
Reel/Frame 066816/0198 →
SECOND LIEN SECURITY AGREEMENT Recorded May 26, 2021
From: CUBIC CORPORATION; PIXIA CORP.; NUVOTRONICS, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 056393/0314 →
FIRST LIEN SECURITY AGREEMENT Recorded May 26, 2021
From: CUBIC CORPORATION; PIXIA CORP.; NUVOTRONICS, INC.
To: BARCLAYS BANK PLC
Reel/Frame 056393/0281 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2020
From: ERNST, RUDOLF O.; THAKKAR, RAHUL C.
To: PIXIA CORP.
Reel/Frame 052311/0139 →