IP Library Granted Patent US 11,244,431
Granted Patent B2
US 11,244,431 · App. 16/657,103 · Granted Feb 8, 2022

Image processing

Inventors: Alexey Kornienko (Cambridge, GB); Vladimir Dolzhenko (Cambridge, GB); Nerhun Yildiz (Cambridge, GB)
Assignee: Apical Limited
G06T5/006G06T3/0018G06T7/70G06T2207/10016H04N5/23238
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,244,431
App. No.
16/657,103
Granted
Feb 8, 2022
Kind
B2
Abstract

A method including streaming input image data representing an input frame of a video into temporary storage. Transformation data representing at least one transformation for application to the input image data is obtained. Ordering data indicating a variable order in which portions of output image data are to be generated is obtained, the ordering data being based on at least one characteristic of the input frame. The transformation data is used to identify portions of input image data to be processed to generate corresponding portions of output image data. The identified portions of input image data are processed to generate the corresponding portions of output image data in the order indicated by the ordering data.

Claims (37)

1. A method comprising:

streaming input image data representing an input frame of a video in parts into temporary storage;

obtaining ordering data indicating an order in which portions of output image data are to be generated, the ordering data being based on at least one characteristic of the input frame that affects a number of vertical pixel locations in the input frame required to generate a given portion of output image data, the at least one characteristic of the input frame including at least one of: a type of geometric distortion in at least part of the input frame, a degree of geometric distortion in at least part of the input frame, or an orientation of at least part of the input frame;

obtaining transformation data representing at least one transformation for application to the input image data to adjust a geometric distortion of the input frame;

using the transformation data, identifying portions of input image data to be processed to generate corresponding portions of output image data representing respective output locations in an output frame; and

processing the identified portions of input image data to generate the corresponding portions of output image data in the order indicated by the ordering data.

2. The method of claim 1 , wherein the processing is performed while the identified portions of input image data are stored in the temporary storage.

3. The method of claim 1 , wherein the input image data representing the input frame is larger than a capacity of the temporary storage.

4. The method of claim 1 , wherein the at least one transformation includes at least one of: a panoramic transformation, a stereographic projection, an equidistant projection, an equisolid projection, a cylindrical projection, or an orthographic projection.

5. The method of claim 1 , comprising:

obtaining frame characteristic data indicating at least one characteristic of the input frame; and

generating the ordering data based on at least the frame characteristic data.

6. The method of claim 1 , wherein the portions of output image data are stored as a plurality of tiles, each tile comprising a plurality of portions of output image data representing a respective plurality of output locations.

7. The method of claim 6 , wherein the ordering data indicates a variable order in which tiles of the plurality of tiles are to be generated.

8. The method of claim 7 , wherein the variable order indicated by the ordering data is dependent on a size of at least one tile of the plurality of tiles.

9. The method of claim 6 , comprising determining the size of at least one tile of the plurality of tiles based on at least a capacity of the temporary storage.

10. The method of claim 7 , wherein determining the size of the at least one tile of the plurality of tiles is based on one or more characteristics of the input frame including at least one of: an orientation of at least part of the input frame, a type of geometric distortion in at least part of the input frame, or a degree of geometric distortion in at least part of the input frame.

11. The method of claim 1 , wherein the input image data comprises pixel data representing a plurality of pixel locations in the input frame and identifying which portion of input image data is to be processed to generate a corresponding portion of output image data comprises, for a said portion of output image data:

using at least part of the transformation data, calculating a position of an input location in the input frame which corresponds to an output location to be represented by the said portion of output image data; and

identifying pixel data representing a set of pixel locations in the input frame which define a region in the input frame which includes the input location.

12. The method of claim 11 , wherein calculating a position of the input location in the input frame which corresponds to the output location comprises performing an interpolation based on:

the position of the output location in the output frame; and

a function representing a transformation of the at least one transformation.

13. The method of claim 12 , wherein the interpolation is a spline interpolation.

14. The method of claim 11 , wherein the output location corresponds to a pixel location in the output frame.

15. The method of claim 11 , wherein at least one identified portion of input image data comprises pixel data representing a plurality of pixel locations in the input frame and generating a corresponding portion of output data comprises calculating at least one output pixel value representing a respective output location in the output frame using the pixel data.

16. The method of claim 15 , wherein calculating the at least one output pixel value representing the respective output location comprises interpolating based on the pixel data.

17. The method of claim 16 , wherein interpolating based on the pixel data comprises using at least one polyphase filter.

18. Image processing apparatus comprising:

temporary storage for storing input image data representing an input frame of a video; and

at least one processor, wherein

the image processing apparatus is configured to:

stream input image data representing an input frame of a video in parts into the temporary storage;

obtain ordering data indicating an order in which portions of output image data are to be generated, the ordering data being based on at least one characteristic of the input frame that affects a number of vertical pixel locations in the input frame required to generate a given portion of output image data, the at least one characteristic of the input frame including at least one of: a type of geometric distortion in at least part of the input frame, a degree of geometric distortion in at least part of the input frame, or an orientation of at least part of the input frame;

obtain transformation data representing at least one transformation for application to the input image data to adjust a geometric distortion of the input frame;

use the transformation data, identify portions of input image data to be processed to generate corresponding portions of output image data representing respective output locations in an output frame; and

process the identified portions of input image data to generate corresponding portions of output image data in the order indicated by the ordering data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2022
From: APICAL LIMITED
To: ARM LIMITED
Reel/Frame 061300/0471 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 3, 2020
From: KORNIENKO, ALEXEY; DOLZHENKO, VLADIMIR; YILDIZ, NERHUN
To: APICAL LIMITED
Reel/Frame 051408/0597 →
Continuity (1)
Related Publication 20210118104A1 · Apr 22, 2021