IP Library Granted Patent US 8,326,077
Granted Patent B2
US 8,326,077 · App. 12/262,363 · Granted Dec 4, 2012

Method and apparatus for transforming a non-linear lens-distorted image

Assignee: General Instrument Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,326,077
App. No.
12/262,363
Granted
Dec 4, 2012
Kind
B2
Abstract

A method and apparatus for image processing a lens-distorted image (e.g., a fisheye image) is provided. The method includes partitioning coordinate points in a selected output image into tiles. The output image is an undistorted rendition of a subset of the lens-distorted image. Coordinate points on a border of the tiles in the output image are selected. For each tile, coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image are calculated. In addition, for each tile, a bounding box on the lens-distorted image is selected. The bounding box includes the calculated coordinates in the lens-distorted image. The bounding boxes are expanded so that they encompass all coordinate points in the lens-distorted image that map to all coordinate points in their respective corresponding tiles. Output pixel values are generated for each tile from pixel values in their corresponding expanded bounding boxes.

Claims (34)

1. A method for image processing a lens-distorted image, the method comprising:

partitioning coordinate points in a selected output image into tiles, wherein the output image is an undistorted rendition of a subset of the lens-distorted image;

selecting coordinate points on a border of the tiles in the output image;

for each tile, calculating coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image;

for each tile, selecting a bounding box on the lens-distorted image that includes the calculated coordinates in the lens-distorted image;

expanding the bounding boxes so that they encompass all coordinate points in the lens-distorted image that map to all coordinate points in their respective corresponding tiles; and

generating output pixel values for each tile from pixel values in their corresponding expanded bounding boxes.

2. A method in accordance with claim 1 , wherein the partitioning the output image into tiles is further dependent upon at least one item selected from the group consisting of available on-chip memory, memory controller burst size, and a complexity of a memory access pattern used to generate the output pixels for each tile.

3. A method in accordance with claim 1 , wherein selecting coordinate points on the border of the tiles comprises selecting corner points on the border of the tiles.

4. A method in accordance with claim 1 , wherein selecting the bounding boxes comprises selecting maximum and minimum coordinates of the selected coordinate points for each tile in the output image.

5. A method in accordance with claim 1 , wherein expanding the bounding boxes is further dependent upon at least one item selected from the group consisting of filter size used in image processing, available on-chip memory, memory controller burst size, a complexity of a memory access pattern used to fetch pixel values from memory, and a location of the bounding box with respect to a center of lens-distortion.

6. A method in accordance with claim 1 , wherein the selected output image is selected in accordance with at least one received input pan, tilt, or zoom parameters.

7. A method in accordance with claim 1 , wherein generating the output pixels comprises:

transferring pixel values from the lens-distorted image for tile K of the output image;

calculating the coordinates of the pixels in the tile K of the output image within the coordinate system of the lens-distorted image

calculating output pixel values for tile K+1 of the output image by interpolating between pixels of the lens-distorted image for tile K+1; and

filtering the output pixel data for tile K+2 of the output image.

8. A method in accordance with claim 7 , wherein the transferring step, the calculating steps, and the filtering step are performed in parallel with one another.

9. A method in accordance with claim 7 , wherein filtering the output pixel values comprises of vertical filtering and horizontal filtering steps for consecutive tile K of the output image.

10. A method in accordance with claim 7 , wherein the pixel values comprises luminance and chrominance color channels.

11. An imaging system for providing undistorted views of a selected portion of a lens-distorted optical image, comprising:

a lens for obtaining a lens-distorted input optical image;

a processor transforming a selected portion of the lens-distorted image to produce an undistorted output image, wherein the processor is configured to perform the transformation by dividing coordinates in the output image into a plurality of tiles;

for each tile, calculating coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image; and

for each tile, selecting a bounding box on the lens-distorted image that includes the calculated coordinates in the lens-distorted image;

wherein the processor is further configured to establish the bounding boxes by:

selecting coordinate points on a border of the tiles in the output image; and wherein the processor is further configured to expand the bounding boxes so that they encompass all coordinate points in the lens-distorted input image that map to all coordinate points in their respective corresponding tiles.

12. The imaging system of claim 11 wherein the processor is further configured to generate the output pixel values in accordance with a pipelining process in which different tasks are performed simultaneously on different ones of the tiles.

13. The imaging system of claim 12 wherein the different tasks include transferring input pixel values in a bounding box to the processor, inverse mapping coordinates in each tile to coordinates in a corresponding one of the bounding boxes, and interpolating and filtering the transferred output pixel values.

14. The imaging system of claim 11 wherein the lens is a fisheye lens.

15. The imaging system of claim 11 wherein the processor is further configured to select coordinate points on the border of the tiles by selecting corner coordinate points on the border of the tiles.

16. The imaging system of claim 15 wherein the processor is further configured to select the bounding boxes by selecting maximum and minimum coordinates of the selected corner coordinate points for each tile in the output image.

17. The imaging system of claim 11 , wherein the processor is further configured to divide the coordinates in the output image into a plurality of tiles in a manner dependent upon at least one item selected from the group consisting of available on-chip memory, memory controller burst size, and a complexity of a memory access pattern used to generate the output pixels for each tile.

18. The imaging system of claim 11 , wherein the processor is further configured to establish the bounding boxes in a manner dependent upon at least one item selected from the group consisting of filter size used in image processing, available on-chip memory, memory controller burst size, a complexity of a memory access pattern to fetch pixel values from memory, and a location of the bounding box with respect to a center of lens-distortion.

Assignments (13)
RELEASE OF SECURITY INTEREST AT REEL/FRAME 049905/0504 Recorded Dec 19, 2024
From: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
To: ARRIS ENTERPRISES LLC (F/K/A ARRIS ENTERPRISES, INC.); ARRIS TECHNOLOGY, INC.; ARRIS SOLUTIONS, INC.; COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; RUCKUS WIRELESS, LLC (F/K/A RUCKUS WIRELESS, INC.)
Reel/Frame 071477/0255 →
SECURITY INTEREST Recorded Dec 17, 2024
From: ARRIS ENTERPRISES LLC; COMMSCOPE TECHNOLOGIES LLC; COMMSCOPE INC., OF NORTH CAROLINA; OUTDOOR WIRELESS NETWORKS LLC; RUCKUS IP HOLDINGS LLC
To: APOLLO ADMINISTRATIVE AGENCY LLC
Reel/Frame 069889/0114 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2022
From: ARRIS TECHNOLOGY, INC.
To: ARRIS ENTERPRISES, INC.
Reel/Frame 060791/0583 →
SECURITY INTEREST Recorded Nov 19, 2021
From: ARRIS SOLUTIONS, INC.; ARRIS ENTERPRISES LLC; COMMSCOPE TECHNOLOGIES LLC; COMMSCOPE, INC. OF NORTH CAROLINA; RUCKUS WIRELESS, INC.
To: WILMINGTON TRUST
Reel/Frame 060752/0001 →
TERM LOAN SECURITY AGREEMENT Recorded Jul 3, 2019
From: COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; ARRIS ENTERPRISES LLC; ARRIS TECHNOLOGY, INC.; RUCKUS WIRELESS, INC.; ARRIS SOLUTIONS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 049905/0504 →
ABL SECURITY AGREEMENT Recorded Jul 3, 2019
From: COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; ARRIS ENTERPRISES LLC; ARRIS TECHNOLOGY, INC.; RUCKUS WIRELESS, INC.; ARRIS SOLUTIONS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 049892/0396 →
PATENT SECURITY AGREEMENT Recorded Jul 3, 2019
From: ARRIS ENTERPRISES LLC
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 049820/0495 →
CHANGE OF NAME Recorded Jul 2, 2019
From: ARRIS ENTERPRISES, INC.
To: ARRIS ENTERPRISES LLC
Reel/Frame 049649/0062 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Apr 8, 2019
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: ARRIS GROUP, INC.; ARRIS ENTERPRISES, INC.; ARRIS SOLUTIONS, INC.; ARRIS KOREA, INC.; ARRIS HOLDINGS CORP. OF ILLINOIS, INC.; BIG BAND NETWORKS, INC.; TEXSCAN CORPORATION; POWER GUARD, INC.; 4HOME, INC.; ACADIA AIC, INC.; AEROCAST, INC.; BROADBUS TECHNOLOGIES, INC.; GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT AUTHORIZATION SERVICES, INC.; GENERAL INSTRUMENT INTERNATIONAL HOLDINGS, INC.; IMEDIA CORPORATION; JERROLD DC RADIO, INC.; LEAPSTONE SYSTEMS, INC.; MODULUS VIDEO, INC.; MOTOROLA WIRELINE NETWORKS, INC.; NETOPIA, INC.; NEXTLEVEL SYSTEMS (PUERTO RICO), INC.; QUANTUM BRIDGE COMMUNICATIONS, INC.; SETJAM, INC.; SUNUP DESIGN SYSTEMS, INC.; UCENTRIC SYSTEMS, INC.; GIC INTERNATIONAL HOLDCO LLC; GIC INTERNATIONAL CAPITAL LLC; CCE SOFTWARE LLC; THE GI REALTY TRUST 1996
Reel/Frame 048825/0294 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2015
From: ARRIS TECHNOLOGY, INC
To: ARRIS ENTERPRISES, INC.
Reel/Frame 037328/0341 →
MERGER AND CHANGE OF NAME Recorded Mar 10, 2015
From: GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT CORPORATION
To: ARRIS TECHNOLOGY, INC.
Reel/Frame 035176/0620 →
SECURITY AGREEMENT Recorded May 28, 2013
From: ARRIS GROUP, INC.; ARRIS ENTERPRISES, INC.; ARRIS SOLUTIONS, INC.; ARRIS KOREA, INC.; ARRIS HOLDINGS CORP. OF ILLINOIS; BIGBAND NETWORKS, INC.; TEXSCAN CORPORATION; POWER GUARD, INC.; 4HOME, INC.; ACADIA AIC, INC.; AEROCAST, INC.; BROADBUS TECHNOLOGIES, INC.; GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT AUTHORIZATION SERVICES, INC.; GENERAL INSTRUMENT INTERNATIONAL HOLDINGS, INC.; IMEDIA CORPORATION; JERROLD DC RADIO, INC.; LEAPSTONE SYSTEMS, INC.; MODULUS VIDEO, INC.; MOTOROLA WIRELINE NETWORKS, INC.; NETOPIA, INC.; NEXTLEVEL SYSTEMS (PUERTO RICO), INC.; QUANTUM BRIDGE COMMUNICATIONS, INC.; SETJAM, INC.; SUNUP DESIGN SYSTEMS, INC.; UCENTRIC SYSTEMS, INC.; GIC INTERNATIONAL HOLDCO LLC; GIC INTERNATIONAL CAPITAL LLC; CCE SOFTWARE LLC; THE GI REALTY TRUST 1996
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 030498/0023 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 21, 2008
From: CHAI, SEK M.; DWYER, MALCOLM R.; LIN, RUEI-SUNG; LINZMEIER, DANIEL A.; BELLAS, NIKOLAOS
To: GENERAL INSTRUMENT CORPORATION
Reel/Frame 021873/0840 →
Continuity (1)
Related Publication 20100111440A1 · May 6, 2010