IP Library Granted Patent US 10,930,253
Granted Patent B2
US 10,930,253 · App. 16/570,478 · Granted Feb 23, 2021

Real-time video frame pre-processing hardware

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,930,253
App. No.
16/570,478
Granted
Feb 23, 2021
Kind
B2
Abstract

A dynamically reconfigurable heterogeneous systolic array is configured to process a first image frame, and to generate image processing primitives from the image frame, and to store the primitives and the corresponding image frame in a memory store. A characteristic of the image frame is determined. Based on the characteristic, the array is reconfigured to process a following image frame.

Claims (39)

1. A digital image acquisition and processing device, comprising:

a lens and an image sensor for acquiring digital images;

an image processing pipeline component configured to receive raw image data of an image frame and distribute corresponding formatted image data to a memory and a hardware component configured to perform image processing;

the hardware component configured to receive the formatted image data of the image frame from the image processing pipeline component at a rate of one pixel per clock cycle, perform the image processing to generate at least one scene processing primitive based at least in part on the formatted image data, and provide the at least one scene processing primitive to the memory, wherein the at least one scene processing primitive comprises at least one processed image map comprising at least one of a threshold map or an integral image map;

the memory configured to store the at least one scene processing primitive provided by the hardware component; and

a processing unit configured to process a subsequent image frame based at least in part on the at least one scene processing primitive.

2. The digital image acquisition and processing device of claim 1 , wherein the at least one scene processing primitive comprises other processed hardware component data.

3. The digital image acquisition and processing device of claim 1 , wherein the hardware component is also configured to provide formatted image data to the memory.

4. The digital image acquisition and processing device of claim 1 , wherein the at least one scene processing primitive comprises at least one of a regional primitive or a frame data primitive.

5. The digital image acquisition and processing device of claim 1 , wherein the processing unit is configured to receive formatted image data from the memory.

6. The digital image acquisition and processing device of claim 1 , further comprising a transcode component configured to:

receive processed image data from the processing unit; and

provide to or receive from the memory, at least one of:

compressed image data; or

compressed video data.

7. The digital image acquisition and processing device of claim 6 , wherein the transcode component is further configured to output, to a display, selected subsets of at least one of the compressed image data or the compressed video data.

8. A method comprising:

acquiring digital images;

receiving raw image data of an image frame and distributing corresponding formatted image data to a memory and a hardware-based image processing component;

receiving, at the hardware-based image processing component, the formatted image data of the image frame at a rate of one pixel per clock cycle, performing image processing to generate at least one scene processing primitive based at least in part on the formatted image data, and providing the at least one scene processing primitive to the memory, wherein the at least one scene processing primitive comprises at least one processed image map comprising at least one of a threshold map or an integral image map;

storing the at least one scene processing primitive provided by the hardware-based image processing component; and

processing a subsequent image frame based on the at least one scene processing primitive.

9. The method of claim 8 , wherein the at least one scene processing primitive comprises other processed hardware component data.

10. The method of claim 8 , further comprising providing formatted image data to the memory from the hardware-based image processing component.

11. The method of claim 8 , wherein the at least one scene processing primitive comprises at least one of a regional primitive or a frame data primitive.

12. The method of claim 8 , further comprising receiving formatted image data from the memory.

13. The method of claim 8 , further comprising transmitting processed image data to a transcode component and communicating at least one of compressed image data or compressed video data from the transcode component.

14. The method of claim 8 , further comprising processing the raw image data within a single clock cycle with the hardware-based image processing component.

15. A hardware-based image processing component of an image acquisition and processing device, the image acquisition and processing device configured to:

receive raw image data and distribute corresponding formatted image data to the hardware-based image processing component; and

the hardware-based image processing component configured to:

receive pixels of the formatted image data at a rate of one pixel per clock cycle and perform image processing to generate at least one scene processing primitive, based at least in part on the pixels of the formatted image data, the at least one scene processing primitive comprising processed image maps comprising one of a threshold map or integral image map;

store the at least one scene processing primitive in a memory; and

process a subsequent image frame, based at least in part on the at least one scene processing primitive.

16. The hardware-based image processing component of claim 15 , wherein the at least one scene processing primitive comprises other processed hardware component data.

17. The hardware-based image processing component of claim 15 , further configured to provide formatted image data to the memory.

18. The hardware-based image processing component of claim 15 , wherein the at least one scene processing primitive comprises at least one of a regional primitive or a frame data primitive.

19. The hardware-based image processing component of claim 15 , further configured to process image data within a single clock cycle with the hardware-based image processing component.

20. The hardware-based image processing component of claim 15 , further configured to output selected subsets of at least one of the image data or video data.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2025
From: TOBII TECHNOLOGIES LTD
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 071572/0855 →
CONVERSION Recorded Jun 12, 2025
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 071577/0875 →
SECURITY INTEREST Recorded May 28, 2025
From: ADEIA INC. (F/K/A XPERI HOLDING CORPORATION); ADEIA HOLDINGS INC.; ADEIA MEDIA HOLDINGS INC.; ADEIA IMAGING LLC; ADEIA MEDIA LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA TECHNOLOGIES INC.; ADEIA GUIDES INC.; ADEIA SOLUTIONS LLC; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR INTELLECTUAL PROPERTY LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA PUBLISHING INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 071454/0343 →
CHANGE OF NAME Recorded Mar 31, 2025
From: FOTONATION LIMITED
To: TOBII TECHNOLOGIES LIMITED
Reel/Frame 070682/0207 →
CHANGE OF NAME Recorded Feb 17, 2025
From: FOTONATION LIMITED
To: TOBII TECHNOLOGY LIMITED
Reel/Frame 070238/0774 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 17, 2019
From: BIGIOI, PETRONEL; CORCORAN, PETER; ZAHARIA, CORNELIU
To: FOTONATION LIMITED
Reel/Frame 050406/0355 →