IP Library Granted Patent US 10,182,236
Granted Patent B2
US 10,182,236 · App. 16/004,112 · Granted Jan 15, 2019

Field programmable object array having image processing circuitry

Inventors: Dean Stuart Susnow (Portland, OR); Richard D. Reohr, Jr. (Hillsboro, OR)
Assignee: NYTELL SOFTWARE LLC
H04N19/433H03K19/17732H04N19/176H04N19/182H04N19/423H04N19/43H04N19/51H04N19/523H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,182,236
App. No.
16/004,112
Granted
Jan 15, 2019
Kind
B2
Abstract

A field programmable object array integrated circuit has video data compression capability. The integrated circuit comprises an array of programmable objects and a video compression co-processor communicatively coupled to the array of objects. The video compression co-processor comprises a set of search engines and a subpixel engine. The subpixel engine can interpolate subpixels from integer pixels and shift the integer pixels by a predetermined number of subpixels. The search engines can perform a plurality of sum of absolute differences (SAD) computations between search window pixels and macroblock pixels to locate the best SAD value using either integer pixels and/or the interpolated subpixels.

Claims (43)

1. A device, comprising:

a direct memory access engine configured to route first search window pixel data from a memory to a video compression processor via a first bus of two multi-bit periphery buses supporting data flow in a first direction and a second bus of the two multi-bit periphery buses supporting data flow in a second direction; and

communication circuitry configured to route second search window pixel data from the memory to the video compression processor via multiple channels of an array of programmable gates at a higher transfer rate relative to the direct memory access engine, wherein the communication circuitry is further configured to transfer at least a portion of the second search widow pixel data via the multiple channels during a single clock cycle.

2. The device of claim 1 , wherein the direct memory access engine is configured to, in response to initiation of a subpixel interpolation operation of an application process executed by the array of programmable gates, route the first search window pixel data via the first bus gates.

3. The device of claim 1 , wherein the communication circuitry is configured to, in response to initiation of an integer search operation of an application process executed by the array of programmable gates, route the second search window pixel data via the multiple channels of the array of programmable gates at the higher transfer rate relative to the direct memory access engine.

4. The device of claim 1 , wherein

the first search window pixel data and the second search window pixel data correspond to the search window pixels, and

the video compression processor comprises a search engine configured to search the search window pixels within a defined portion of a video frame.

5. The device of claim 4 , wherein

the video compression processor comprises a subpixel engine configured to interpolate subpixels from integer pixels,

the video compression processor is configured to shift the integer pixels by a defined number of subpixels, and

the search engine is further configured to select at least some of the search window pixels from the integer pixels or the subpixels and to select baseline pixels from integer baseline pixels or subpixel baseline pixels.

6. The device of claim 5 , wherein the subpixels comprise at least two of half-sampled subpixels, quarter-sampled subpixels, or one-eighth sampled subpixels.

7. The device of claim 5 , wherein the search engine is further configured to determine a sum of absolute differences between corresponding pairs of the baseline pixels and the search window pixels, to determine

distances between the search window pixels and a reference point, and

a coordinate location of the sum of absolute differences.

8. The device of claim 7 , wherein the search engine is further configured to send, to the array of programmable gates, information identifying the sum of absolute differences, the distances, and the coordinate location.

9. The device of claim 4 , wherein the video compression processor comprises search engines, including the search engine, that are configured to perform subpixel searching with respective different pixel strides.

10. The device of claim 1 , wherein

at least one of the first search window pixel data or the second search window pixel data represents pixel information from search window pixels being searched within a defined portion of a video frame,

the video compression processor comprises at least two logical memories, and

the video compression processor is configured to

store the at least one of the first search window pixel data or the second search window pixel data in the at least two logical memories,

determine a sum of absolute differences between corresponding pairs of baseline pixels and the search window pixels based on the at least one of the first search window pixel data or the second search window pixel data stored in one of the at least two logical memories, and

store additional search window pixel data other than the first search window pixel data and the second search window pixel data in at least one other logical memory of the at least two logical memories.

11. The device of claim 1 , wherein the communication circuitry comprises a bus interface to a unidirectional segmented bus.

12. A circuit, comprising:

a memory communicatively coupled to an array of programmable gates via communication circuitry; and

a video compression processor is configured to:

collect search window pixel data from the memory via the communication circuitry and at least one of the programmable gates in response to initiation of a first type operation of an application process; and

collect the search window pixel data from the memory via a data transfer engine in response to initiation of a second type of operation of the application process,

wherein at least a portion of the search window pixel data is configured to be transferred via multiple channels of the communication circuitry during a given clock cycle, and

wherein the data transfer engine is configured to transfer the search window pixel data at a lower transfer rate than the communication circuitry via two multi-bit periphery buses that bypass the array of programmable gates and respectively facilitate two different data flow directions.

13. The circuit of claim 12 , wherein the first type of operation is an integer search operation of the application process.

14. The circuit of claim 12 , wherein the second type of operation is a subpixel interpolation operation of the application process.

15. The circuit of claim 12 , wherein the communication circuitry comprises a bus interface to a unidirectional segmented bus.

16. The circuit of claim 12 , wherein the video compression processor is further configured to identify search window pixels from the search window pixel data based, at least in part, on a determined degree of similarity between the search window pixels and baseline pixels as a function of a comparison of pixel values of the search window pixels and corresponding pixel values of the baseline pixels.

17. The circuit of claim 12 , wherein the video compression processor is further configured to identify search window pixels from the search window pixel data based on a minimization of a distance between the search window pixels and a reference position.

18. A method, comprising:

in response to determining that an application process is performing a first type of operation, transferring, by a system comprising a processor, search window pixel data, from a memory of an array of programmable gates to a video compression processor via multiple channels, per clock cycle, of a circuit that connects the memory to the video compression processor via the array of programmable gates; and

in response to determining that the application process is performing a second type of operation, transferring the search window pixel data from the array of programmable gates to the video compression processor via two multi-bit periphery buses associated with a direct memory access engine without routing the search window pixel data via the array of programmable gates, wherein the two multi-bit periphery buses support clockwise data flow around the array of programmable gates and counterclockwise data flow around the array of programmable gates, respectively.

19. The method of claim 18 , wherein the first type of operation is an integer search operation, and the second type of operation is a subpixel interpolation operation.

20. The method of claim 18 , wherein the search window pixel data represents pixel data of pixels being searched within a defined portion of a video frame.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2018
From: SAJAN, INC.
To: OLK GRUN GMBH, LLC
Reel/Frame 046833/0252 →
MERGER Recorded Sep 10, 2018
From: OLK GRUN GMBH, LLC
To: NYTELL SOFTWARE LLC
Reel/Frame 046833/0275 →
Continuity (5)
Continuation 15481566 · Apr 7, 2017
Continuation 15233919 · Aug 10, 2016
Continuation 12492033 · Jun 25, 2009
Provisional Application 61082779 · Jul 22, 2008
Related Publication 20180295374A1 · Oct 11, 2018