IP Library Granted Patent US 11,954,819
Granted Patent B1
US 11,954,819 · App. 17/187,335 · Granted Apr 9, 2024

System and methods for fast and scalable 2D convolutions and cross-correlations for processing image databases and videos on CPUs

Inventors: Marios Stephanou Pattichis (Albuquerque, NM); Cesar Carranza (Miami Beach, FL); Daniel Llamocca Obregon (Clawson, MI)
Assignee: UNM RAINFOREST INNOVATIONS
G06T3/40G06T1/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,954,819
App. No.
17/187,335
Granted
Apr 9, 2024
Kind
B1
Abstract

An optimal approach for computing convolutions and cross-correlations of large databases of images that can be arbitrarily large. Throughput is maximized by breaking each image into optimal blocks and then using overlap-and-add method to compute the final result. A parallelized 2D FFT is applied over each block that runs a thread for each physical core.

Claims (19)

1. A maximum throughput method for computing 2D convolutions, comprising the steps of finding the optimal value for N that maximizes throughput:

receiving an image of arbitrarily large size, and a convolution kernel of size Q×Q;

decomposing the image into block sizes P×P;

computing fast convolutions of each image block with the convolution kernel;

storing an output in N×N blocks wherein N≥P+Q−1;

calculating the throughput as a function of a number of output pixels N×N over the execution time of each block to obtain a result;

combining results using an overlap-and-add method to compute a final output image; and

determining an optimal size N that provides the maximum throughput among all considered block sizes.

2. The method according to claim 1 , wherein the step of computing fast convolutions of each image block further comprises the steps of:

receiving an image block of size P×P and the convolution kernel of size Q×Q;

pre-allocating memory N×N;

using vector operations to zero-pad and copy the image block, the convolution kernel, and the output, wherein N≥P+Q−1;

computing a 2D FFT of the zero-padded kernel only once and storing the result for all subsequent image blocks;

computing a 2D FFT of the zero-padded image block;

computing a product of the 2D FFTs using vector multiplication operations;

computing an inverse 2D FFT of the product; and

cropping an output image block to N×N using vector operations.

3. The method according to claim 2 , wherein the pre-allocated memory is larger than a minimum required for computing linear convolutions.

4. The method according to claim 1 , wherein the image is from a large image database or a database including videos of large image sizes.

Assignments (3)
CONFIRMATORY LICENSE Recorded Feb 3, 2025
From: UNIVERSITY OF NEW MEXICO
To: NATIONAL SCIENCE FOUNDATION
Reel/Frame 070088/0384 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2023
From: PATTICHIS, MARIOS STEPHANOU
To: THE REGENTS OF THE UNIVERSITY OF NEW MEXICO
Reel/Frame 064288/0823 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2023
From: THE REGENTS OF THE UNIVERSITY OF NEW MEXICO
To: UNM RAINFOREST INNOVATIONS
Reel/Frame 064288/0890 →
Continuity (1)
Provisional Application 62983265 · Feb 28, 2020
Cited By (1)
US 12,524,637