IP Library Granted Patent US 11,734,006
Granted Patent B2
US 11,734,006 · App. 17/868,438 · Granted Aug 22, 2023

Deep vision processor

Inventors: Wajahat Qadeer (Los Altos, CA); Rehan Hameed (Los Altos, CA)
Assignee: Deep Vision, Inc.
G06F9/3013G06F9/3001G06F9/30036G06F9/30112G06F9/30134G06F17/16G06N3/045G06N3/063G06N3/08G06T15/005G06T1/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,734,006
App. No.
17/868,438
Granted
Aug 22, 2023
Kind
B2
Abstract

Disclosed herein is a processor for deep learning. In one embodiment, the processor comprises: a load and store unit configured to load and store image pixel data and stencil data; a register unit, implementing a banked register file, configured to: load and store a subset of the image pixel data from the load and store unit, and concurrently provide access to image pixel values stored in a register file entry of the banked register file, wherein the subset of the image pixel data comprises the image pixel values stored in the register file entry; and a plurality of arithmetic logic units configured to concurrently perform one or more operations on the image pixel values stored in the register file entry and corresponding stencil data of the stencil data.

Claims (58)

1. A processor comprising:

a banked register file configured to store image pixel data;

an interconnect unit configured to access:

multiple subsets of the image pixel data from the banked register file; and

corresponding stencil data for the multiple subsets of image pixel data; and

a set of arithmetic logic units configured to concurrently execute a set of one-dimensional stencil operations on subsets of the image pixel data and corresponding stencil data, the set of one-dimensional stencil operations representing a multi-dimensional stencil operation.

2. The processor of claim 1 , wherein the set of arithmetic logic units is configured to concurrently execute a set of smaller convolution operations on each subset of the image pixel data and corresponding stencil data, the set of smaller convolution operations:

characterized by a set of smaller kernel sizes; and

representing a larger convolution operation on the image pixel data characterized by a larger kernel size.

3. The processor of claim 2 , wherein the set of arithmetic logic units is configured to concurrently execute the set of smaller convolution operations on each subset of the image pixel data and the corresponding stencil data, the set of smaller convolution operations:

representing the larger convolution operation on the image pixel data characterized by the larger kernel size; and

characterized by the set of smaller kernel sizes selected from a group of kernel sizes comprising:

a three-by-three kernel size;

a four-by-four kernel size;

a one-by-three kernel size;

a three-by-one kernel size;

a one-by-four kernel size; and

a four-by-one kernel size.

4. The processor of claim 2 , wherein the set of arithmetic logic units is configured to concurrently execute the set of smaller convolution operations on each subset of the image pixel data and the corresponding stencil data, the set of smaller convolution operations representing the larger convolution operation on the image pixel data characterized by the larger kernel size selected from a group of kernel sizes comprising:

a five-by-five kernel size;

a seven-by-seven kernel size;

an eight-by-eight kernel size;

a one-by-five kernel size;

a one-by-seven kernel size;

a one-by-eight kernel size;

a five-by one kernel size;

a seven-by-one kernel size; and

an eight-by-one kernel size.

5. The processor of claim 1 , wherein the banked register file represents a two-dimensional register abstraction.

6. The processor of claim 1 , wherein the multi-dimensional stencil operation comprises a two-dimensional stencil operation.

7. A processor comprising:

a shift register unit:

configured to store image pixel data; and

representing a multi-dimensional register abstraction;

a mapping unit in communication with the shift register unit:

configured to concurrently access multiple subsets of the image pixel data from the shift register unit based on a multi-dimensional stencil pattern within the multi-dimensional register abstraction;

configured to concurrently access corresponding stencil data for each subset of the image pixel data in the multiple subsets of the image pixel data; and

comprising a set of arithmetic logic units configured to concurrently execute a set of operations on each subset of the image pixel data and the corresponding stencil data, the set of operations representing a stencil operation characterized by the multi-dimensional stencil pattern.

8. The processor of claim 7 , wherein the mapping unit is configured to concurrently access the multiple subsets of the image pixel data, each subset of the image pixel data defining a smaller stencil pattern in a set of smaller stencil patterns, the set of smaller stencil patterns combining to define the multi-dimensional stencil pattern.

9. The processor of claim 7 :

wherein the set of arithmetic logic units is further configured to generate a set of outputs resulting from the set of operations; and

further comprising a reduction unit, configured to combine the set of outputs to generate a reduced output of the stencil operation.

10. The processor of claim 9 :

wherein the set of arithmetic logic units is configured to concurrently execute a set of multiplication operations to generate the set of outputs; and

wherein the reduction unit is configured to execute a summation operation on the set of outputs to generate the reduced output.

11. The processor of claim 9 , wherein the set of arithmetic logic units are configured to concurrently execute a set of subtraction operations to generate the set of outputs.

12. The processor of claim 9 :

wherein the set of arithmetic logic units are configured to concurrently execute a set of absolute difference operations to generate the set of outputs; and

wherein the reduction unit is configured to execute a summation operation on the set of outputs to generate the reduced output.

13. The processor of claim 9 , wherein the set of arithmetic logic units are configured to concurrently execute a set of averaging operations to generate the set of outputs.

14. The processor of claim 9 , further comprising a set of accumulator registers configured to accumulate a set of reduced outputs comprising the reduced output to execute the stencil operation.

15. The processor of claim 9 , further comprising an output register implemented as a shift register and configured to access the set of outputs from the mapping unit.

16. The processor of claim 7 , further comprising a single-input-multiple-data unit configured to execute general purpose computations.

17. A register unit of a processor core:

comprising a set of register banks;

configured to store image pixel data within the set of register banks;

representing a multi-dimensional register abstraction; and

configured to provide access to multiple subsets of the image pixel data in the set of register banks, the multiple subsets of the image pixel data defining a multi-dimensional stencil pattern within the multi-dimensional register abstraction.

Assignments (2)
MERGER Recorded Jun 11, 2026
From: KINARA, INC.
To: NXP USA, INC.
Reel/Frame 074926/0187 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2022
From: QADEER, WAJAHAT; HAMEED, REHAN
To: DEEP VISION, INC.
Reel/Frame 061141/0902 →
Continuity (5)
Continuation 17356372 · Jun 23, 2021
Continuation 16670800 · Oct 31, 2019
Continuation 16026480 · Jul 3, 2018
Provisional Application 62528796 · Jul 5, 2017
Related Publication 20220357946A1 · Nov 10, 2022
Cited By (1)
US 12,423,222