IP Library Granted Patent US 10,474,464
Granted Patent B2
US 10,474,464 · App. 16/026,480 · Granted Nov 12, 2019

Deep vision processor

Inventors: Wajahat Qadeer (Cupertino, CA); Rehan Hameed (Palo Alto, CA)
Assignee: DEEP VISION, INC.
G06F9/3013G06F9/3001G06F17/16G06N3/0454G06N3/063G06N3/08G06T15/005G06T1/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,474,464
App. No.
16/026,480
Granted
Nov 12, 2019
Kind
B2
Abstract

Disclosed herein is a processor for deep learning. In one embodiment, the processor comprises: a load and store unit configured to load and store image pixel data and stencil data; a register unit, implementing a banked register file, configured to: load and store a subset of the image pixel data from the load and store unit, and concurrently provide access to image pixel values stored in a register file entry of the banked register file, wherein the subset of the image pixel data comprises the image pixel values stored in the register file entry; and a plurality of arithmetic logic units configured to concurrently perform one or more operations on the image pixel values stored in the register file entry and corresponding stencil data of the stencil data.

Claims (31)

1. A processor comprising:

a load and store unit configured to load and store image pixel data and stencil data;

a register unit, implementing a banked register file comprising a plurality of registers included in respective banks of registers, the register unit configured to:

load and store a subset of the image pixel data from the load and store unit; and

concurrently provide access to image pixel values stored in a register file entry of the banked register file, wherein the subset of the image pixel data comprises the image pixel values stored in the register file entry,

wherein the stencil data comprises a stencil associated with a stencil size, the stencil size being implemented via one or more smaller stencil sizes of a plurality of smaller stencil sizes, wherein the banked register file stores the subset of the image pixel data in a group of registers of the plurality of registers selected based on the one or more smaller stencil sizes;

an interconnect unit in communication with the register unit, the interconnect unit configured to:

provide the image pixel values stored in the register file entry; and

provide corresponding stencil data to the image pixel values stored in the register file entry; and

a plurality of arithmetic logic units (ALUs) in communication with the interconnect configured to concurrently perform one or more operations on the image pixel values stored in the register file entry and the corresponding stencil data to the image pixel value is stored in the register file entry from the interconnect unit.

2. The processor of claim 1 , wherein registers from multiple banks combine to form a two-dimensional register, wherein one register from each bank comprises a 1-dimensional row of the two-dimensional register, and wherein the group of registers comprise registers in one or more of the 1-dimensional rows.

3. The processor of claim 1 , wherein the banks of registers are banks of vector registers, and wherein a width of each bank of vector registers is a size of one register file entry of the banked register file.

4. The processor of claim 1 , wherein the banks of registers are banks of vector registers, and wherein the banks of vector registers comprise four banks of vector registers.

5. The processor of claim 4 , wherein the four banks of registers are configured to implement 32 1-row 1D vector registers, 16 2-row 2D vector registers, 8 4-row, 2D vector registers, or a combination thereof.

6. The processor of claim 1 , wherein the processor is configured to implement the plurality of smaller stencil instructions using the banked register file.

7. The processor of claim 6 , wherein the plurality of smaller stencil instructions comprises a 3×3 Stencil2D instruction, a 4×4 Stencil2D instruction, a 1×3 Stencil1D instruction, a 1×4 Stencil1D instruction, a 3×1 Stencil1D instruction, a 4×1 Stencil1D instruction, or a combination thereof.

8. The processor of claim 7 , wherein the plurality of smaller stencil instructions comprises 1×1 Stencil instruction implemented using the 3×1 Stencil1D instruction, the 4×1 Stencil1D instruction, or a combination thereof.

9. The processor of claim 6 , wherein the processor is configured to implement a plurality of larger stencil operations using the plurality of smaller stencil instructions.

10. The processor of claim 9 , wherein the plurality of larger stencil operations comprises a 5×5 Stencil2D operation, a 7×7 Stencil2D operation, a 8×8 Stencil2D operation, a 1×5 Stencil1D operation, a 1×7 Stencil1D operation, a 1×8 Stencil1D operation, a 5×1 Stencil1D operation, a 7×1 Stencil1D operation, a 8×1 Stencil1D operation, or a combination thereof.

11. The processor of claim 9 , wherein the plurality of larger stencil instructions comprises an n×1 Stencil1D instruction or a 1×n Stencil1D instruction, wherein n is a positive integer.

12. The processor of claim 9 , wherein the plurality of larger stencil instructions comprises an n×m Stencil2D instruction, wherein n and m are positive integers.

13. The processor of claim 1 , wherein the interconnect unit is configured to provide 3×3 image pixel values of the image pixel values stored in the register file entry.

14. The processor of claim 13 , wherein the processor is configured to accumulate x×y image pixel values from 3×3 image pixel values received from the interconnect unit, wherein x and y are positive integers.

15. The processor of claim 1 , wherein the processor is configured to implement a DOTV2R instruction using the banked register file.

16. The processor of claim 1 , wherein the register unit is configured to: load and store results of the ALUs.

17. The processor of claim 1 , further comprising a plurality of accumulator registers of an accumulator register file configured to: load and store results of the ALUs.

18. A register unit of a processor core, implementing a banked register file, the banked register file comprising a plurality of registers included in respective banks of registers, and the register unit being configured to:

load and store a subset of image pixel data and associated stencil data; and

concurrently provide access to image pixel values stored in a register file entry of the banked register file, wherein the subset of the image pixel data comprises the image pixel values stored in the register file entry, wherein the stencil data comprises a stencil associated with a stencil size, the stencil size being implemented via one or more smaller stencil sizes of a plurality of smaller stencil sizes, wherein the banked register file stores the subset of the image pixel data in a group of registers of the plurality of registers selected based on the one or more smaller stencil sizes.

19. The register unit of claim 18 , wherein the banks of registers form a two-dimensional register, wherein each bank of registers comprises a 1-dimensional column of the two-dimensional register, and wherein the group of registers comprise registers included in one or more of the 1-dimensional columns.

20. The register unit of claim 18 , wherein the banks of registers are banks of vector registers, and wherein the banks of vector registers comprise four banks of vector registers.

Assignments (2)
MERGER Recorded Jun 11, 2026
From: KINARA, INC.
To: NXP USA, INC.
Reel/Frame 074926/0187 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2019
From: QADEER, WAJAHAT; HAMEED, REHAN
To: DEEP VISION, INC.
Reel/Frame 050374/0060 →
Continuity (2)
Provisional Application 62528796 · Jul 5, 2017
Related Publication 20190012170A1 · Jan 10, 2019