IP Library Granted Patent US 10,042,819
Granted Patent B2
US 10,042,819 · App. 15/280,903 · Granted Aug 7, 2018

Convolution accelerators

Inventors: Miao Hu (Palo Alto, CA); John Paul Strachan (San Carlos, CA); Naveen Muralimanohar (Santa Clara, CA)
Assignee: Hewlett Packard Enterprise Development LP
G06F17/14G06F17/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,042,819
App. No.
15/280,903
Granted
Aug 7, 2018
Kind
B2
Abstract

Examples herein relate to convolution accelerators. An example convolution accelerator may include a transformation crossbar array programmed to calculate a Fourier Transformation of a first vector with a transformation matrix and a Fourier Transformation of a second vector with the transformation matrix. A circuit of the example convolution accelerator may multiply the Fourier Transformation of the first vector with the Fourier Transformation of the second vector to calculate a product vector. The example convolution accelerator may have an inverse transformation crossbar array programmed to calculate an Inverse Fourier Transformation of the product vector according to an inverse transformation matrix.

Claims (55)

1. A convolution accelerator, comprising:

a transformation crossbar array programmed to calculate a Fourier Transformation of a first vector with a transformation matrix and a Fourier Transformation of a second vector with the transformation matrix;

a circuit to multiply the Fourier Transformation of the first vector with the Fourier Transformation of the second vector to calculate a product vector; and

an inverse transformation crossbar array programmed to calculate an Inverse Fourier Transformation of the product vector according to an inverse transformation matrix,

wherein the transformation crossbar array comprises a plurality of row lines, a plurality of column lines, and a memory cell coupled between each unique combination of one row line and one column line, wherein the memory cells are programmed according to the transformation matrix; and

the inverse transformation crossbar array comprises a plurality of row lines, a plurality of column lines, and a memory cell coupled between each unique combination of one row line and one column line, wherein the memory cells are programmed according to the inverse transformation matrix.

2. The convolution accelerator of claim 1 , wherein the memory cells of the transformation crossbar array are programmed to an array of resistances corresponding to the transformation matrix, and the memory cells of the inverse transformation crossbar array are programmed to an array of resistances corresponding to the inverse transformation matrix.

3. The convolution accelerator of claim 2 , wherein a set of analog voltages corresponding to the first vector is delivered to the transformation crossbar array to calculate the Fourier Transformation of the first vector, and a set of analog values corresponding to the second vector is delivered to the transformation crossbar array to calculate the Fourier Transformation of the second vector.

4. The convolution accelerator of claim 2 , wherein each memory cell of the transformation crossbar array and each memory cell of the inverse transformation crossbar array comprises a memristor.

5. The convolution accelerator of claim 1 , wherein:

the transformation crossbar array is to calculate a Fourier Transformation of a third vector with the transformation matrix and a Fourier Transformation of a fourth vector with the transformation matrix;

the circuit is to multiply the Fourier Transformation of the third vector with the Fourier Transformation of the fourth vector to calculate a second product vector; and

the inverse transformation crossbar array is to calculate an Inverse Fourier Transformation of the second product vector according to the inverse transformation matrix.

6. The convolution accelerator of claim 1 , wherein the first vector is a part of an image signal comprising the first vector, and the second vector is a part of a kernel signal comprising the second vector.

7. The convolution accelerator of claim 5 , further comprising a circuit to concatenate the Inverse Fourier Transformation of the product vector and the Inverse Fourier Transformation of the second product vector.

8. The convolution accelerator of claim 1 , wherein the circuit to calculate the product vector comprises:

an analog-to-digital converter to convert the Fourier Transformation of the first vector to a digitized Fourier Transformation of the first vector and convert the Fourier Transformation of the second vector to a digitized Fourier Transformation of the second vector;

a digital multiplier circuit to multiply the digitized Fourier Transformation of the first vector with the digitized Fourier Transformation of the second vector to calculate a digitized product vector; and

a digital-to-analog converter to convert the digital product vector to the product vector, wherein the product vector comprises analog values.

9. A circuit, comprising:

a first transformation crossbar array programmed to calculate a Fourier Transformation of a first vector with a transformation matrix,

a second transformation crossbar array programmed to calculate a Fourier Transformation of a second vector with the transformation matrix, wherein each of the first transformation crossbar and the second transformation crossbar array comprises a plurality of row lines, a plurality of column lines, and a memory cell coupled between each unique combination of one row line and one column line, wherein the memory cells are programmed according to an array of resistances corresponding to the transformation matrix;

a digital circuit to multiply the Fourier Transformation of the first vector with the Fourier Transformation of the second vector to calculate a product vector; and

an inverse transformation crossbar array programmed to calculate an Inverse Fourier Transformation of the product vector according to an inverse transformation matrix, wherein the inverse transformation crossbar array comprises a plurality of row lines, a plurality of column lines, and a memory cell coupled between each unique combination of one row line and one column line, wherein the memory cells are programmed to an array of resistances corresponding to the inverse transformation matrix.

10. The circuit of claim 9 , wherein a set of analog voltages corresponding to the first vector is delivered to the first transformation crossbar array to calculate the Fourier Transformation of the first vector, and a set of analog values corresponding to the second vector is delivered to the second transformation crossbar array to calculate the Fourier Transformation of the second vector.

11. The circuit of claim 9 , wherein the first vector is a part of an image signal comprising the first vector, and the second vector is a part of a kernel signal comprising the second vector.

12. The circuit of claim 9 , wherein:

the first transformation crossbar array is to calculate a Fourier Transformation of a third vector with the transformation matrix;

the second transformation crossbar array is to calculate a Fourier Transformation of a fourth vector with the transformation matrix;

the digital circuit is to multiply the Fourier Transformation of the third vector with the Fourier Transformation of the fourth vector to calculate a second product vector; and

the inverse transformation crossbar array is to calculate an Inverse Fourier Transformation of the second product vector according to the inverse transformation matrix.

13. The circuit of claim 12 , further comprising a circuit to concatenate the Inverse Fourier Transformation of the product vector and the Inverse Fourier Transformation of the second product vector.

14. The circuit of claim 9 , wherein the digital circuit to calculate the product vector comprises:

an analog-to-digital converter to convert the Fourier Transformation of the first vector to a digitized Fourier Transformation of the first vector and convert the Fourier Transformation of the second vector to a digitized Fourier Transformation of the second vector;

a digital multiplier circuit to multiply the digitized Fourier Transformation of the first vector with the digitized Fourier Transformation of the second vector to calculate a digitized product vector; and

a digital-to-analog converter to convert the digital product vector to the product vector, wherein the product vector comprises analog values.

15. A method, comprising:

receiving, by a transformation crossbar array programmed to calculate Fourier Transformations, a first set of analog voltages corresponding to a first vector;

calculating, by the transformation crossbar array, a Fourier Transformation of the first vector with a transformation matrix;

receiving, by the transformation crossbar array, a second set of analog voltages corresponding to a second vector;

calculating, by the transformation crossbar array, a Fourier Transformation of the second vector with the transformation matrix;

multiplying, by a circuit, the Fourier Transformation of the first vector with the Fourier Transformation of the second vector to calculate a product vector;

calculating, by an inverse transformation crossbar array programmed to calculate Inverse Fourier Transformations, an Inverse Fourier Transformation of the product vector according to an inverse transformation matrix;

calculating, by the transformation crossbar array, a Fourier Transformation of a third vector with the transformation matrix and a Fourier Transformation of a fourth vector with the transformation matrix;

multiplying, by the circuit, the Fourier Transformation of the third vector with the Fourier Transformation of the fourth vector to calculate a second product vector;

calculating, by the inverse transformation crossbar array, an Inverse Fourier Transformation of the second product vector according to the inverse transformation matrix; and

concatenating, by a second circuit, the Inverse Fourier Transformation of the product vector and the Inverse Fourier Transformation of the second product vector.

16. The method of claim 15 , wherein the first vector and the third vector are parts of an image signal comprising the first vector and the third vector, and the second vector and the fourth vector are parts of a kernel signal comprising the second vector and the fourth vector.

17. The method of claim 15 , wherein:

the transformation crossbar array comprises a plurality of row lines, a plurality of column lines, and a memory cell coupled between each unique combination of one row line and one column line, wherein the memory cells are programmed according to an array of resistances corresponding to the transformation matrix; and

the inverse transformation crossbar array comprises a plurality of row lines, a plurality of column lines, and a memory cell coupled between each unique combination of one row line and one column line, wherein the memory cells are programmed to an array of resistances corresponding to the inverse transformation matrix.

18. The method of claim 15 , wherein the circuit to calculate the product vector comprises:

an analog-to-digital converter to convert the Fourier Transformation of the first vector to a digitized Fourier Transformation of the first vector and convert the Fourier Transformation of a second vector to a digitized Fourier Transformation of the second vector;

a digital multiplier circuit to multiply the digitized Fourier Transformation of the first vector with the digitized Fourier Transformation of the second vector to calculate a digitized product vector; and

a digital-to-analog converter to convert the digital product vector to the product vector, wherein the product vector comprises analog values.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: HU, MIAO; STRACHAN, JOHN PAUL; MURALIMANOHAR, NAVEEN
To: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
Reel/Frame 039900/0692 →
Continuity (1)
Related Publication 20180089146A1 · Mar 29, 2018
Cited By (1)
US 12,626,136