IP Library › Granted Patent US 12,664,232
Granted Patent B2
US 12,664,232 · App. 17/949,093 · Granted Jun 23, 2026

Method and apparatus for distributed and cooperative computation in artificial neural networks

Inventors: Frederico C. Pratas (Mirandela, ES); Ayose J. Falcon (Barcelona, ES); Marc Lupon (Barcelona, ES); Fernando Latorre (Cambridge, GB); Pedro Lopez (Barcelona, ES); Enric Herrero Abellanas (Barcelona, ES); Georgios Tournavitis (Barcelona, ES)
Assignee: Intel Corporation
G06F17/153G06F12/0875G06N3/063G06F2212/1024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,664,232
App. No.
17/949,093
Granted
Jun 23, 2026
Kind
B2
Abstract

An apparatus and method are described for distributed and cooperative computation in artificial neural networks. For example, one embodiment of an apparatus comprises: an input/output (I/O) interface; a plurality of processing units communicatively coupled to the I/O interface to receive data for input neurons and synaptic weights associated with each of the input neurons, each of the plurality of processing units to process at least a portion of the data for the input neurons and synaptic weights to generate partial results; and an interconnect communicatively coupling the plurality of processing units, each of the processing units to share the partial results with one or more other processing units over the interconnect, the other processing units using the partial results to generate additional partial results or final results. The processing units may share data including input neurons and weights over the shared input bus.

Claims (53)

1 . An apparatus comprising:

an input/output (I/O) interface to a dynamic random access memory (DRAM);

a processor coupled with the I/O interface, the processor having a plurality of cores and a shared cache to be shared by the plurality of cores;

a scratchpad memory;

a plurality of processing circuits coupled with the I/O interface, the plurality of processing circuits to perform neural network (NN) processing, the plurality of processing circuits to operate at a lower clock frequency than the I/O interface, the plurality of processing circuits including a first processing circuit and a second processing circuit,

the first processing circuit to receive data for neurons and weights associated with the neurons via the I/O interface, wherein the I/O interface is to receive the weights as packed weights, the first processing circuit to process the data for the neurons and the weights associated with the neurons to generate partial results, wherein the data for the neurons and the partial results are to be stored in the scratchpad memory, which is to be shared by the plurality of processing circuits, and

the second processing circuit to process the partial results to generate additional partial results or final results; and

an interconnect coupled between the first and second processing circuits to provide the partial results from the first processing circuit to the second processing circuit.

2 . The apparatus of claim 1 , wherein the second processing circuit is communicatively coupled with the first processing circuit by a unidirectional point-to-point link via the interconnect.

3 . The apparatus of claim 1 , further comprising a plurality of latches to store the partial results prior to the partial results being provided to the second processing circuit.

4 . The apparatus of claim 1 , wherein the first processing circuit comprises a plurality of latches to store the data for the neurons.

5 . The apparatus of claim 1 , wherein the partial results are to be provided to the second processing circuit without being sent through the I/O interface to the DRAM.

6 . The apparatus of claim 1 , wherein the first processing circuit is to generate the partial results which represent only part of a layer of a neural network, and wherein the second processing circuit is to use the partial results to further process the layer.

7 . The apparatus of claim 1 , further comprising a control circuit to control providing the data to the first processing circuit.

8 . The apparatus of claim 1 , wherein the first processing circuit includes multipliers and adders to perform dot product operations.

9 . An apparatus comprising:

an input/output (I/O) interface to a dynamic random access memory (DRAM);

a processor coupled with the I/O interface, the processor having a plurality of cores and a shared cache to be shared by the plurality of cores;

a scratchpad memory;

a plurality of processing circuits coupled with the I/O interface, the plurality of processing circuits to perform neural network (NN) processing, the plurality of processing circuits to operate at a lower clock frequency than the I/O interface, the plurality of processing circuits including a first processing circuit and a second processing circuit,

the first processing circuit to receive data for neurons and weights associated with the neurons via the I/O interface, the first processing circuit to process the data for the neurons and the weights associated with the neurons to generate partial results, wherein the first processing circuit includes multipliers and adders to perform dot product operations, wherein the partial results represent only part of a layer of a neural network, and wherein the data for the neurons and the partial results are to be stored in the scratchpad memory, which is to be shared by the plurality of processing circuits, and which is closer to the plurality of processing circuits than the DRAM, and

the second processing circuit to process the partial results to generate additional partial results or final results, wherein the partial results are to be provided to the second processing circuit without being sent through the I/O interface to the DRAM, and wherein the second processing circuit is to use the partial results to further process the layer; and

an interconnect coupled between the first and second processing circuits to provide the partial results from the first processing circuit to the second processing circuit.

10 . The apparatus of claim 9 , wherein the second processing circuit is communicatively coupled with the first processing circuit by a unidirectional point-to-point link via the interconnect.

11 . The apparatus of claim 9 , wherein the first processing circuit comprises a plurality of latches to store the data for the neurons.

12 . A system comprising:

a system memory;

an input/output (I/O) interface coupled with the system memory;

a processor coupled with the I/O interface, the processor having a plurality of cores and a shared cache to be shared by the plurality of cores;

a scratchpad memory;

a plurality of processing circuits coupled with the I/O interface, the plurality of processing circuits to perform neural network (NN) processing, the plurality of processing circuits to operate at a lower clock frequency than the I/O interface, the plurality of processing circuits including a first processing circuit and a second processing circuit,

the first processing circuit to receive data for neurons and weights associated with the neurons via the I/O interface, the first processing circuit to process the data for the neurons and the weights associated with the neurons to generate partial results, wherein the data for the neurons and the partial results are to be stored in in scratchpad memory, which is to be shared by the plurality of processing circuits, and

the second processing circuit to process the partial results to generate additional partial results or final results; and

an interconnect coupled between the first and second processing circuits to provide the partial results from the first processing circuit to the second processing circuit.

13 . The system of claim 12 , wherein the second processing circuit is communicatively coupled with the first processing circuit by a unidirectional point-to-point link via the interconnect, and further comprising a communication device coupled with the processor.

14 . The system of claim 12 , further comprising circuitry to store the partial results prior to the partial results being provided to the second processing circuit, and further comprising a network controller coupled with the processor.

15 . The system of claim 12 , further comprising a plurality of latches to store the partial results prior to the partial results being provided to the second processing circuit, and further comprising an I/O device coupled with the processor.

16 . The system of claim 12 , wherein the first processing circuit comprises a plurality of latches to store the data for the neurons, and further comprising a network controller coupled with the processor.

17 . The system of claim 12 , wherein the partial results are to be provided to the second processing circuit without being sent through the I/O interface to the system memory, and wherein the system memory comprises a dynamic random access memory (DRAM).

18 . The system of claim 12 , wherein the first processing circuit is to generate the partial results which represent only part of a layer of a neural network, and wherein the second processing circuit is to use the partial results to further process the layer, and further comprising a communication device coupled with the processor.

19 . A method comprising:

exchanging data with a system memory through an input/output (I/O) interface;

processing the data with a plurality of cores of a processor;

storing data from each of the plurality of cores to a shared cache of the processor;

performing neural network (NN) processing with a plurality of processing circuits, including a first processing circuit and a second processing circuit, the NN processing to be performed at a lower clock frequency than a clock frequency used by the I/O interface for the exchanging of the data with the system memory, the NN processing including:

receiving, by the first processing circuit, data for neurons and weights associated with the neurons via the I/O interface,

processing, by the first processing circuit, the data for the neurons and the weights associated with the neurons to generate partial results,

storing the data and the partial results in a scratchpad memory closer to the plurality of processing circuits than the system memory and shared by the plurality of processing circuits, and

processing, by the second processing circuit, the partial results to generate additional partial results or final results; and

providing the partial results from the first processing circuit to the second processing circuit via an interconnect.

20 . The method of claim 19 , wherein providing the partial results from the first processing circuit to the second processing circuit is done without sending the partial results through the I/O interface to the system memory.

21 . The method of claim 19 , wherein providing the partial results from the first processing circuit to the second processing circuit via an interconnect comprises communicating the partial results by a unidirectional point-to-point link via the interconnect.

22 . The apparatus of claim 9 , wherein the I/O interface is to receive the weights as a packed weights.

Priority Claims (1)
EP 14382553 · Dec 19, 2014 · regional
Continuity (3)
Continuation 17306877 · May 3, 2021
Continuation 15521856
Related Publication 20230020571A1 · Jan 19, 2023
References Cited (74)
US 5465375A · Thepaut et al. · 1995 [cited by applicant]
US 5467428A · Ulug · 1995 [cited by applicant]
US 5600843A · Kato et al. · 1997 [cited by applicant]
US 5682491A · Pechanek et al. · 1997 [cited by applicant]
US 10078620B2 · Farabet · 2018 [cited by examiner]
US 20090192958A1 · Todorokihara · 2009 [cited by applicant]
US 20110004579A1 · Snider · 2011 [cited by applicant]
US 20110119467A1 · Cadambi · 2011 [cited by examiner]
US 20120011087A1 · Aparin · 2012 [cited by applicant]
US 20120109864A1 · Modha · 2012 [cited by applicant]
US 20120323832A1 · Snook et al. · 2012 [cited by applicant]
US 20140142929A1 · Seide et al. · 2014 [cited by applicant]
US 20140351190A1 · Levin et al. · 2014 [cited by applicant]
US 20150112911A1 · Jackson et al. · 2015 [cited by applicant]
US 20150170021A1 · Lupon et al. · 2015 [cited by applicant]
US 20150261702A1 · Culurciello · 2015 [cited by examiner]
US 20210326405A1 · Pratas et al. · 2021 [cited by applicant]
CN 1251136C · 2006 [cited by applicant]
CN 101971166A · 2011 [cited by applicant]
CN 102971754A · 2013 [cited by applicant]
CN 103201610A · 2013 [cited by applicant]
CN 103455843A · 2013 [cited by applicant]
CN 103853872A · 2014 [cited by applicant]
CN 103890787A · 2014 [cited by applicant]
EP 0482375A2 · 1992 [cited by applicant]
EP 0525543A2 · 1993 [cited by applicant]
EP 0575716A1 · 1993 [cited by applicant]
KR 1020040040075A · 2004 [cited by applicant]
TW 201338541A · 2013 [cited by applicant]
WO 9118347A1 · 1991 [cited by applicant]
WO 2014025765A2 · 2014 [cited by applicant]
Jin, Jonghoon, et al. “An efficient implementation of deep convolutional neural networks on a mobile coprocessor.” 2014 IEEE 57th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE, Aug. 2014. (Year:… [cited by examiner]
Farabet, Clément, et al. “Neuflow: A runtime reconfigurable dataflow processor for vision.” CVPR 2011 workshops. IEEE, 2011. ( Year: 2011). [cited by examiner]
Kung, “Why systolic architectures?,” in Computer, vol. 15, No. 1, Jan. 1982, pp. 37-46. [cited by applicant]
Lopez, “Programmable VLSI systolic processors for neural network and matrix computations”, PhD thesis, the Ecole Polytechnique Federale de Lausanne, and the Switzerland, 1996, 160 pages. [cited by applicant]
Request for the Submission of an Opinion, KR App. No. 10-2022-7037483, Mar. 22, 2023, 21 pages (11 pages of English Translation and 10 pages of Original Document). [cited by applicant]
Written Decision on Registration, KR App. No. 10-2017-7013506, Mar. 9, 2023, 6 pages (3 pages of English Translation and 3 pages of Original Document). [cited by applicant]
Al Maashri A et al., “A Hardware Architecture for Accelerating Neuromorphic Vision Algorithms”, Signal Processing Systems (SIPS), 2011, IEEE Workshop On, IEEE, Oct. 4, 2011, pp. 355-360. [cited by applicant]
Chen, Tianshi, et al. “Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning.” ACM SIGARCH Computer Architecture News 42.1 (2014): 269-284. (Year: 2014). [cited by applicant]
Clement Farabet et al., “Hardware accelerated convolutional neural networks for synthetic vision systems”, IEEE International Symposium on Circuits and Systems. ISCAS 2010, May 30, 2010, pp. 257-260. [cited by applicant]
Communication pursuant to Article 71(3) EPC for Application No. 14382553.7, Jun. 18, 2019, 7 pages. [cited by applicant]
Communication pursuant to Article 94(3) EPC for Application No. 14382553.7, May 31, 2018, 6 pages. [cited by applicant]
Conti Francesco et al., “Energy-Efficient Vision on the PULP Platform for Ultra-Low Power Parallel Computing”, 2014 IEEE Workshop on Signal Processing Systems (SIPS), IEEE, Oct. 20, 2014, pp. 1-6. [cited by applicant]
Decision of Rejection for foreign counterpart Taiwan Application No. 104137730, Apr. 28, 2017, 6 pages. [cited by applicant]
Decision to grant a European patent pursuant to Article 97(1) EPC, EP App. No. 14382553.7, Oct. 31, 2019, 2 pages. [cited by applicant]
Extended European Search Report for foreign counterpart European Application No. 14382553.7, Jun. 19, 2015, 7 pages. [cited by applicant]
Farabet, et al., “NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision”, IEEE Workshop of Embedded Computer Vision, Jun. 2011, pp. 109-116. [cited by applicant]
Gokhale, Vinayak, et al. “A 240 g-ops/s mobile coprocessor for deep neural networks.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014. (Year: 2014). [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2015/061453, Jun. 29, 2017, 9 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2015/061453, Feb. 26, 2016, 12 pages. [cited by applicant]
Jagdish Sabarad et al., “A Reconfigurable Accelerator for Neuromorphic Object Recognition”, Design Automation Conference (ASP-DAG), 2012 17th Asia and South Pacific, IEEE, Jan. 30, 2012, pp. 813-818. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 15/521,856, Jun. 25, 2020, 17 pages. [cited by applicant]
Notice of Allowance from foreign counterpart Taiwan Patent Application No. 104137730, Jan. 29, 2018, 4 pages. (Translation available only for Search report). [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/521,856, Jan. 13, 2021, 8 pages. [cited by applicant]
Notification to Grant Patent Right for Invention, CN App. No. 201580067143.7, Apr. 15, 2021, 8 pages (4 pages of English Translation and 4 pages of Original Document). [cited by applicant]
Office Action and Search Report for foreign counterpart Taiwan Application No. 104137730, Jan. 23, 2017, 10 pages. [cited by applicant]
Office Action, CN App. No. 201580067143.7, Mar. 4, 2020, 15 pages (8 pages of English Translation and 7 pages of Original Document). [cited by applicant]
Putnam, Andrew, et al. “A reconfigurable fabric for accelerating large-scale datacenter services.” 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA). IEEE, 2014. (Year: 2014). [cited by applicant]
Savich, Antony, and Shawki Areibi. “A low-power scalable stream compute accelerator for general matrix multiply (Gemm).” VLSI Design 2014 (2014). (Year: 2014). [cited by applicant]
Second Office Action, CN App. No. 201580067143.7, Oct. 10, 2020, 18 pages (11 pages of English Translation and 7 pages of Original Document). [cited by applicant]
Srimat Chakradhar et al. “A Dynamically Configurable Coprocessor for Convolutional Neural Networks”, Proceedings of the 37th Annual International Symposium on Computer Architecture, ISCA '10, Jun. 19, 2010, pp. 247-257. [cited by applicant]
Supplemental Notice of Allowability, U.S. Appl. No. 15/521,856, Feb. 4, 2021, 3 pages. [cited by applicant]
Third Office Action, CN App. No. 201580067143.7, Jan. 5, 2021, 20 pages (13 pages of English Translation and 7 pages of Original Document). [cited by applicant]
Office Action, KR App. No. 10-2017-7013506, Aug. 30, 2022, 20 pages (11 pages of English Translation and 9 pages of Original Document). [cited by applicant]
Rojas, Raul, “Hardware for Neural Networks”, Neural Network and a Systematic Introduction, Springer-Verlag, 1996, pp. 451-478. [cited by applicant]
Texas Instruments, “OMAP(trademark) 5 mobile application platform”, Product Bulletin, 2011, 5 pages. [cited by applicant]
Notice of Allowance, KR App. No. 10-2022-7037483, Feb. 23, 2024, 05 pages of Original Document Only. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 17/306,877, Apr. 5, 2024, 2 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 17/306,877, Feb. 27, 2024, 5 pages. [cited by applicant]
“Improved Algorithm of RBF Artificial Neural Networks and its Hardware Implementation”, 2013, 80 pages. [cited by applicant]
First Office Action , CN App. No. 202110725024.3, Feb. 18, 2024, 23 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 17/306,877, Jun. 8, 2023, 11 pages. [cited by applicant]
Office Action, KR App. No. 10-2022-7037483, Sep. 22, 2023, 5 pages (1 pages of English Translation and 4 pages of Original Document). [cited by applicant]
Second Office Action , CN App. No. 202110725024.3, Sep. 9, 2024, 22 pages (13 pages of English Translation and 09 pages of Original Document). [cited by applicant]