IP Library › Granted Patent US 12,266,031
Granted Patent B2
US 12,266,031 · App. 17/242,392 · Granted Apr 1, 2025

Memory mapping of activations for convolutional neural network executions

Inventors: Martino Dazzi (Zurich, CH); Pier Andrea Francese (Adliswil, CH); Abu Sebastian (Adliswil, CH); Evangelos Stavros Eleftheriou (Rueschlikon, CH)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06T1/60G06N3/063G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,266,031
App. No.
17/242,392
Granted
Apr 1, 2025
Kind
B2
Abstract

A memory controller circuit for mapping data of a convolutional neural network to a physical memory is disclosed. The memory controller circuit comprises a receiving unit to receive a selection parameter value, and a mapping unit to map pixel values of one layer of the convolutional neural network to memory words of the physical memory according to one of a plurality of mapping schemas, wherein the mapping is dependent on the value of the received selection parameter value.

Claims (24)

1. A memory controller circuit for mapping data of a convolutional neural network to a physical memory, said memory controller circuit comprising:

a receiving unit to receive a selection parameter value; and

a mapping unit to map pixel values of one layer of said convolutional neural network to memory words of said physical memory according to one of a plurality of mapping schemas, wherein said mapping is dependent on said value of said received selection parameter value, and wherein the one of the plurality of mapping schemas is an intra-word adjacent mapping schema, and wherein bit values of portions of different pixel values are mapped to one physical memory word in a contiguous manner.

2. The memory controller circuit according to claim 1 , wherein the one of the plurality of mapping schemas is a kernel word interleaving schema, wherein bit values of portions of different pixel values are mapped to said physical memory such that at most a number of pixels equal to a size of a kernel of a convolutional neural network operation is mapped contiguously to adjacent memory words.

3. The memory controller circuit according to claim 1 , wherein the one of the plurality of mapping schemas is an activation word interleaving schema, wherein bit values of pixel values are mapped such that a memory word comprises the bit values of at most one pixel.

4. The memory controller circuit according to claim 1 , wherein said selection parameter value is comprised in a set of parameter values of a layer of said convolutional neural network.

5. The memory controller circuit according to claim 1 , wherein said selection parameter value is independent of said parameter values of a layer of said convolutional neural network.

6. The memory controller circuit according to claim 1 , wherein said physical memory is selected out of a group comprising a static random access memory, a latched based memory, and a dynamic random access memory.

7. The memory controller circuit according to claim 1 , wherein said memory controller circuit is integrated together with physical memory.

8. The memory controller circuit according to claim 1 , wherein said physical memory stores input values of a computational memory and is integrated with said computational memory.

9. The memory controller circuit according to claim 1 , wherein physical memory stores input values of a digital memory and is integrated with a digital processing unit.

10. A method for mapping data of a convolutional neural network to a physical memory, said method comprising:

receiving a selection parameter value; and

mapping pixel values of one layer of said convolutional neural network to memory words of said physical memory according to one of a plurality of mapping schemas, wherein said mapping is dependent on said value of said received selection parameter value, and wherein the one of the plurality of mapping schemas is an intra-word adjacent mapping schema, and wherein bit values of portions of different pixel values are mapped to one physical memory word in a contiguous manner.

11. The method according to claim 10 , wherein the one of the plurality of mapping schemas is a kernel word interleaving schema, wherein bit values of portions of different pixel values are mapped to said physical memory such that at most a number of pixels equal to a size of a kernel of a convolutional neural network operation is mapped contiguously to adjacent memory words.

12. The method according to claim 10 , wherein the one of the plurality of mapping schemas is an activation word interleaving schema, wherein bit values of pixel values are mapped such that a memory word comprises the bit values of at most one pixel.

13. The method according to claim 10 , wherein said selection parameter value is comprised in a set of parameter values of a layer of said convolutional neural network.

14. The method according to claim 10 , wherein said selection parameter value is independent of said parameter values of a layer of said convolutional neural network.

15. The method according to claim 10 , wherein said physical memory is selected out of a group comprising a static random access memory, a latched based memory, and a dynamic random access memory.

16. The method according to claim 10 , wherein said memory controller circuit is integrated together with physical memory.

17. The method according to claim 10 , wherein said physical memory stores input values of a computational memory and is integrated with said computational memory or with a digital processing unit.

18. A computer program product for mapping data of a convolutional neural network to a physical memory, said computer program product comprising a computer readable storage medium having program instructions embodied therewith, said program instructions being executable by one or more computing systems or controllers to cause said one or more computing systems to:

receive a selection parameter value; and

map pixel values of one layer of said convolutional neural network to memory words of said physical memory according to one of a plurality of mapping schemas, wherein said mapping is dependent on said value of said received selection parameter value, and wherein the one of the plurality of mapping schemas is an intra-word adjacent mapping schema, and wherein bit values of portions of different pixel values are mapped to one physical memory word in a contiguous manner.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2021
From: DAZZI, MARTINO; FRANCESE, PIER ANDREA; SEBASTIAN, ABU; ELEFTHERIOU, EVANGELOS STAVROS
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 056064/0714 →
Continuity (1)
Related Publication 20220350514A1 · Nov 3, 2022
References Cited (13)
US 6157751A · Olson · 2000 [cited by applicant]
US 9665799B1 · Munteanu · 2017 [cited by applicant]
US 20170103299A1 · Aydonat · 2017 [cited by applicant]
US 20180189645A1 · Chen · 2018 [cited by examiner]
US 20190102671A1 · Cohen · 2019 [cited by applicant]
US 20190187963A1 · Bokhari · 2019 [cited by applicant]
US 20200057919A1 · Lovin · 2020 [cited by applicant]
US 20200126178A1 · Munteanu · 2020 [cited by applicant]
US 20200151019A1 · Yu · 2020 [cited by examiner]
CN 111178519A · 2020 [cited by applicant]
Wei et al (“Memory Access Optimization of a Neural Network Accelerator Based on Memory Controller” Feb. 2021) (Year: 2021). [cited by examiner]
International Search Report, International Application No. PCT/IB2022/052592, International Filing Date: Mar. 22, 2022, Mailing Date of Search Report: Jun. 28, 2022, 6 pages. [cited by applicant]
Stoutchinin, et al., “Optimally Scheduling CNN Convolutions for Efficient Memory Access,” arXiv:1902.01492v1 [cs.NE], Feb. 4, 2019, pp. 1-14. [cited by applicant]