IP Library › Granted Patent US 12670366
Granted Patent B2
US 12670366 · App. 18/090,628 · Granted Jun 30, 2026

Memory device for optimizing computation of convolution layer, method for controlling memory device, and recording medium storing instruction to perform method for controlling memory device

Inventors: Johnny Rhe (Suwon-si, KR); Sung Min Moon (Suwon-si, KR); Jong Hwan Ko (Suwon-si, KR)
Assignee: Research & Business Foundation SUNGKYUNKWAN UNIVERSITY
G06N3/0464
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12670366
App. No.
18/090,628
Granted
Jun 30, 2026
Kind
B2
Abstract

There is a method of controlling a memory device. The method comprises acquiring a size of a PIM array provided to compute a convolution layer included in a deep neural network, a size of input data input to the convolution layer, and a size of a kernel filtering the input data; and determining a size of a parallel window such that a number of times of cycles of the PIM array for the convolution layer is minimized based on the size of the PIM array, the size of the input data, and the size of the kernel.

Claims (34)

1 . A method performed by a controller included in a memory device, the method comprising:

acquiring a size of a PIM array provided to compute a convolution layer included in a deep neural network, a size of input data input to the convolution layer, and a size of a kernel filtering the input data; and

determining a size of a parallel window such that a number of times of cycles of the PIM array for the convolution layer is minimized based on the size of the PIM array, the size of the input data, and the size of the kernel,

wherein the determining the size of the parallel window includes:

calculating the number of times of cycles based on at least one of a number of times of shifts of the parallel window for the input data, a number of times of inputs to the PIM array according to the size of the parallel window, and a number of times of outputs outputted from the PIM array according to the size of the parallel window.

2 . The method of claim 1 , wherein

the number of times of shifts are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel,

the number of times of inputs are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel; and

the number of times of outputs are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel.

3 . The method of claim 2 , wherein the calculating of the number of times of shifts is configured to calculate the number of times of shifts based on at least one of the size of the input data and the size of the kernel.

4 . The method of claim 2 , wherein the calculating of the number of times of inputs is configured to calculate the number of times of inputs based on at least one of the size of the PIM array, the size of the input data, and the number of input channels according to the input data.

5 . The method of claim 2 , wherein the calculating of the number of times of outputs is configured to calculate the number of times of outputs based on at least one of the size of the PIM array, the size of the input data, the size of the kernel, and the number of output channels according to the kernel.

6 . A memory device, comprising:

a PIM array configured to compute a convolution layer included in a deep neural network; and

a controller configured to acquire a size of the PIM array, a size of input data input to the convolution layer, and a size of a kernel filtering the input data, determine a size of a parallel window such that a number of times of cycles of the PIM array for the convolution layer is minimized based on the size of the PIM array, the size of the input data, and the size of the kernel, and calculate the number of times of cycles based on at least one of a number of times of shifts of the parallel window for the input data, a number of times of inputs to the PIM array according to the size of the parallel window, and a number of times of outputs outputted from the PIM array according to the size of the parallel window.

7 . The memory device of claim 6 , wherein

the number of times of shifts are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel;

the number of times of inputs are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel; and

the number of times of outputs are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel.

8 . The memory device of claim 7 , wherein the controller is configured to calculate the number of times of inputs based on at least one of the size of the PIM array, the size of the input data, and the number of input channels according to the input data.

9 . The memory device of claim 7 , wherein the controller is configured to calculate the number of times of outputs based on at least one of the size of the PIM array, the size of the input data, the size of the kernel, and the number of output channels according to the kernel.

10 . The memory device of claim 6 , wherein the controller is configured to calculate the number of times of shifts based on at least one of the size of the input data and the size of the kernel.

11 . A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method of controlling a memory device, the method comprising:

acquiring a size of a PIM array provided to compute a convolution layer included in a deep neural network, a size of input data input to the convolution layer, and a size of a kernel filtering the input data; and

determining a size of a parallel window such that a number of times of cycles of the PIM array for the convolution layer is minimized based on the size of the PIM array, the size of the input data, and the size of the kernel,

wherein the determining the size of the parallel window includes:

calculating the number of times of cycles based on at least one of a number of times of shifts of the parallel window for the input data, a number of times of inputs to the PIM array according to the size of the parallel window, and a number of times of outputs outputted from the PIM array according to the size of the parallel window.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein

the number of times of shifts are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel,

the number of times of inputs are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel, and

the number of times of outputs are calculated based on the size of the PIM array, the size of the input data, and the size of the kernel.

13 . The non-transitory computer-readable storage medium of claim 12 , wherein the calculating of the number of times of shifts is configured to calculate the number of times of shifts based on at least one of the size of the input data and the size of the kernel.

14 . The non-transitory computer-readable storage medium of claim 12 , wherein the calculating of the number of times of inputs is configured to calculate the number of times of inputs based on at least one of the size of the PIM array, the size of the input data, and the number of input channels according to the input data.

15 . The non-transitory computer-readable storage medium of claim 12 , wherein the calculating of the number of times of outputs is configured to calculate the number of times of outputs based on at least one of the size of the PIM array, the size of the input data, the size of the kernel, and the number of output channels according to the kernel.