Granted Patent B2

US 12,361,270 · App. 18/597,728 · Granted Jul 15, 2025

Inventors: Jinseok Kim (Seongnam-si, KR); Kyeongryeol Bong (Seongnam-si, KR); Jinwook Oh (Seongnam-si, KR); Yoonho Boo (Seongnam-si, KR)

Assignee: Rebellions Inc.

G06N3/063G06F9/544G06F12/084

Abstract

Claims (52)

1. A neural core comprising:

a processing unit configured to perform operations;

an L0 memory configured to store input data, wherein the input data comprises a plurality of parts and a number of data granules of each of the plurality of parts is identical to each other; and

a load/store unit (LSU) configured to perform a load task and a store task of data between the processing unit and the L0 memory,

wherein the LSU comprises a local memory load unit configured to transmit the input data in the L0 memory to the processing unit, and

the local memory load unit configured to

retrieve the input data in the L0 memory,

transform an order of data granules of the input data based on an instruction and thereby generate transformed data, and

transmit the transformed data to the processing unit,

wherein the instruction is a shuffle operation instruction that generates the transformed data in which the order of data granules of the input data is changed by sequentially and alternately arranging the data granules of each of the plurality of parts.

2. The neural core of claim 1 , wherein the input data comprise first to j-th data granules of same size each other, and the transformed data comprises first to j-th data granules of same size each other,

the processing unit receives the i input data simultaneously, and

said j is an integer multiple of said i.

3. The neural core of claim 1 , wherein the input data has a size of an even multiple of one of the data granules.

4. The neural core of claim 3 , wherein the input data is i times larger than one of the data granules at most,

the processing unit receives i input data simultaneously, and

said i is a natural number.

5. The neural core of claim 1 , wherein the local memory load unit comprises:

a target decision module configured to decode the instruction and identify the input data within the L0 memory,

a transformation logic configured to perform the shuffle operation based on the instruction, and

an output first in first out (FIFO) configured to transmit the transformed data to the processing unit based on the instruction.

6. The neural core of claim 5 , wherein the local memory load unit further comprises a tensor register file configured to receive the input data from the target decision module, provide the input data to the transformation logic, and receive the transformed data from the transformation logic.

7. The neural core of claim 6 ,

wherein the tensor register file has i entries,

a number of FIFOs of the output FIFO is i, and

said i is a natural number.

8. The neural core of claim 1 , wherein the instruction comprises a layout transform instruction for transforming a layout of the input data.

9. A neural processing device comprising:

at least one neural processor;

a shared memory shared by the at least one neural processor; and

a global interconnection configured to transmit data between the at least one neural processor and the shared memory,

wherein each of the at least one neural processor comprises:

at least one neural core; and

an L1 shared memory shared by the at least one neural core,

wherein the at least one neural core comprises:

a processing unit configured to perform operations;

an LSU configured to transmit input data to the processing unit, wherein the input data comprises a plurality of parts and a number of data granules of each of the plurality of parts is identical to each other; and

an L0 memory configured to store the input data, and

wherein the LSU transforms an order of data granules of the input data based on an instruction, thereby generating transformed data, and transfers the transformed data to the processing unit,

wherein the instruction is a shuffle operation instruction that generates the transformed data in which the order of data granules of the input data is changed by sequentially and alternately arranging the data granules of each of the plurality of parts.

10. The neural processing device of claim 9 , wherein the input data comprise first to j-th data granules of same size each other, and the transformed data comprises first to j-th data granules of same size each other,

the processing unit receives the i input data simultaneously, and

said j is an integer multiple of said i.

11. A method for loading data of a neural processing device, comprising:

receiving a layout transform instruction;

storing input data in a tensor register file, wherein the input data comprises a plurality of parts and a number of data granules of each of the plurality of parts is identical to each other;

transforming an order of data granules of the input data based on an instruction, thereby generating transformed data, wherein the instruction is a shuffle operation instruction that generates the transformed data in which the order of data granules of the input data is changed by sequentially and alternately arranging the data granules of each of the plurality of parts;

storing the transformed data in an output FIFO; and

transferring the transformed data to a processing unit.

12. The method for loading data of the neural processing device of claim 11 , further comprising:

storing the transformed data in the tensor register file after generating the transformed data; and

transmitting the transformed data stored in the tensor register file to the output FIFO.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (13)