Granted Patent B2

US 12,299,597 · App. 17/459,356 · Granted May 13, 2025

Inventor: Hsiang-Lan Lung (Ardsley, NY)

Assignee: MACRONIX INTERNATIONAL CO., LTD.

G06N5/04G06F9/30003G06F11/2263

Abstract

Claims (35)

1. A reconfigurable inference platform, comprising:

a processor chip including a runtime processor core, an accelerator core, on-chip memory and a processor-memory interface exposed on a chip-to-chip bonding surface of the processor chip;

a first memory chip accessible by the processor chip to store a collection of executable models of an inference engine, each model including a set of weights to be applied in execution of the model;

a second memory chip to store the set of weights of a selected executable model, the second memory chip including a memory-processor interface exposed on a surface of the second memory chip and complementary to the processor-memory interface; and

direct vertical connections between the processor-memory interface and memory-processor interface,

wherein the processor chip and the second memory chip are stacked and disposed on an interposer, and the first memory chip is disposed on the interposer, the interposer including interconnection wiring forming part of a data path between the first memory chip and the second memory chip, and

wherein the processor chip includes a second input/output interface, the data path including a connection from the interconnection wiring of the interposer to the second input/output interface on the processor chip.

2. The platform of claim 1 , wherein the direct vertical connections comprise via-to-via connections.

3. The platform of claim 1 , wherein the processor core has access to instruction memory, storing executable instructions to perform a procedure including:

selecting an executable model from the collection of executable models stored in the first memory chip, loading a computation graph for the selected model including configuring the accelerator core, transferring the set of weights of the selected model to the second memory chip, and executing the selected model.

4. The platform of claim 1 , wherein the processor core has access to instruction memory, storing executable instructions to perform a procedure in response to a control event, including changing the selected model to a different model in the collection of executable models, load a computation graph for the different model including configuring the accelerator core, transferring the set of weights of the different model to the second memory chip, and executing the different model.

5. The platform of claim 1 , wherein the interposer is below the second memory chip, and the processor chip is disposed above the second memory chip.

6. The platform of claim 1 , wherein the interposer is below the processor chip and the second memory chip is disposed above the processor chip.

7. The platform of claim 1 , wherein the first memory chip comprises a charge trapping, NAND-architecture memory, and the second memory chip comprises nonvolatile random access memory.

8. The platform of claim 7 , wherein the nonvolatile random access memory is phase change memory.

9. The platform of claim 7 , wherein the nonvolatile random access memory is a charge trapping, NOR-architecture memory.

10. The platform of claim 1 , wherein the processor chip, first memory chip and second memory chip are disposed in a multichip package.

11. A reconfigurable inference method, comprising:

providing a processor chip including a runtime processor core, an accelerator core, on-chip memory and a processor-memory interface exposed on a chip-to-chip bonding surface of the processor chip;

storing a collection of executable models of an inference engine for a model implemented by machine learning in a first memory chip accessible by the processor chip, each model including a set of weights to be applied in execution of the model;

selecting in response to a control event an executable model from the collection of executable models stored in the first memory chip, loading a computation graph for the selected model including configuring the accelerator core, and transferring the set of weights of the selected executable model from the first memory chip to a second memory chip, the second memory chip including a memory-processor interface disposed on a surface the second memory chip and complementary to the processor-memory interface; and

executing the selected executable model using direct vertical connections between the processor-memory interface and memory-processor interface,

wherein the processor chip and the second memory chip are stacked and disposed on an interposer, and the first memory chip is disposed on the interposer, the interposer including interconnection wiring forming part of a data path between the first memory chip and the second memory chip, and

wherein the processor chip includes a second processor-memory interface, and including transferring data from the first memory chip to the processor chip on a data path including a connection from the interconnection wiring of the interposer to the second processor-memory interface on the processor chip.

12. The method of claim 11 , wherein the direct vertical connections comprise via-to-via connections.

13. The method of claim 11 , including changing, in response to a second control event, the selected model to a different model in the collection of executable models, loading a computation graph for the different model including configuring the accelerator core, transferring the set of weights of the different model to the second memory chip, and executing the different model.

14. The method of claim 11 , wherein the first memory chip comprises a charge trapping, NAND-architecture memory, and the second memory chip comprises nonvolatile random access memory.

15. The method of claim 14 , wherein the nonvolatile random access memory is phase change memory.

16. A reconfigurable inference platform, comprising:

a processor chip including a runtime processor core, an accelerator core, on-chip memory and a processor-memory interface exposed on a chip-to-chip bonding surface of the processor chip;

a first memory chip accessible by the processor chip to store a collection of executable models of an inference engine, each model including a set of weights to be applied in execution of the model;

a second memory chip to store the set of weights of a selected executable model, the second memory chip including a memory-processor interface exposed on a surface of the second memory chip and complementary to the processor-memory interface; and

direct vertical connections between the processor-memory interface and memory-processor interface,

wherein the processor chip and the second memory chip are stacked and disposed on an interposer, and the first memory chip is disposed on the interposer, the interposer including interconnection wiring forming part of a data path between the first memory chip and the second memory chip, and

wherein (i) the interposer is below the second memory chip and the processor chip, and (ii) the processor chip is disposed above the second memory chip or the second memory chip is disposed above the processor chip.

Assignments (1)

Continuity (1)

References Cited (337)

Cited By (1)

US 12,585,931