IP Library Granted Patent US 12,210,473
Granted Patent B2
US 12,210,473 · App. 17/980,685 · Granted Jan 28, 2025

Application-transparent near-memory processing architecture with memory channel network

Inventors: Nam Sung Kim (Champaign, IL); Mohammad Alian (Urbana, IL)
Assignee: The Board of Trustees of the University of Illinois
G06F13/4027G06F12/10G06F13/1673G06F2212/1016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,210,473
App. No.
17/980,685
Granted
Jan 28, 2025
Kind
B2
Abstract

A computing device includes a host processor to execute a host driver to create a host-side interface, the host-side interface emulating a first Ethernet interface, assign the host-side interface a first medium access control (MAC) address and a first Internet Protocol (IP) address. Memory components are disposed on a substrate. A memory channel network (MCN) processor is disposed on the substrate and coupled between the memory components and the host processor. The MCN processor is to execute an MCN driver to create a MCN-side interface, the MCN-side interface emulating a second Ethernet interface. The MCN processor is to assign the MCN-side interface a second MAC address and a second IP address, which identify the MCN processor as a MCN network node to the host processor.

Claims (65)

1. A computing device comprising:

a host processor to:

execute a host driver to create a host-side interface, the host-side interface emulating a first virtual Ethernet interface; and

assign the host-side interface a first medium access control (MAC) address and a first Internet Protocol (IP) address;

a substrate on which is disposed memory components; and

a memory channel network (MCN) processor disposed on the substrate and coupled between the memory components and the host processor, the MCN processor to:

execute an MCN driver to create an MCN-side interface, the MCN-side interface emulating a second virtual Ethernet interface; and

assign the MCN-side interface a second MAC address and a second IP address, which identify the MCN processor as an MCN network node to the host processor; and

wherein the MCN driver, when executed by the MCN processor, is to:

create a network device object;

set up the network device object as an Ethernet device; and

register the Ethernet device with a kernel of the host processor, to make the MCN-side interface visible to a host operating system executing on the host processor.

2. The computing device of claim 1 , wherein, upon receipt of a network packet having a third IP address directed to a second MCN-side interface, the host-side interface is to forward the network packet to the second MCN-side interface comprising a second MCN network node.

3. The computing device of claim 1 , wherein the MCN processor is further to configure the MCN-side interface so that outgoing network packets from the MCN network node are forwarded to the host-side interface for routing.

4. The computing device of claim 1 , wherein the MCN processor comprises:

a core to execute the MCN driver; and

a local buffer coupled to the core, the local buffer to store MCN messages that are sent to and received from the host-side interface, each MCN message comprising a packet length and packet data.

5. The computing device of claim 4 , wherein, in response to a network packet received at the MCN-side interface from a network stack, the MCN-side interface is to:

read a transmit start pointer and a transmit end pointer from the local buffer for the network packet;

in response to sufficient space being available in a transmission buffer of the local buffer, write the packet length followed by the packet data into the transmission buffer, starting at a buffer address to which points the transmit end pointer;

update a value for the transmit end pointer in the local buffer; and

set a transmit polling field of the local buffer to a non-zero value to indicate that a new packet is enqueued in the transmission buffer.

6. The computing device of claim 4 , wherein the local buffer comprises a transmit polling field, and the host-side interface comprises a polling agent to:

periodically read the transmit polling field in a plurality of local buffers across a plurality of MCN nodes, to determine whether there is a pending network packet; and

upon detection of a pending network packet in one of the plurality of local buffers:

read a transmit start pointer and a transmit end pointer from the local buffer for the pending network packet;

read a cache line at a location of the local buffer to which points the transmit start pointer;

retrieve the packet length and a packet destination MAC address from the cache line; and

transmit the pending network packet to a packet forwarding engine of the host-side interface.

7. The computing device of claim 6 , wherein the host-side interface is further to:

if the transmit start pointer moved by a number of bytes read from a transmission buffer of the local buffer is not equal to a value of the transmit end pointer, read an additional at least one more cache line from the plurality of local buffers; and

if the transmit start pointer moved by a number of bytes read from the transmission buffer of the plurality of local buffers is equal to a value of the transmit end pointer, reset a value of a transmit polling field of the local buffer and exit.

8. The computing device of claim 4 , wherein the host-side interface comprises a packet forwarding engine to, in response to receipt of a network packet:

determine that a destination MAC address of the network packet matches the first MAC address;

allocate a socket buffer;

copy data of the network packet from a receiving buffer of the local buffer to the socket buffer; and

transmit the data from the socket buffer to a network stack for processing.

9. The computing device of claim 4 , wherein the host-side interface comprises a packet forwarding engine to, in response to receipt of a network packet:

determine that a destination MAC address of the network packet matches a reserved address for broadcast;

allocate a socket buffer;

copy data of the network packet from a receiving buffer of the local buffer to the socket buffer;

transmit the data from the socket buffer to a network stack for processing; and

transmit the data from the socket buffer to a plurality of MCN network nodes.

10. The computing device of claim 4 , wherein the host-side interface comprises a packet forwarding engine to, in response to receipt of a network packet:

determine that a destination MAC address of the network packet matches a second address of a destination MCN node of a plurality of MCN network nodes; and

transmit the network packet to the destination MCN node.

11. The computing device of claim 1 , wherein the substrate is an MCN dual in-line memory module (DIMM).

12. The computing device of claim 1 , wherein the MCN processor is further to operate according to a memory-based physical layer (PHY) protocol.

13. A method comprising:

executing, by a host processor, a host driver to create a host-side interface, the host-side interface emulating a first virtual Ethernet interface; and

assigning, by the host processor, the host-side interface a first medium access control (MAC) address and a first Internet Protocol (IP) address;

executing, by an MCN processor co-located on a substrate with memory components, an MCN driver to create an MCN-side interface, the MCN-side interface emulating a second virtual Ethernet interface, wherein the MCN processor is coupled between the memory components and the host processor, wherein executing the MCN driver further comprises:

creating a network device object;

setting up the network device object as an Ethernet device; and

registering the Ethernet device with a kernel of the host processor, to make the MCN-side interface visible to a host operating system executing on the host processor; and

assigning, by the MCN processor, the MCN-side interface a second MAC address and a second IP address, which identify the MCN processor as an MCN network node to the host processor.

14. The method of claim 13 , wherein, in response to receiving a network packet having a third IP address directed to a second MCN-side interface, the method further comprises forwarding, by the host-side interface, the network packet to the second MCN-side interface, wherein the second MCN-side interface comprises a second MCN network node.

15. The method of claim 13 , further comprising configuring, by the MCN processor, the MCN-side interface so that outgoing network packets from the MCN network node are forwarded to the host-side interface for routing.

16. The method of claim 13 , further comprising storing, by the MCN processor, at a local buffer, MCN messages that are sent to and received from the host-side interface, each MCN message comprising a packet length and packet data.

17. The method of claim 16 , wherein, in response to receiving a network packet at the MCN-side interface from a network stack, the method further comprises:

reading, by the MCN-side interface, a transmit start pointer and a transmit end pointer from the local buffer for the network packet;

wherein, in response to sufficient space being available in a transmission buffer of the local buffer, the method further comprises writing the packet length followed by the packet data into the transmission buffer, starting at a buffer address to which points the transmit end pointer;

updating a value for the transmit end pointer in the local buffer; and

setting a transmit polling field of the local buffer to a non-zero value to indicate that a new packet is enqueued in the transmission buffer.

18. The method of claim 13 , wherein the method further comprises operating the MCN processor according to a memory-based physical layer (PHY) protocol.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2022
From: KIM, NAM SUNG; ALIAN, MOHAMMAD
To: THE BOARD OF TRUSTEES OF THE UNIVERSITY OF ILLINOIS
Reel/Frame 061968/0038 →
Continuity (3)
Division 17250785
Provisional Application 62728416 · Sep 7, 2018
Related Publication 20230071386A1 · Mar 9, 2023
References Cited (98)
US 5243596A · Port et al. · 1993 [cited by applicant]
US 5819051A · Murray et al. · 1998 [cited by applicant]
US 5903718A · Marik et al. · 1999 [cited by applicant]
US 6987961B1 · Pothana · 2006 [cited by examiner]
US 7620526B2 · Back · 2009 [cited by examiner]
US 7941613B2 · Pong · 2011 [cited by examiner]
US 9189264B1 · Steffen · 2015 [cited by examiner]
US 10191865B1 · Machulsky et al. · 2019 [cited by applicant]
US 10798059B1 · Singh · 2020 [cited by examiner]
US 10860511B1 · Thompson · 2020 [cited by examiner]
US 11520724B2 · Kim · 2022 [cited by examiner]
US 20030055929A1 · Ding · 2003 [cited by examiner]
US 20030095560A1 · Arita · 2003 [cited by examiner]
US 20030130832A1 · Schulter · 2003 [cited by examiner]
US 20040081005A1 · Wayne, Jr. et al. · 2004 [cited by applicant]
US 20070008989A1 · Jogelkar · 2007 [cited by applicant]
US 20070136594A1 · Jordan · 2007 [cited by examiner]
US 20100005219A1 · Loughner et al. · 2010 [cited by applicant]
US 20100262819A1 · Yang · 2010 [cited by examiner]
US 20110047302A1 · Hausman et al. · 2011 [cited by applicant]
US 20110235573A1 · Crain, Jr. · 2011 [cited by examiner]
US 20120151048A1 · Kitazawa · 2012 [cited by examiner]
US 20120324177A1 · Shin · 2012 [cited by examiner]
US 20130100963A1 · Hartman · 2013 [cited by applicant]
US 20150378932A1 · Souri · 2015 [cited by examiner]
US 20160055021A1 · Beveridge et al. · 2016 [cited by applicant]
US 20160162404A1 · Lee et al. · 2016 [cited by applicant]
US 20160314094A1 · Schade · 2016 [cited by examiner]
US 20160364347A1 · Yeung et al. · 2016 [cited by applicant]
US 20180239725A1 · Kumar et al. · 2018 [cited by applicant]
US 20180365185A1 · Risinger · 2018 [cited by examiner]
US 20190155693A1 · Bomma · 2019 [cited by examiner]
US 20190212797A1 · Karidis · 2019 [cited by examiner]
Larrivee, Steve, “SSD 101: Everythingi You Ever Wanted to Know”, 2016, Cactus Technologies, Inc., p. 15 (Year: 2016). [cited by examiner]
“Coral Benchmark Codes,” available at https://asc.llnl.gov/CORAL-benchmarks/, 8 pages. [cited by applicant]
Wang et al., “Bigdatabench: A big data benchmark suite from internet services,” High Performance Computer Architecture (HPCA), 2014 IEEE 20th International Symposium on. IEEE, 2014, 12 pages. [cited by applicant]
Grossman, Leonid, “Large receive offload implementation in neterion 10gbe ethernet driver,” Linux Symposium, 2005, 6 pages. [cited by applicant]
Melzer, et al., “IP checksum offload,” Apr. 27, 1999, U.S. Pat. No. 5,898,713, 11 pages. [cited by applicant]
Jeong, et al., “mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems,” Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation, serial NSDI'14, Berkeley, CA, USA: Usenix As… [cited by applicant]
Chan et al., “Improving server application performance via pure TCP ACK receive optimization,” Presented as part of the 2013 USENIX Annual Technical Conference (USENIX ATC 13), San Jose, CA: USENIX, 2013, available at h… [cited by applicant]
Singh et al., “Jupiter rising: A decade of close topologies and centralized control in Google's datacenter network,” Sigcomm '15, 2015, 15 pages. [cited by applicant]
Andreyev, Alexey, “Introducing data center fabric, the next-generation Facebook data center network,” 2014, available at https://code.facebook.com/posts/360346274145943/, 11 pages. [cited by applicant]
Deering et al., “FBRAM: a New Form of Memory Optimized for 3D Graphics,” SIGGRAPH, Jul. 1994, 8 pages. [cited by applicant]
Draper, et al., “The Architecture of the DIVA Processing-in-memory Chip,” ICS, Jun. 2002, 12 pages. [cited by applicant]
Elliot et al., “Computational RAM: A Memory-SIMD Hybrid and its Application to DSP,” CICC, May 1992, 6 pages. [cited by applicant]
Mai et al., “Smart Memories: A Modular Reconfigurable Architecture,” ISCA, Jun. 2000, 11 pages. [cited by applicant]
Oskin et al., “Active Pages: A Computation Model for Intelligent Memory,” ISCA, Jun. 1998, 12 pages. [cited by applicant]
Patterson et al., “A Case for Intelligent RAM,” Micro, IEEE, vol. 17, Mar. 1997, 11 pages. [cited by applicant]
Loh, Gabriel H., “3D-Stacked Memory Architectures for Multi-core Processors,” ISCA, Jun. 2008, 14 pages. [cited by applicant]
Pawlowski, J. Thomas, “Hybrid Memory Cube,” Hot Chips, Aug. 2011, 24 pages. [cited by applicant]
Zhu et al., “A 3D-stacked Logic-in-memory Accelerator for Application-Specific Data Intensive Computing,” 3DIC, Oct. 2013, 7 pages. [cited by applicant]
Nai et al., “Graphpim: Enabling instruction-level pim offloading in graph computing frameworks,” High Performance Computer Architecture (HPCA), 2017 IEEE International Symposium on. IEEE, 2017, 12 pages. [cited by applicant]
Pugsley et al., “NDC: Analyzing the impact of 3D-stacked memory+ logic devices on MapReduce workloads,” Performance Analysis of Systems and Software (ISPASS), 2014 IEEE International Symposium on. IEEE, 2014, 11 pages. [cited by applicant]
Awan et al., “Identifying the potential of near data processing for apache spark,” Proceedings of the International Symposium on Memory Systems. ACM, 2017, 8 pages. [cited by applicant]
Intel, “An Introduction to the Intel R QuickPath Interconnect,” Document No. 320412-001US, Jan. 2009, https://www.intel.com/content/dam/doc/white-paper/quick-path-interconnect-introduction-paper.pdf, 22 pages. [cited by applicant]
Intel, “IvyTown Xeon + FPGA: The HARP Program,” Workshop: CPU+FPGA—OpenCL Based High Level Synthesis for CPU+FPGA Coherent Systems, International Symposium on Computer Architecture, ISCA, 2016, https://cpufpga.files.wor… [cited by applicant]
Asghari-Moghaddam, et al., “Chameleon: Versatile and Practical Near-DRAM Acceleration Architecture for Large Memory Systems,” MICRO, 2016, 13 pages. [cited by applicant]
Asghari-Moghaddam, et al., “Near-DRAM acceleration with single-ISA heterogeneous processing in standard memory modules,” IEEE Micro, vol. 36, 2016, 11 pages. [cited by applicant]
Gao et al., “Practical near-data processing for in-memory analytics frameworks,” Parallel Architecture and Compilation (PACT), 2015 International Conference on. IEEE, 2015, 12 pages. [cited by applicant]
Lloyd et al., 8. S. Lloyd and M. Gokhale, “Near memory key/value lookup acceleration,” Proceedings of the International Symposium on Memory Systems. ACM, 2017, 8 pages. [cited by applicant]
Kang et al., “FlexRAM: Toward an Advanced Intelligent Memory System,” ICCD, Oct. 1999, 10 pages. [cited by applicant]
Patterson et al., “Intelligent RAM (IRAM): the industrial setting, applications, and architectures,” ICCD, Oct. 1997, 2 pages. [cited by applicant]
Farmahini-Farahani et al., “NDA: Near-DRAM Acceleration Architecture Leveraging Commodity DRAM Devices and Standard Memory Modules,” in HPCA, Feb. 2015, 13 pages. [cited by applicant]
Meany et al., “The IBM z13 memory subsystem for big data,” IBM Journal of Research and Development, vol. 59, Jul. 2015, 11 pages. [cited by applicant]
Sukhwani et al., “Contutto: A Novel FPGA-based Prototyping Platform Enabling Innovation in the Memory Subsystem of a Server Class Processor,” in Proceedings of the 50th Annual IEEE/ACM International Symposium on Microar… [cited by applicant]
Chu et al., “Highlevel Programming Model Abstractions for Processing in Memory,” Workshop on Near-Data Processing, Dec. 2013, 4 pages. [cited by applicant]
Kersey et al., “Lightweight simt core designs for intelligent 3d stacked dram,” Proceedings of the International Symposium on Memory Systems. ACM, 2017, 11 pages. [cited by applicant]
Stuecheli et al., “CAPI: A Coherent Accelerator Processor Interface,” IBM Journal of Research and Development, vol. 59, Jan. 2015, 7 pages. [cited by applicant]
Micron, “3D XPointTM Technology,” https://www.micron.com/products/advanced-solutions/3d-xpoint-technology, Oct. 24, 2019, 1 page. [cited by applicant]
Ahn et al., “A scalable processing-in-memory accelerator for parallel graph processing” ACM/IEEE International Symposium on Computer Architecture (ISCA), Jun. 2015, 13 pages. [cited by applicant]
Zhang et al., “TOP-PIM: throughput-oriented programmable processing in memory,” in Proceedings of the 23rd International symposium on High-performance parallel and distributed computing. ACM, 2014, 14 pages. [cited by applicant]
Ahn et al., “PIM-enabled instructions: A low-overhead, locality-aware processing-in-memory architecture,” in Computer Architecture (ISCA), 2015 ACM/IEEE 42nd Annual International Symposium on. IEEE, 2015, 13 pages. [cited by applicant]
Gutierrez et al., “Integrated 3D-stacked server designs for increasing physical density of key-value stores,” in 19th International Conference on Architectural Support for Programming Languages and Operating Systems, AS… [cited by applicant]
Seshadri et al., “Ambit: In-memory accelerator for bulk bitwise operations using commodity dram technology,” in Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture. ACM, 2017, 15 pages. [cited by applicant]
Nair et al., “Active memory cube: A processing-in-memory architecture for exascale systems,” IBM Journal of Research and Development, vol. 59, 2015,14 pages. [cited by applicant]
Shvachko et al., “The hadoop distributed file system,” Mass storage systems and technologies (MSST), 2010 IEEE 26th symposium on. IEEE, 2010, 20 pages. [cited by applicant]
Zaharia et al., “Spark: Cluster computing with working sets,” Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, ser. HotCloud'10. Berkeley, CA, USA: USENIX Association, 2010, 7 pages. [cited by applicant]
Bridges et al., “Users Guide to MPICH, a Portable Implementation of MPI,” Argonne National Laboratory, vol. 9700, 1995, 35 pages. [cited by applicant]
Qualcomm, “Snapdragon 835 Mobile Platform,” 2016. [Online]. Available: https://www.qualcomm.com/products/snapdragon/processors/835, 9 pages. [cited by applicant]
Jacob et al., “Virtual memory in contemporary microprocessors,” IEEE Micro, vol. 18, 1998, 17 pages. [cited by applicant]
Dumazet, Eric, “Busy polling: Past, present, future,” NetDev 2.1, 2017, 4 pages. [cited by applicant]
Pugsley et al., “Comparing Implementations of Near-Data Computing with In-Memory MapReduce Workloads,” Micro, IEEE, vol. 34, Jul. 2014, 9 pages. [cited by applicant]
The Tech Report, “The Exynos 5433 SoC,,” Jan. 31, 2015, http://techreport.com/review/27539/samsung-galaxy-note-4-with-the-exynos-5433-processor/2, 59 pages. [cited by applicant]
Wei, Andy, “Qualcomm snapdragon 835 first to 10 nm,” http://www.techinsights.com/about-techinsights/overview/blog/qualcomm-snapdragon-835-first-to-10-nm/, 2017, 2 pages. [cited by applicant]
Nvidia, “Nvidia R TegraR X1,” https://international.download.nvidia.com/pdf/tegra/Tegra-X1-whitepaper-v1.0.pdf, 2015, 41 pages. [cited by applicant]
Micron, “Nvdimm,” 2016, available at https://www.micron.com/products/dram-modules/nvdimm/, 2 pages. [cited by applicant]
Turner et al., “Diversifying the Internet,” in Global Telecommunications Conference, 2005, GLOBECOM'05. IEEE, vol. 2. IEEE, 2005, 6 pages. [cited by applicant]
Anderson, et al., “Overcoming the Internet impasse through Virtualization,” Computer, vol. 38, 2005, 7 pages. [cited by applicant]
“IPv4 standard,” https://en.wikipedia.org/wiki/IPv4, accessed: Mar. 25, 2018, 9 pages. [cited by applicant]
“Ethernet frame” available at https://en.wikipedia.org/wiki/Ethernet frame, accessed: Mar. 25, 2018, 8 pages. [cited by applicant]
Connery et al., “Offload of top segmentation to a smart adapter,” Aug. 10, 1999, U.S. Pat. No. 5,937,169, 19 pages. [cited by applicant]
“TCP frame,” available at https://en.wikipedia.org/wiki/Transmission Control Protocol, accessed: Mar. 25, 2018, 14 pages. [cited by applicant]
Intel, “Nios II Processor,” available at https://www.altera.com/products/processors/overview.html, 2017, 11 pages. [cited by applicant]
Intel, “Avalon R Interface Specifications,” available at https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/manual/mnl_avalon_spec.pdf, 2017, 70 pages. [cited by applicant]
Alian et al., “Dist-gem5: Distributed Simulation of Computer Clusters,” in ISPASS, 2017, 29 pages. [cited by applicant]
Li et al., “McPAT: An Integrated Power, Area, and Timing Modeling Framework for Multicore and Manycore Architectures” in MICRO, 2009, 12 pages. [cited by applicant]
“IPERF: The ultimate speed test tool for TCP, UDP and SCTP,” available at https://iperf.fr/, 3 pages. [cited by applicant]
Bailey et al., “The NAS parallel benchmarks,” The International Journal of Supercomputing Applications, vol. 5, 1991, 13 pages. [cited by applicant]