IP Library Granted Patent US 12,445,532
Granted Patent B2
US 12,445,532 · App. 17/545,730 · Granted Oct 14, 2025

Network attached memory using selective resource migration

Inventors: Isaac R. Nassi (Los Gatos, CA); David P. Reed (Needham, MA)
Assignee: Hewlett Packard Enterprise Development LP
H04L67/568G06F9/45558G06F12/0815H04L67/10H04L67/1097H04L67/2885G06F2009/45583
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,445,532
App. No.
17/545,730
Granted
Oct 14, 2025
Kind
B2
Abstract

Maintaining cache coherency in the presence of a network attached memory is disclosed. A computer system includes a plurality of physical nodes. An operating system is run collectively across the plurality of physical nodes. The physical nodes are configured to communicate with a network attached memory. Based at least in part on an operation to be performed with respect to page on a first physical node included in the plurality of physical nodes, the network attached memory is configured to receive a message. The network attached memory is configured to perform an action based at least in part on the received message.

Claims (27)

1. A computer system, comprising:

a plurality of interconnected computing nodes, wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on at least a portion of the interconnected computing nodes;

a persistent memory array separate from the plurality of interconnected computing nodes and separate from the virtual environment, wherein the persistent memory array communicates with at least some of the computing nodes over a network according to a coherency protocol;

wherein, based at least in part on an operation to be performed with respect to a portion of memory on a first computing node included in the plurality of interconnected computing nodes, the persistent memory array receives, in accordance with the coherency protocol, a message, and wherein the portion of memory on the first computing node corresponds to a guest physical address in the virtual environment; and

wherein the persistent memory array is configured to perform an action based at least in part on the coherency protocol and the received message.

2. The computer system of claim 1 , wherein portions of the persistent memory array correspond to guest physical addresses in the virtual environment defined by the set of hyper-kernels running on the at least portion of the interconnected computing nodes.

3. The computer system of claim 1 , wherein the operation to be performed with respect to the portion of memory on the first computing node comprises writing into the portion of memory.

4. The computer system of claim 3 , wherein the message comprises an invalidation request.

5. The computer system of claim 4 , wherein the action performed by the persistent memory array comprises invalidating a copy of the portion of memory stored on the persistent memory array.

6. The computer system of claim 5 , wherein the persistent memory array is configured to return an acknowledgment indicating completion of the invalidation.

7. The computer system of claim 6 , wherein the writing into the portion of memory is performed based at least in part on the returned acknowledgment.

8. The computer system of claim 1 , wherein the operation to be performed with respect to the portion of memory on the first computing node comprises a read request for the portion of memory.

9. The computer system of claim 8 , wherein the action performed by the persistent memory array comprises receiving a copy of the portion of memory.

10. The computer system of claim 1 , wherein in response to detection of a failure, the persistent memory array is configured to request copies of one or more portions of memory from the computing nodes in the plurality of interconnected computing nodes.

11. The computer system of claim 10 , wherein the one or more portions of memory for which copies are requested comprise portions of memory determined to be invalid on the persistent memory array.

12. The computer system of claim 11 , wherein the persistent memory array is configured to update an invalid portion of memory with a corresponding obtained copy of a requested page.

13. The computer system of claim 1 , wherein the persistent memory array comprises a centralized appliance.

14. The computer system of claim 1 , wherein the persistent memory array includes at least one of flash memory, phase change memory, and 3D-Xpoint memory.

15. The computer system of claim 1 , wherein the persistent memory array is configured to store a read-only copy of the portion of memory.

16. A method, comprising:

receiving, at a persistent memory array, a message, wherein the message is received in accordance with a coherency protocol, and wherein the message is received based at least in part on an operation to be performed with respect to a portion of memory on a first computing node in a plurality of interconnected computing nodes;

wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on at least a portion of the interconnected computing nodes, the persistent memory array is separate from the interconnected computing nodes, the persistent memory array is separate from the virtual environment, and wherein the persistent memory array communicates with at least some of the computing nodes over a network according to the coherency protocol; and

performing, by the persistent memory array, an action based at least in part on the coherency protocol and the received message.

17. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving, at a persistent memory array, a message, wherein the message is received in accordance with a coherency protocol, and wherein the message is received based at least in part on an operation to be performed with respect to a portion of memory on a first computing node in a plurality of interconnected computing nodes;

wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on at least a portion of the interconnected computing nodes, the persistent memory array is separate from the interconnected computing nodes, the persistent memory array is separate from the virtual environment, and wherein the persistent memory array communicates with at least some of the computing nodes over a network according to the coherency protocol; and

performing, by the persistent memory array, an action based at least in part on the coherency protocol and the received message.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 13, 2025
From: NASSI, ISAAC R.; REED, DAVID P.
To: TIDALSCALE, INC.
Reel/Frame 072011/0595 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2023
From: TIDALSCALE, INC.
To: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
Reel/Frame 062282/0452 →
Continuity (3)
Continuation 15279187 · Sep 28, 2016
Provisional Application 62236076 · Oct 1, 2015
Related Publication 20220174130A1 · Jun 2, 2022
References Cited (359)
US 5055999A · Frank · 1991 [cited by applicant]
US 5109486A · Seymour · 1992 [cited by applicant]
US 5230069A · Brelsford et al. · 1993 [cited by applicant]
US 5247673A · Costa · 1993 [cited by applicant]
US 5745778A · Alfieri · 1998 [cited by applicant]
US 5991893A · Snider · 1999 [cited by applicant]
US 6016542A · Gottleib et al. · 2000 [cited by applicant]
US 6026472A · James et al. · 2000 [cited by applicant]
US 6035378A · James · 2000 [cited by applicant]
US 6105053A · Kimmel et al. · 2000 [cited by applicant]
US 6247109B1 · Kleinsorge et al. · 2001 [cited by applicant]
US 6266745B1 · De et al. · 2001 [cited by applicant]
US 6336170B1 · Dean et al. · 2002 [cited by applicant]
US 6601083B1 · Reznak · 2003 [cited by applicant]
US 6625709B2 · Aiken et al. · 2003 [cited by applicant]
US 6658447B2 · Cota-Robles · 2003 [cited by applicant]
US 6748593B1 · Brenner et al. · 2004 [cited by applicant]
US 6799202B1 · Hankinson · 2004 [cited by applicant]
US 6804766B1 · Noel · 2004 [cited by applicant]
US 6807616B1 · McGrath et al. · 2004 [cited by applicant]
US 6823472B1 · Dekoning et al. · 2004 [cited by applicant]
US 7222221B1 · Agesen et al. · 2007 [cited by applicant]
US 7308498B1 · Olsen et al. · 2007 [cited by applicant]
US 7328371B1 · Kalyanasundharam et al. · 2008 [cited by applicant]
US 7596654B1 · Wong · 2009 [cited by applicant]
US 7613882B1 · Akkawi · 2009 [cited by applicant]
US 7685391B1 · Cholleti et al. · 2010 [cited by applicant]
US 7711789B1 · Jnagal · 2010 [cited by applicant]
US 7715400B1 · Krakirian · 2010 [cited by applicant]
US 7782869B1 · Chitlur Srinivasa · 2010 [cited by applicant]
US 7783788B1 · Quinn · 2010 [cited by applicant]
US 7797375B2 · Vincent · 2010 [cited by applicant]
US 7802073B1 · Cheng · 2010 [cited by applicant]
US 7844709B2 · Aman et al. · 2010 [cited by applicant]
US 7852786B2 · Wang et al. · 2010 [cited by applicant]
US 7865895B2 · Anand et al. · 2011 [cited by applicant]
US 7934035B2 · Miloushev et al. · 2011 [cited by applicant]
US 8032875B2 · Kosche et al. · 2011 [cited by applicant]
US 8090914B2 · Tierney · 2012 [cited by applicant]
US 8156288B2 · Karamcheti et al. · 2012 [cited by applicant]
US 8307191B1 · Jain · 2012 [cited by applicant]
US 8346909B2 · Dan et al. · 2013 [cited by applicant]
US 8413161B2 · Blackburn et al. · 2013 [cited by applicant]
US 8423694B2 · Naylor · 2013 [cited by applicant]
US 8424010B2 · Stegaru et al. · 2013 [cited by applicant]
US 8521685B1 · Chatterjee · 2013 [cited by applicant]
US 8544004B2 · Fultheim · 2013 [cited by applicant]
US 8547840B1 · Kumar et al. · 2013 [cited by applicant]
US RE44610E · Krakirian · 2013 [cited by applicant]
US 8650296B1 · Herington et al. · 2014 [cited by applicant]
US RE44818E · Rohit · 2014 [cited by applicant]
US 8693391B2 · Garcia-Martin et al. · 2014 [cited by applicant]
US 8776050B2 · Plouffe · 2014 [cited by applicant]
US 8788672B2 · Heller, Jr. · 2014 [cited by applicant]
US 8832692B2 · Fultheim · 2014 [cited by applicant]
US 9020801B2 · Fultheim · 2015 [cited by applicant]
US 9191435B2 · Nassi · 2015 [cited by applicant]
US 9201689B2 · Scott et al. · 2015 [cited by applicant]
US 9286104B1 · Ghosh et al. · 2016 [cited by applicant]
US 9372752B2 · Das Sharma · 2016 [cited by applicant]
US 9552233B1 · Tsirkin et al. · 2017 [cited by applicant]
US 9774401B1 · Borrill · 2017 [cited by applicant]
US 10187452B2 · Nassi · 2019 [cited by applicant]
US 10353736B2 · Nassi et al. · 2019 [cited by applicant]
US 10812408B1 · Dhoolam et al. · 2020 [cited by applicant]
US 10817347B2 · Nassi et al. · 2020 [cited by applicant]
US 20010056456A1 · Cota-Robles · 2001 [cited by applicant]
US 20020087611A1 · Tanaka et al. · 2002 [cited by applicant]
US 20020129128A1 · Gold · 2002 [cited by applicant]
US 20020166031A1 · Chen · 2002 [cited by applicant]
US 20030009641A1 · Arimilli · 2003 [cited by examiner]
US 20030037185A1 · Davis et al. · 2003 [cited by applicant]
US 20030217234A1 · Rowlands · 2003 [cited by applicant]
US 20040151018A1 · Chen · 2004 [cited by applicant]
US 20040226026A1 · Glass et al. · 2004 [cited by applicant]
US 20040249947A1 · Novaes · 2004 [cited by applicant]
US 20050015430A1 · Rothman et al. · 2005 [cited by applicant]
US 20050039180A1 · Fultheim · 2005 [cited by applicant]
US 20050044301A1 · Vasilevsky · 2005 [cited by applicant]
US 20050193169A1 · Ahluwalia · 2005 [cited by applicant]
US 20050223382A1 · Lippett · 2005 [cited by applicant]
US 20060037017A1 · Accapadi et al. · 2006 [cited by applicant]
US 20060136653A1 · Traut · 2006 [cited by applicant]
US 20060143350A1 · Miloushev et al. · 2006 [cited by applicant]
US 20060242389A1 · Browning et al. · 2006 [cited by applicant]
US 20060259818A1 · Howell · 2006 [cited by applicant]
US 20070094310A1 · Passey · 2007 [cited by applicant]
US 20070186054A1 · Kruckemyer · 2007 [cited by applicant]
US 20070226795A1 · Conti et al. · 2007 [cited by applicant]
US 20070283125A1 · Manczak et al. · 2007 [cited by applicant]
US 20070288718A1 · Cholleti et al. · 2007 [cited by applicant]
US 20070288720A1 · Cholleti et al. · 2007 [cited by applicant]
US 20080005495A1 · Lowe et al. · 2008 [cited by applicant]
US 20080028408A1 · Day et al. · 2008 [cited by applicant]
US 20080109876A1 · Hitomi et al. · 2008 [cited by applicant]
US 20080270659A1 · Grewal et al. · 2008 [cited by applicant]
US 20090037585A1 · Miloushev et al. · 2009 [cited by applicant]
US 20090064158A1 · Carter · 2009 [cited by applicant]
US 20090138887A1 · Uehara et al. · 2009 [cited by applicant]
US 20090172690A1 · Zimmer et al. · 2009 [cited by applicant]
US 20090187904A1 · Serebrin et al. · 2009 [cited by applicant]
US 20090193122A1 · Krishamurthy · 2009 [cited by applicant]
US 20090199177A1 · Edwards et al. · 2009 [cited by applicant]
US 20090217276A1 · Brenner et al. · 2009 [cited by applicant]
US 20090288087A1 · Ringseth et al. · 2009 [cited by applicant]
US 20090320022A1 · Ries et al. · 2009 [cited by applicant]
US 20100031254A1 · Chin et al. · 2010 [cited by applicant]
US 20100161908A1 · Nation · 2010 [cited by applicant]
US 20100235847A1 · Brehmer et al. · 2010 [cited by applicant]
US 20100241785A1 · Chen · 2010 [cited by applicant]
US 20110004729A1 · Akkawi · 2011 [cited by applicant]
US 20110004732A1 · Krakirian · 2011 [cited by applicant]
US 20110004733A1 · Krakirian · 2011 [cited by applicant]
US 20110010709A1 · Anand et al. · 2011 [cited by applicant]
US 20110041131A1 · Srivatsa et al. · 2011 [cited by applicant]
US 20110083134A1 · Song et al. · 2011 [cited by applicant]
US 20110087822A1 · Bennett et al. · 2011 [cited by applicant]
US 20110119422A1 · Grouzdev · 2011 [cited by applicant]
US 20110179132A1 · Mayo et al. · 2011 [cited by applicant]
US 20110179229A1 · Chen · 2011 [cited by applicant]
US 20110191783A1 · Le Moal · 2011 [cited by applicant]
US 20110247000A1 · Eidus et al. · 2011 [cited by applicant]
US 20110277038A1 · Sahita et al. · 2011 [cited by applicant]
US 20110296406A1 · Bhandari et al. · 2011 [cited by applicant]
US 20120020370A1 · Sonnier et al. · 2012 [cited by applicant]
US 20120023311A1 · Yamamoto et al. · 2012 [cited by applicant]
US 20120054762A1 · Moon et al. · 2012 [cited by applicant]
US 20120079232A1 · Hinton et al. · 2012 [cited by applicant]
US 20120096462A1 · Kim · 2012 [cited by applicant]
US 20120117299A1 · Waldspurger et al. · 2012 [cited by applicant]
US 20120174097A1 · Levin · 2012 [cited by applicant]
US 20120185866A1 · Couvee et al. · 2012 [cited by applicant]
US 20120215990A1 · Li et al. · 2012 [cited by applicant]
US 20120239825A1 · Xia · 2012 [cited by applicant]
US 20120272029A1 · Zhang et al. · 2012 [cited by applicant]
US 20120297163A1 · Breternitz et al. · 2012 [cited by applicant]
US 20120303799A1 · Hadas et al. · 2012 [cited by applicant]
US 20130097613A1 · Shin et al. · 2013 [cited by applicant]
US 20130159997A1 · Cawlfield et al. · 2013 [cited by applicant]
US 20130167146A1 · Dong et al. · 2013 [cited by applicant]
US 20130173806A1 · Newton et al. · 2013 [cited by applicant]
US 20130179897A1 · Archer et al. · 2013 [cited by applicant]
US 20130263138A1 · Aho et al. · 2013 [cited by applicant]
US 20130332778A1 · Spracklen · 2013 [cited by applicant]
US 20140059110A1 · Nassi · 2014 [cited by applicant]
US 20140059543A1 · Nassi · 2014 [cited by applicant]
US 20140081918A1 · Srivas et al. · 2014 [cited by applicant]
US 20140082145A1 · Lacapra · 2014 [cited by applicant]
US 20140115601A1 · Suzuki et al. · 2014 [cited by applicant]
US 20140201421A1 · Schoenberg · 2014 [cited by applicant]
US 20140201477A1 · Greenfield et al. · 2014 [cited by applicant]
US 20140237192A1 · Liu et al. · 2014 [cited by applicant]
US 20140244891A1 · Tsirkin · 2014 [cited by applicant]
US 20140245294A1 · Kaul · 2014 [cited by applicant]
US 20140245295A1 · Tsirkin et al. · 2014 [cited by applicant]
US 20140245304A1 · Zheng et al. · 2014 [cited by applicant]
US 20140245444A1 · Lutas et al. · 2014 [cited by applicant]
US 20140337393A1 · Burchall · 2014 [cited by applicant]
US 20150013008A1 · Lukacs et al. · 2015 [cited by applicant]
US 20150033228A1 · Lee et al. · 2015 [cited by applicant]
US 20150052287A1 · Venkatasubramanian et al. · 2015 [cited by applicant]
US 20150058863A1 · Karamanolis · 2015 [cited by applicant]
US 20150127895A1 · Malwankar · 2015 [cited by applicant]
US 20150161048A1 · Patil · 2015 [cited by applicant]
US 20150277908A1 · Bradbury et al. · 2015 [cited by applicant]
US 20150309830A1 · Cawlfield et al. · 2015 [cited by applicant]
US 20150356125A1 · Golander · 2015 [cited by applicant]
US 20160088327A1 · Cronk et al. · 2016 [cited by applicant]
US 20160127492A1 · Malwankar · 2016 [cited by applicant]
US 20160269313A1 · Brooker et al. · 2016 [cited by applicant]
US 20160366183A1 · Smith et al. · 2016 [cited by applicant]
US 20170031819A1 · Venkatasubramanian et al. · 2017 [cited by applicant]
US 20170149921A1 · Nassi et al. · 2017 [cited by applicant]
US 20170249255A1 · Edwards et al. · 2017 [cited by applicant]
US 20170371561A1 · Cai et al. · 2017 [cited by applicant]
US 20180060103A1 · Tsirkin · 2018 [cited by applicant]
US 20180062764A1 · Borrill · 2018 [cited by applicant]
US 20180150232A1 · Tsirkin et al. · 2018 [cited by applicant]
US 20180217859A1 · Triplett · 2018 [cited by applicant]
US 20190370049A1 · Gopalan · 2019 [cited by applicant]
CN 1577294 · 2005 [cited by applicant]
EP 0322117 · 2001 [cited by applicant]
GB 2419701A · 2006 [cited by applicant]
JP 2004355124 · 2004 [cited by applicant]
WO 2015034506 · 2015 [cited by applicant]
WO 2015034508 · 2015 [cited by applicant]
Aho et al., “Principles of Optimal Page Replacement”, from the Journal of the Association for Computing Machinery, vol. 18, No. 1, Jan. 1971. [cited by applicant]
ALNÆS et al., “Scalable Coherent Interface”, Jan. 1990. [cited by applicant]
Ami Litman, “The DUNIX Distributed Operating System”, Jan. 1988. [cited by applicant]
Amza et al., “TreadMarks: Shared Memory Computing on Networks of Workstations”, 1996. [cited by applicant]
Amza et al., “TreadMarks: Shared Memory Computing on Networks of Workstations”, Feb. 1996. [cited by applicant]
Andrew W. Wilson Jr., “Hierarchical Cache / Bus Architecture for Shared Memory Multiprocessors”, 1987. [cited by applicant]
Andrew Wilkins Wilson Jr., “Organization and Statistical Simulation of Hierarchical Multiprocessors”, Department of Electrical and Computer Engineering, Carnegie-Mellon University, Aug. 1985. (part 1 of 4). [cited by applicant]
Andrew Wilkins Wilson Jr., “Organization and Statistical Simulation of Hierarchical Multiprocessors”, Department of Electrical and Computer Engineering, Carnegie-Mellon University, Aug. 1985. (part 2 of 4). [cited by applicant]
Andrew Wilkins Wilson Jr., “Organization and Statistical Simulation of Hierarchical Multiprocessors”, Department of Electrical and Computer Engineering, Carnegie-Mellon University, Aug. 1985. (part 3 of 4). [cited by applicant]
Andrew Wilkins Wilson Jr., “Organization and Statistical Simulation of Hierarchical Multiprocessors”, Department of Electrical and Computer Engineering, Carnegie-Mellon University, Aug. 1985. (part 4 of 4). [cited by applicant]
Aral et al. Process Control Structures for Multiprocessors. Encore Computer Corporation. IEEE, 1991. [cited by applicant]
Aral, Ziya et al. “Variable Weight Processes with Flexible Shared Resources ?.” (1989). [cited by applicant]
Assaf Schuster, Resume, Jul. 2017. [cited by applicant]
Author Unknown, “A research and development strategy for high performance computing”, Executive Office of the President, Office of Science and Technology Policy, Nov. 20, 1987. [cited by applicant]
Author Unknown, “IBM System/360 Model 67: Functional Characteristics”, IBM Systems Reference Library, First Edition, 1967. [cited by applicant]
Author Unknown, “IBM Time Sharing System: Command System User's Guide”, IBM Systems Reference Library, Aug. 1976. (1 of 3). [cited by applicant]
Author Unknown, “IBM Time Sharing System: Command System User's Guide”, IBM Systems Reference Library, Aug. 1976. (2 of 3). [cited by applicant]
Author Unknown, “IBM Time Sharing System: Command System User's Guide”, IBM Systems Reference Library, Aug. 1976. (3 of 3). [cited by applicant]
Author Unknown, “Last Transaction Safety on MongoDB with-out compromising on Performance”, downloaded from http://www.plexistor.com/wp-content/uploads/2016/02/Last_Transaction_Safety_On_MongoDB_WP_Long.pdf, Feb. 2016. [cited by applicant]
Author Unknown, “Multimax Technical Summary”, Encore Computer Corporation, May 1985. [cited by applicant]
Author Unknown, “System/360 Model 67: Time Sharing System Preliminary Technical Summary”, IBM Systems Reference Library, 1966. [cited by applicant]
Author Unknown, Technical Summary, Kendall Square Research, 1992. [cited by applicant]
Author Unknown. Features—multicians.org/features.html. Multics Software Features. Jul. 21, 2012. [cited by applicant]
Barak et al., “MOS: A Multicomputer Distributed Operating System”, The Hebrew University of Jerusalem, Software—Practice and Experience, vol. 15(8), 725-737, Aug. 1985. [cited by applicant]
Beck et al. Harness: a next generation distributed virtual machine. Future Generation Computer Systems 15 (1999). pp. 571-582. [cited by applicant]
Bell et al., “The Encore Continuum: a complete distributed work station—multiprocessor computing environment”, National Computer Conference, 1985. [cited by applicant]
Bendtsen et al. Experience With the KSR-1 Parallel Computer. Jul. 1996. [cited by applicant]
Bensoussan et al. The Multics Virtual Memory: Concepts and Design. Association for Computing Machinery, Inc., 1972. [cited by applicant]
Bilas et al., “Shared virtual memory clusters: bridging the cost-performance gap between SMPs and hardware DSM systems”, Journal of Parallel and Distributed Computing, Oct. 19, 2001. [cited by applicant]
Blakeney et al. An Application-Oriented Multiprocessing System: II Design Characteristics of the 9020 System. IBM Systems Journal, vol. 6, No. 2. 1967, part 2. [cited by applicant]
Boykin et al. Programming under Mach. UNIX and Open Systems Series. pp. 63-97.Copyright (c) 1993 by Addison-Wesley Publishing Company, Inc. 1993. [cited by applicant]
Breit et al. Technical Applications on the KSR1: High Performance and Ease of Use. IEEE, 1993. [cited by applicant]
Bugnion et al., “Disco: Running Commodity Operating Systems on Scalable Multiprocessors”, Computer Systems Laboratory, Stanford University, 1997. [cited by applicant]
Büttner et al., “ARTS of PEACE—A High-Performance Middleware Layer for Parallel Distributed Computing”, Mar. 31, 1995. [cited by applicant]
Buyya et al., “Single System Image (SSI)”, The International Journal of High Performance Computing Applications, vol. 15, No. 2, 2001. [cited by applicant]
Buzen et al., “The evolution of virtual machine architecture”, Honeywell Information Systems, Inc. and Harvard University, National Computer Conference, 1973. [cited by applicant]
C. Gordon Bell, “Multis: A New Class of Multiprocessor Computers”, Science, vol. 228, pp. 462-467, Apr. 26, 1985. [cited by applicant]
Carl J. Young. Extended Architecture and Hypervisor Performance. IBM VM/370 Development Group. Published in: Proceedings of the workshop on virtual computer systems. pp. 177-183 Cambridge, Massachusetts, USA—Mar. 26-27,… [cited by applicant]
Chapman et al., “vNUMA: A Virtual Shared-Memory Multiprocessor”, 2009. [cited by applicant]
Chen et al., “A Transparent Remote Paging Model for Vitual Machines”, Department of Computer Science and Technology at Peking University, and Department of Computer Science at Michigan Technological University, 2008. [cited by applicant]
Chen et al., “When Virtual Is Better Than Real”, 2001. [cited by applicant]
Chester Gordon Bell. Three Decades of Multiprocessors. Feb. 1991. [cited by applicant]
Chester Gordon Bell. Darpa Proposal: Encore Computer Corporation Research and Development in High Performance and Fail Soft Multiprocessors for Parallelism; and Systolic Array Processors for Signal Processing. Mar. 10, … [cited by applicant]
Chris Mellor, “Flexi-Plexistor's software-defined memory roadmap”, Apr. 25, 2016. [cited by applicant]
Chris Mellor, “Plexistor rolls out storage-defined memory for the masses”, Jan. 27, 2016. [cited by applicant]
Chris Mellor, “Plexistor's latest box sounds a lot like flash memory as virtual DRAM”, Jul. 27, 2015. [cited by applicant]
Cisternino et al. “Using NVMe Flash as DRAM Expansion for TPC-C workload with ScaleMP vSMP Foundation FLX” Technical Report. Universita Di Pisa. Jul. 28, 2016. [cited by applicant]
Corbató et al., “Introduction and Overview of the Multics System”, 1965. [cited by applicant]
Corbató et al., “Multics—The first seven years”, Spring Joint Computer Conference, 1972. [cited by applicant]
Dave Taylor. Distributed Operating Systems—An Overview of Current Research. Jun. 13, 1998. [cited by applicant]
David B. Gustavson, “Applications for the Scalable Coherent Interface”, Stanford Linear Accelerator Center, Apr. 1990. [cited by applicant]
David B. Gustavson. [RPRWG] Dave James. Jan. 18, 2008. [cited by applicant]
David J. Schanin. The Design and Development of a Very High Speed System Bus—The Encore Multimax Nanobus. IEEE, 1986. [cited by applicant]
Edmnd Burke. An Overview of System Software for the KSR1. Kendall Square Research Corporation, Waltham, MA. IEEE. 1993. [cited by applicant]
Ehab S. Al-Shaer, Distributed Memory Management: Design Issues and Future Trends, Mar. 19, 1993. [cited by applicant]
Einar Rustad, “numascale: NumaConnect”, Sep. 2013. [cited by applicant]
Elizabeth Corcoran, “Strategic computing: a status report”, Apr. 1987. [cited by applicant]
Feeley et al., “Implementing Global Memory Management in a Workstation Cluster”, Department of Computer Science and Engineering at University of Washington, and Chandramohan A. Thekkath DEC Systems Research Center, Dec.… [cited by applicant]
Fleisch et al., Mirage: A Coherent Distributed Shared Memory Design, ACM, 1989, pp. 211-223. [cited by applicant]
Frank et al. The KSR1: Bridging the Gap Between Shared Memory and MPPs. IEEE, 1993. [cited by applicant]
Fraser et al. 3Leaf Virtual Server Software Architecture. Copyright (c) 2004 3Leaf Networks. [cited by applicant]
Fu et al., “Distributed shared arrays: A distributed virtual machine with mobility support for reconfiguration”, Wayne State University, Sep. 1, 2003. [cited by applicant]
Gabriel Southern. Analysis of SMP VM CPU Scheduling. George Mason University. 2008. [cited by applicant]
Gardner Hendrie, “Oral History of Gordon Bell”, Computer History Museum, Jun. 23, 2005. [cited by applicant]
Gertner et al., “Symmetric Parallel Processing”, 1990. [cited by applicant]
Ghormley et al., “GLUnix: a Global Layer Unix for a Network of Workstations”, University of California at Berkeley, Aug. 14, 1997. [cited by applicant]
Giloi W.K., Hastedt C., Schoen F., Schroeder-Preikschat W. (1991) A distributed implementation of shared virtual memory with strong and weak coherence. In: Bode A. (eds) Distributed Memory Computing. EDMCC 1991. Lecture… [cited by applicant]
Golander et al., Storage Industry Summit: Convergence of Storage and Memory Developing the Needed Ecosystem, NVDIMM Panel, Jan. 20, 2016. [cited by applicant]
Goodman et al. Scalability and Its Application to Multicube. Computer Sciences Technical Report #835. Mar. 1989. [cited by applicant]
Gordon Bell, “Scalable, Parallel Computers: Alternatives, Issues, and Challenges”, International Journal of Parallel Programming, vol. 22, No. 1, 1994. [cited by applicant]
Govil et al., “Cellular Disco: resource management using virtual clusters on shared-memory multiprocessors”, 17th ACM Symposium on Operating Systems Principles, Dec. 1999. [cited by applicant]
Gregory F. Pfister, “An Introduction to the InfiniBand Architecture”, 2001. [cited by applicant]
Gregory F. Pfister. In Search of Clusters The Ongoing Battle in Lowly Parallel Computing. Chapters. 6-7, 11, 12.11.4.2, 13. Second Edition. (c) 1998 Prentice Hall PTR. 1998 (part 1 of 3). [cited by applicant]
Gregory F. Pfister. In Search of Clusters The Ongoing Battle in Lowly Parallel Computing. Chapters. 6-7, 11, 12.11.4.2, 13. Second Edition. (c) 1998 Prentice Hall PTR. 1998 (part 2 of 3). [cited by applicant]
Gregory F. Pfister. In Search of Clusters The Ongoing Battle in Lowly Parallel Computing. Chapters. 6-7, 11, 12.11.4.2, 13. Second Edition. (c) 1998 Prentice Hall PTR. 1998 (part 3 of 3). [cited by applicant]
Gregory Pfister, “Aspects of the InfiniBand Architecture”, IBM Server Technology & Architecture, Austin, TX, Oct. 11, 2001. [cited by applicant]
Healy et al. Single System Image: A Survey. Article in Journal of Parallel and Distributed Computing. Feb. 2016. [cited by applicant]
History of IBM Mainframe Operating Systems. From Wikipedia, the free encyclopedia. https://web.archive.org/web/20120622012058/https://en.wikipedia.org/wiki/History_of_IBM_mainframe_operating_systems. Jun. 22, 2012. [cited by applicant]
Holley et al., “VM/370 asymmetric multiprocessing”, IBM Systems Journal, vol. 18, No. 1, 1979. [cited by applicant]
Hu et al. The Memory and Communication Subsystem of Virtual Machines for Cluster Computing. Jan. 2002. [cited by applicant]
Hudzia et al., “Memory Aggregation For KVM”, Hecatonchire Project, Nov. 2012. [cited by applicant]
Hwang et al. Distributed and Cloud Computing From Parallel Processing to the Internet of Things. Oct. 17, 2011. [cited by applicant]
Intel. Intel 64 and IA-32 Architectures Software Developer's Manual, May 2019. [cited by applicant]
Intel. White Paper—System Memory at a Fraction of the DRAM Cost. “Software-Defined Memory for SCM”. retrieved from archive.com on Sep. 20, 2016. [cited by applicant]
Ion Stoica. Virtual Machines Disco and Xen (Lecture 10, cs262a). UC Berkeley, Sep. 28, 2016. [cited by applicant]
J. F. Keeley. An Application-Oriented Multiprocessing System: I Introduction. IBM Systems Journal, vol. 6, No. 2. 1967, part 1. [cited by applicant]
Janjua et al. CEJVM: “Cluster Enabled Java Virtual Machine”. Faculty of Computer Science & Engineering, GIK Institute, Topi, Pakistan. IEEE, Sep. 2002. [cited by applicant]
Jeff Dike. User Mode Linux. Bruce Peren's Open Source Series. Chapter 12. Copyright (c) 2006 Pearson Education, Inc. Apr. 2006. [cited by applicant]
Jeff Dike. UML Clustering. Sep. 14, 2001. [cited by applicant]
Jiang et al., “SODA: a Service-On-Demand Architecture for Application Service Hosting Utility Platforms”, Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC'03), 2003. [cited by applicant]
Jim Elliot. IBM Mainframes—45+ Years of Evolution. (c) 2010 IBM Corporation. 2010. [cited by applicant]
Juhász et al., “Distributed and Parallel Systems: Cluster and Grid Computing”, the Kluwer international series in engineering and computer science, 2005. [cited by applicant]
Konthothanassis et al., “VM-Based Shared Memory on Low-Latency, Remote-Memory-Access Networks”, Nov. 1996. [cited by applicant]
Kai Hwang. Advanced Computer Architecture: Parallelism, Scalability, Programmability. Chapter 7. Copyright (c) 1993 McGraw-Hill, Inc. [cited by applicant]
Kaneda et al. A Virtual Machine Monitor for Providing a Single System Image. 2006. [cited by applicant]
King et al., “Operating System Support for Virtual Machines”, Proceedings of the 2003 USENIX Technical Conference, 2003. [cited by applicant]
Kronenberg et al., “VAXclusters: A Closely-Coupled Distributed System”, ACM Transactions on Computer Systems, vol. 4, No. 2, pp. 130-146, May 1986. [cited by applicant]
Lenoski et al. The Stanford Dash Multiprocessor. IEEE, Mar. 1992. [cited by applicant]
Li et al., “Memory Coherence in Shared Virtual Memory Systems”, ACM Transactions on Computer Systems, vol. 7, No. 4, pp. 321-359, Nov. 1989. [cited by applicant]
Li et al., “Memory Coherence in Shared Virtual Memory Systems”, Distributed Information Processing, 2014. [cited by applicant]
Li et al., “Memory Coherence in Shared Virtual Memory Systems”, Yale University, 1986. [cited by applicant]
Liu et al., “Design and Implementation of a Single System Image Operating System for Ad Hoc Networks”, Department of Computer Science, Cornell University, Jun. 2005. [cited by applicant]
Ma et al., “JESSICA: Java-Enable Single-System-Image Computing Architecture”, The University of Hong Kong, Oct. 2000. [cited by applicant]
Maurice Herlihy. Wait-Free Synchronization. ACM Transactions on Programming Languages and Systems, vol. 11, No. 1. pp. 124-149. Jan. 1991. [cited by applicant]
Michael James Carlton, “Multiple-Bus, Scaleable, Shared-Memory Multiprocessors”, University of California at Berkeley, 1995. [cited by applicant]
Michael T. Alexander, “Organization and features of the Michigan terminal system”, Spring Joint Computer Conference, 1972. [cited by applicant]
Nitzberg et al., Distributed Shared Memory: A Survey of Issues and Algorithms, 1991 IEEE, 1991. [cited by applicant]
Oracle. Oracle Buys Virtual Iron—Adds advanced virtualization management technology to enhance Oracle VM. Jun. 15, 2009. [cited by applicant]
Osisek et al. ESA/390 Interpretive-Execution Architecture, Foundation for VM/ESA. IBM Systems Journal, vol. 30, No. 1. Feb. 1991. [cited by applicant]
Ossanna et al. Communications and Input/Output Switching in a Multiplex Computing System. Fall Joint Computer Conference, 1965. [cited by applicant]
Peng et al. DVMM: a Distributed VMM for Supporting Single System Image on Clusters. The 9th international Conference for Young Computer Scientists. IEEE, 2008. [cited by applicant]
Pete Keleher, “CVM: The Coherent Virtual Machine”, University of Maryland, Aug. 1, 1998. [cited by applicant]
Peter J. Denning. The Working Set Model for Program Behavior. Massachusetts Institute of Technology, Cambridge, Massachusetts. Communications of the ACM. vol. 11, No. 5. May 1968. [cited by applicant]
Pieter J. Muller, “An Environment for Distributed Programming on a Multicomputer”, a thesis submitted to the department of computer science of the University of Stellenbosch in partial fulfillment of the requirements fo… [cited by applicant]
Pinkston et al., “InfiniBand: The ‘De Facto’ Future Standard for System and Local Area Networks or Just a Scalable Replacement for PCI Buses?”, Cluster Computing 6, pp. 95-104, 2003. [cited by applicant]
Popek and Goldberg Virtualization Requirements. From Wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/Popek_and_Goldberg_virtualization_requirements. Jun. 22, 2012. [cited by applicant]
Popek et al. Formal Requirements for Virtualizable Third Generation Architectures. Communications of the ACM, vol. 17, No. 7. Jul. 1974. [cited by applicant]
Popek, Gerald J. et al. “LOCUS—A Network Transparent, High Reliability Distributed System.” SOSP (1981). [cited by applicant]
Protić et al., “A Survey of Distributed Shared Memory Systems”, Proceedings of the 28th Annual Hawaii International Conference on System Sciences, 1995. [cited by applicant]
Protić et al., “Distributed Shared Memory: Concepts and Systems”, University of Belgrade, 1996. [cited by applicant]
R. A. MacKinnon, “The changing virtual machine environment: Interfaces to real hardware, virtual hardware, and other virtual machines”, 1979. [cited by applicant]
R. J. Creasy. The Origin of the VM/370 Time-Sharing System. IBM J. Res. Develop. vol. 25, No. 5. Sep. 1981. [cited by applicant]
Rajkumar Buyya, “PARMON: a portable and scalable monitoring system for clusters”, Monash University, 2000. [cited by applicant]
Ramachandran et al. Scalability Study of the KSR-1—Appeared in Parallel Computing, vol. 22, pp. 739-759. 1996. [cited by applicant]
Reports Leading to the National High Performance Computing Program: “A Research and Development Strategy for High Performance Computing”, Nov. 1987; “The U.S. Supercomputer Industry,” Dec. 1987; “The Federal High Perfor… [cited by applicant]
Reports Leading to the National High Performance Computing Program: “A Research and Development Strategy for High Performance Computing”, Nov. 1987; “The U.S. Supercomputer Industry,” Dec. 1987; “The Federal High Perfor… [cited by applicant]
Reports Leading to the National High Performance Computing Program: “A Research and Development Strategy for High Performance Computing”, Nov. 1987; “The U.S. Supercomputer Industry,” Dec. 1987; “The Federal High Perfor… [cited by applicant]
Rich Oehler, “Ideas for a Dependable ‘Industry Standard Architecture’ Platform”, Newisys, Inc., Jan. 27, 2005. [cited by applicant]
Richard A. Jenkins, “New Approaches in Parallel Computing”, Computers in Physics, 1989. [cited by applicant]
Robert P. Goldberg. Survey of Virtual Machine Research. Honeywell Information Systems and Harvard University. Jun. 1974. [cited by applicant]
Robert P. Goldberg. Survey of Virtual Machine Research (2). Honeywell Information Systems and Harvard University. Jun. 1974. [cited by applicant]
Robin Bloor. “ScaleMP—The Dawn of the Virtualized Data Center”. The Bloor Group—Product Watch. Sep. 16, 2013. [cited by applicant]
Roland et al. Strategic Computing—Darpa and the Quest for Machine Intelligence, 1983-1993. Chapter 5 and Chapter 9. (c)2002 Massachusetts Institute of Technology. 2002. [cited by applicant]
Rudolph et al., “Dynamic Decentralized Cache Schemes for MIMD Parallel Processors”, Carnegie-Mellon University, 1984. [cited by applicant]
ScaleMP. ScaleMP's Software-Defined Memory for NVME and Intel Optane Technology to be Presented at Intel Developer Forum. Aug. 16, 2016. [cited by applicant]
ScaleMP. System Memory at a Fraction of the DRAM Cost. Software-Defined Memory for SCM. retrieved from archive.org on May 8, 2016. [cited by applicant]
ScaleMP. vSMP Foundation for Flash Expansion. May 2016. [cited by applicant]
ScaleMP. vSMP Foundation for Memory Expansion. May 2016. [cited by applicant]
Scales et al., “Towards Transparent and Efficient Software Distributed Shared Memory”, Oct. 1997. [cited by applicant]
Scott Lurndal. 3Leaf Virtual Server—Low-Level Architectural Design. May 9, 2018. [cited by applicant]
Sirer et al. Distributed Virtual Machines: A System Architecture for Network Computing. Dept. of Computer Science & Engineering, University of Washington. 1998. [cited by applicant]
Southern et al. FPGA Implementation of High Throughput Circuit for Trial Division by Small Primes. George Mason University. 2007. [cited by applicant]
Brodawa et al., “An efficient virtual machine implementation”, Wayne State University, National Computer Conference, 1973. [cited by applicant]
Steier et al. Mind Matters—A Tribute to Allen Newell. Chapter 6. Copyright (c) 1996 by Lawrence Associates, Inc. 1996. [cited by applicant]
Tanenbaum et al., Distributed Systems: Principles and Paradigms, 2002, pp. 28-32, Prentice-Hall, Inc. [cited by applicant]
Tanenbaum et al., Distributed Systems: Principles and Paradigms, Oct. 20, 2003, 1st Edition, pp. 28-32, Pearson Education Ltd. [cited by applicant]
Tetzlaff et al., “VM/370, Attached Processor, and multiprocessor performance study”, IBM Systems Journal, vol. 23, No. 4, 1984. [cited by applicant]
Tod Newcombe, “Public-Sector Multiprocessing Emerges”, Nov. 30, 1995. [cited by applicant]
USENIX Association. Proceedings of the 5th Annual Linux Showcase & Conference. Oakland, CA. Nov. 5-10, 2001. [cited by applicant]
Vasilevsky et al., “LINUX Virtualization on Virtual Iron VFe”, Virtual Iron Software, Inc., 2005. [cited by applicant]
VM (operating system). From Wikipedia, the fee encyclopedia. https://en.wikipedia.org/wiki/VM_(operating_system). Jun. 22, 2012. [cited by applicant]
Walker et al. The LOCUS Distributed Operating System. ACM, 1983. [cited by applicant]
Walker et al. The LOCUS Distributed Operating System. Presentation, 1983. [cited by applicant]
Walker, Bruce J. and Douglas Steel. “Implementing a Full Single System Image UnixWare Cluster: Middleware vs Underware.” PDPTA (1999). [cited by applicant]
Wang et al. NEX: Virtual Machine Monitor Level Single System Support in Xen. 2009 First International Workshop on Education Technology and Computer Science. IEEE, 2009. [cited by applicant]
Whitaker et al. Scale and Performance in the Denali Isolation Kernel. USENIX Association 5th Symposium on Operating Systems Design and Implementation. Dec. 2002. [cited by applicant]
Woodbury et al., “Shared Memory Multiprocessors: The Right Approach to Parallel Processing”, Encore Computer Corporation, IEEE, 1989. [cited by applicant]
Yeo C.S., Buyya R., Pourreza H., Eskicioglu R., Graham P., Sommers F. (2006) Cluster Computing: High-Performance, High-Availability, and High-Throughput Processing on a Network of Computers. In: Zomaya A.Y. (eds) Handbo… [cited by applicant]
Younge et al. Evaluation of SMP Shared Memory Machines for Use With In-Memory and OpenMP Big Data Applications. 2016 IEEE International Parallel and Distributed Processing Symposium Workshops. 2016. [cited by applicant]
Zhu et al., “JESSICA2: A Distributed Java Virtual Machine with Transparent Thread Migration Support”, The University of Hong Kong, 2002. [cited by applicant]
Bell et al., “DSM Perspective: Another Point of View”, Proceedings of the IEEE, vol. 87, No. 3, Mar. 1999. [cited by applicant]
Dean et al., “MapReduce: Simplified Data Processing on Large Clusters”, USENIX Association, OSDI '04: 6th Symposium on Operating Systems Design and Implementation, pp. 137-149, 2004. [cited by applicant]
Ike Nassi, “Scaling the Computer to the Problem: Application Programming with Unlimited Memory”, Cover Feature Computer Design Starts Over, Aug. 2017. [cited by applicant]
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US17/48903, mailed on Jan. 2, 2018, 11 pages. [cited by applicant]
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US18/39835, mailed on Oct. 30, 2018, 8 pages. [cited by applicant]
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US18/48743, mailed on Nov. 14, 2018, 6 pages. [cited by applicant]
Mendel Rosenblum, “The Reincarnation of Virtual Machines”, Jul./Aug. 2004. [cited by applicant]
Ousterhout et al., “The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM” SIGOPS Operating Systems Review, vol. 43, No. 4, pp. 92-105, Dec. 2009. [cited by applicant]
PCT2013/058271, “Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority, or Declaration”, dated Mar. 18, 2014. [cited by applicant]
PCT20131058262, “Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority, or Declaration”, dated Mar. 25, 2014. [cited by applicant]
Extended European Search Report, EP App. No. 17847309.6, Apr. 2, 2020, 07 pages. [cited by applicant]
Hui et al., “Study on Application of Cloud Computing Technology in Nuclear Power Plant Design Platform”, Science and Technology on Reactor System Design Technology Laboratory, Jul. 2013, 4 pages. [cited by applicant]
Wikipedia, “Processor affinity”, available online at <https://en.wikipedia.org/w/index.php? title=Processor_affinity&oldid=730409119>, Jul. 18, 2016, 3 pages. [cited by applicant]