IP Library › Granted Patent US 12,229,605
Granted Patent B2
US 12,229,605 · App. 18/538,171 · Granted Feb 18, 2025

Disaggregated computing for distributed confidential computing environment

Inventors: Reshma Lal (Portland, OR); Pradeep Pappachan (Tualatin, OR); Luis Kida (Beaverton, OR); Soham Jayesh Desai (Rochester, MN); Sujoy Sen (Beaverton, OR); Selvakumar Panneer (Portland, OR); Robert Sharp (Austin, TX)
Assignee: INTEL CORPORATION
G06F9/5083G06F9/3814G06F9/5027G06T1/20G06T1/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,229,605
App. No.
18/538,171
Filed
Dec 13, 2023
Granted
Feb 18, 2025
Kind
B2
Art Unit
2196
USPC
718/105
Abstract

An apparatus to facilitate disaggregated computing for a distributed confidential computing environment is disclosed. The apparatus includes one or more processors to facilitate receiving a manifest corresponding to graph nodes representing regions of memory of a remote client machine, the graph nodes corresponding to a command buffer and to associated data structures and kernels of the command buffer used to initialize a hardware accelerator and execute the kernels, and the manifest indicating a destination memory location of each of the graph nodes and dependencies of each of the graph nodes; identifying, based on the manifest, the command buffer and the associated data structures to copy to the host memory; identifying, based on the manifest, the kernels to copy to local memory of the hardware accelerator; and patching addresses in the command buffer copied to the host memory with updated addresses of corresponding locations in the host memory.

Claims (52)

1. An apparatus comprising:

a processor executing a secure environment comprising a field-programmable gate array (FPGA) driver to interface with an FPGA device that is remote to the apparatus, wherein the processor is to:

enumerate the FPGA device using FPGA enumeration data provided by a remote management controller of the FPGA device, the FPGA enumeration data comprising a configuration space and device details;

load function drivers for the FPGA device in the secure environment;

create corresponding device files in the secure environment based on the FPGA enumeration data; and

handle remote reads and remote writes to the FPGA device via a network transport protocol.

2. The apparatus of claim 1 , wherein the secure environment comprises a trusted execution environment (TEE).

3. The apparatus of claim 1 , wherein the FPGA enumeration data comprises a size of a base address register (BAR) regions utilized by the FPGA device.

4. The apparatus of claim 3 , wherein the processor is further to create the corresponding device files representing the BAR regions of the FPGA device.

5. The apparatus of claim 1 , wherein the processor to handle the remote reads and the remote writes further comprises the processor to:

convert a memory-mapped input/output (MMIO) request received from a host driver of the secure environment into a remote MMIO request, the remote MMIO request comprising at least one of the remote reads or the remote writes;

packetize the remote MMIO request into a packetized remote MMIO request; and

send the packetized remote MMIO request to the FPGA device directly via the network transport protocol.

6. The apparatus of claim 1 , wherein the network transport protocol comprises remote direct memory access (RDMA).

7. The apparatus of claim 5 , wherein the remote management controller of the FPGA device is to:

receive the packetized remote MMIO request;

parse the packetized remote MMIO request;

perform a corresponding memory read or a corresponding memory write to registers of the FPGA device; and

return a status message indicating success or failure of the corresponding memory write or indicating a read response.

8. The apparatus of claim 1 , wherein the remote management controller comprises a message parser to initiate memory read and write requests to the FPGA device and a buffer for storing messages.

9. A method comprising:

enumerating, by a processor a field-programmable gate array (FPGA) device using FPGA enumeration data provided by a remote management controller of the FPGA device, the FPGA enumeration data comprising a configuration space and device details;

loading, by the processor, function drivers for the FPGA device in a secure environment of the processor and create corresponding device files in the secure environment based on the FPGA enumeration data; and

handling, by the processor, remote reads and remote writes to the FPGA device via a network transport protocol, wherein an FPGA driver to interface with an FPGA device.

10. The method of claim 9 , wherein the secure environment comprises a trusted execution environment (TEE).

11. The method of claim 9 , wherein the FPGA enumeration data comprises a size of a base address register (BAR) regions utilized by the FPGA device.

12. The method of claim 9 , wherein handling the remote reads and the remote writes further comprises:

converting a memory-mapped input/output (MMIO) request received from a host driver of the secure environment into a remote MMIO request, the remote MMIO request comprising at least one of the remote reads or the remote writes;

packetizing the remote MMIO request into a packetized remote MMIO request; and

sending the packetized remote MMIO request to the FPGA device directly via the network transport protocol.

13. The method of claim 9 , wherein the network transport protocol comprises remote direct memory access (RDMA).

14. The method of claim 12 , wherein the remote management controller of the FPGA device is to:

receive the packetized remote MMIO request;

parse the packetized remote MMIO request;

perform a corresponding memory read or a corresponding memory write to registers of the FPGA device; and

return a status message indicating success or failure of the corresponding memory write or indicating a read response.

15. The method of claim 9 , wherein the remote management controller comprises a message parser to initiate memory read and write requests to the FPGA device and a buffer for storing messages.

16. A non-transitory machine readable storage medium having stored thereon executable computer program instructions that, when executed by one or more processors, cause the one or more processors to perform operations to:

enumerating, by the one or more processors a field-programmable gate array (FPGA) device using FPGA enumeration data provided by a remote management controller of the FPGA device, the FPGA enumeration data comprising a configuration space and device details;

loading function drivers for the FPGA device in a secure environment of the one or more processors and create corresponding device files in the secure environment based on the FPGA enumeration data; and

handling remote reads and remote writes to the FPGA device via a network transport protocol, wherein an FPGA driver to interface with an FPGA device.

17. The non-transitory machine readable storage medium of claim 16 , wherein the FPGA enumeration data comprises a size of a base address register (BAR) regions utilized by the FPGA device.

18. The non-transitory machine readable storage medium of claim 17 , wherein the operations further comprise creating the corresponding device files representing the BAR regions of the FPGA device.

19. The non-transitory machine readable storage medium of claim 16 , wherein handling the remote reads and the remote writes further comprises:

converting memory-mapped input/output (MMIO) request received from a host driver of the secure environment into a remote MMIO request, the remote MMIO request comprising at least one of the remote reads or the remote writes;

packetizing the remote MMIO request into a packetized remote MMIO request; and

sending the packetized remote MMIO request to the FPGA device directly via the network transport protocol.

20. The non-transitory machine readable storage medium of claim 19 , wherein the remote management controller of the FPGA device is to:

receive the packetized remote MMIO request;

parse the packetized remote MMIO request;

perform a corresponding memory read or a corresponding memory write to registers of the FPGA device; and

return a status message indicating success or failure of the corresponding memory write or indicating a read response.

Continuity (4)
Continuation 17531005 · Nov 19, 2021
Continuation 17133066 · Dec 23, 2020
Provisional Application 63083565 · Sep 25, 2020
Related Publication 20240184639A1 · Jun 6, 2024
References Cited (94)
US 7971236B1 · Lentini · 2011 [cited by applicant]
US 8930717B2 · Smith · 2015 [cited by applicant]
US 10074206B1 · Ingegneri · 2018 [cited by applicant]
US 10482291B2 · Woodall · 2019 [cited by applicant]
US 10649790B1 · Ingegneri · 2020 [cited by applicant]
US 10652108B2 · Guim Bernat · 2020 [cited by applicant]
US 10528765B2 · Smith et al. · 2020 [cited by applicant]
US 10776145B2 · Iyer et al. · 2020 [cited by applicant]
US 11449963B1 · Beeler et al. · 2022 [cited by applicant]
US 11893425B2 · Lal et al. · 2024 [cited by applicant]
US 11941457B2 · Lal et al. · 2024 [cited by applicant]
US 11989595B2 · Lal et al. · 2024 [cited by applicant]
US 12033005B2 · Lal et al. · 2024 [cited by applicant]
US 12093748B2 · Lal et al. · 2024 [cited by applicant]
US 20060168091A1 · Makhervaks et al. · 2006 [cited by applicant]
US 20060259570A1 · Feng et al. · 2006 [cited by applicant]
US 20120254587A1 · Biran et al. · 2012 [cited by applicant]
US 20130162661A1 · Bolz et al. · 2013 [cited by applicant]
US 20140240327A1 · Lustig et al. · 2014 [cited by applicant]
US 20140281169A1 · Mehrotra et al. · 2014 [cited by applicant]
US 20150326684A1 · Takefman et al. · 2015 [cited by applicant]
US 20160093012A1 · Rao et al. · 2016 [cited by applicant]
US 20160147710A1 · Franke et al. · 2016 [cited by applicant]
US 20160342547A1 · Liss · 2016 [cited by examiner]
US 20160358306A1 · Begeman et al. · 2016 [cited by applicant]
US 20170213053A1 · Areno et al. · 2017 [cited by applicant]
US 20170300361A1 · Lanka et al. · 2017 [cited by applicant]
US 20170351639A1 · Borikar · 2017 [cited by applicant]
US 20180082083A1 · Smith et al. · 2018 [cited by applicant]
US 20180205553A1 · Hoppert et al. · 2018 [cited by applicant]
US 20190044519A1 · Atsatt et al. · 2019 [cited by applicant]
US 20190044875A1 · Murty et al. · 2019 [cited by applicant]
US 20190102568A1 · Hausauer et al. · 2019 [cited by applicant]
US 20190179755A1 · Mudumbai et al. · 2019 [cited by applicant]
US 20190286479A1 · Tian et al. · 2019 [cited by applicant]
US 20190355163A1 · Imbrogno et al. · 2019 [cited by applicant]
US 20200004701A1 · Subbarao et al. · 2020 [cited by applicant]
US 20200004993A1 · Volos et al. · 2020 [cited by applicant]
US 20200127836A1 · Pappachan et al. · 2020 [cited by applicant]
US 20200127850A1 · Scarlata et al. · 2020 [cited by applicant]
US 20200132761A1 · Rahardjo et al. · 2020 [cited by applicant]
US 20200167488A1 · Yitbarek et al. · 2020 [cited by applicant]
US 20200211148A1 · Mackinnon · 2020 [cited by applicant]
US 20200218684A1 · Sen et al. · 2020 [cited by applicant]
US 20200226009A1 · Bachmutsky et al. · 2020 [cited by applicant]
US 20200228388A1 · Schulz et al. · 2020 [cited by applicant]
US 20200242258A1 · Smith et al. · 2020 [cited by applicant]
US 20200342112A1 · Plusquellic · 2020 [cited by applicant]
US 20200364516A1 · Krasner et al. · 2020 [cited by applicant]
US 20210117246A1 · Lal et al. · 2021 [cited by applicant]
US 20210406178A1 · Enrici et al. · 2021 [cited by applicant]
US 20220004397A1 · Ringlein et al. · 2022 [cited by applicant]
US 20220019356A1 · Hong et al. · 2022 [cited by applicant]
US 20220100579A1 · Lal et al. · 2022 [cited by applicant]
US 20220100580A1 · Lal et al. · 2022 [cited by applicant]
US 20220100581A1 · Lal et al. · 2022 [cited by applicant]
US 20220100582A1 · Lal et al. · 2022 [cited by applicant]
US 20220100583A1 · Lal et al. · 2022 [cited by applicant]
US 20220100584A1 · Lal et al. · 2022 [cited by applicant]
US 20220206969A1 · Que · 2022 [cited by examiner]
US 20220214912A1 · Julien et al. · 2022 [cited by applicant]
US 20240086258A1 · Lal et al. · 2024 [cited by applicant]
US 20240184639A1 · Lal et al. · 2024 [cited by applicant]
US 20240281302A1 · Lal et al. · 2024 [cited by applicant]
DE 102021207514A1 · 2022 [cited by applicant]
EP 2383648A1 · 2011 [cited by applicant]
EP 3719657A1 · 2020 [cited by applicant]
NL 2029026B1 · 2022 [cited by applicant]
WO 2016101288A1 · 2016 [cited by applicant]
WO 2022066304A1 · 2022 [cited by applicant]
U.S. Appl. No. 17/532,562 “Notice of Allowance” mailed Feb. 27, 2024, 8 pages. [cited by applicant]
U.S. Appl. No. 17/528,374 “Advisory Action” mailed Apr. 9, 2024, 3 pages. [cited by applicant]
U.S. Appl. No. 17/532,569 “Advisory Action” mailed May 2, 2024, 3 pages. [cited by applicant]
Anonymous: CUDA Runtime API version v11.1.74, Chapter 5, section 5.29, Sep. 15, 2020, XP055865248, retrieved from the Internet [retrieved on Jan. 31, 2022]. [cited by applicant]
International Patent Application No. PCT/US2021/045185 “International Preliminary Report on Patentability”, mailed Apr. 6, 2023, 9 pages. [cited by applicant]
International Patent Application No. PCT/US2021/045185 “International Search Report and Written Opinion”, mailed on Dec. 3, 2021, 13 pages. [cited by applicant]
International Patent Application No. PCT/US2021/045185 “Notification Concerning the Availability of the Publication of the International Application”, mailed on Mar. 31, 2022, 1 page. [cited by applicant]
Notice of Grant for Dutch Application No. 2029026 mailed Jul. 27, 2022, 6 pages. [cited by applicant]
Search Report and Written Opinion for Dutch Application No. 2029026 mailed May 25, 2022, 12 pages. [cited by applicant]
U.S. Appl. No. 17/133,066 “Notice of Allowance” mailed Aug. 17, 2023, 9 pages. [cited by applicant]
U.S. Appl. No. 17/525,143 “Non-Final Office Action” mailed Sep. 13, 2023, 19 pages. [cited by applicant]
U.S. Appl. No. 17/525,143 “Notice of Allowance” mailed Dec. 8, 2023, 8 pages. [cited by applicant]
U.S. Appl. No. 17/526,097 “Non-Final Office Action” mailed Sep. 13, 2023, 19 pages. [cited by applicant]
U.S. Appl. No. 17/528,374 “Non-Final Office Action” mailed Sep. 13, 2023, 18 pages. [cited by applicant]
U.S. Appl. No. 17/531,005 “Notice of Allowance” mailed Sep. 27, 2023, 11 pages. [cited by applicant]
U.S. Appl. No. 17/532,562 “Non-Final Office Action” mailed Sep. 26, 2023, 15 pages. [cited by applicant]
U.S. Appl. No. 17/532,569 “Non-Final Office Action” mailed Sep. 26, 2023, 13 pages. [cited by applicant]
Taranov, K. et al. “sRDMA—Efficient NIC-based Authentication and Encryption for Remote Direct Memory Access”, 2020 USENIX Annual Technical Conference, Jul. 15-17, 2020, pp. 691-704. [cited by applicant]
U.S. Appl. No. 17/526,097 “Notice of Allowance” mailed Jan. 24, 2024, 8 pages. [cited by applicant]
U.S. Appl. No. 17/528,374 “Final Office Action” mailed Jan. 29, 2024, 19 pages. [cited by applicant]
U.S. Appl. No. 17/532,569 “Final Office Action” mailed Feb. 23, 2024, 16 pages. [cited by applicant]
U.S. Appl. No. 17/133,066 “Notice of Allowance” mailed May 28, 2024, 10 pages. [cited by applicant]
U.S. Appl. No. 18/511,296 “Non-Final Office Action” mailed Jun. 3, 2024, 11 pages. [cited by applicant]
U.S. Appl. No. 18/511,296 “Notice of Allowance” mailed Aug. 14, 2024, 9 pages. [cited by applicant]
Cited By (1)
US 12,405,838