IP Library Granted Patent US 12,489,718
Granted Patent B2
US 12,489,718 · App. 18/493,541 · Granted Dec 2, 2025

System and method for supporting target groups for congestion control in a private fabric in a high performance computing environment

Inventors: Bjørn Dag Johnsen (Oslo, NO); Ajoy Siddabathuni (Santa Clara, CA); Avneesh Pant (Redwood City, CA); Amarnath Jolad (Redwood City, CA)
Assignee: Oracle International Corporation
H04L49/358H04L41/12H04L41/5003H04L47/17H04L47/2433H04L47/265H04L47/30H04L47/33H04L47/805H04W28/0983G06F9/45558G06F2009/45583G06F2009/45595G06F15/17331H04L49/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,489,718
App. No.
18/493,541
Filed
Oct 24, 2023
Granted
Dec 2, 2025
Kind
B2
Art Unit
2471
USPC
370/229
Abstract

Systems and methods for supporting target groups for congestion control in a private fabric in a high performance computing environment. An exemplary method can provide, at one or more microprocessors, a first subnet, the first subnet comprising a plurality of switches, a plurality of host channel adapters, and a plurality of end nodes, including a plurality of virtual machines. The method can define a target group on one of an inter-switch link or at a port of a switch of the plurality of switches, wherein the target group defines a bandwidth limit on the at least one of an inter-switch link between two switches of the plurality of switches or at a port of a switch of the plurality of switches. The method can provide a target group repository stored in a memory of the host channel adapter where the defined target group in the target group repository is recorded.

Claims (58)

1 . A system for providing congestion control in a high performance computing environment, comprising:

a computer comprising one or more microprocessors and a memory;

wherein the one or more microprocessors perform a method comprising:

defining a target group within a subnet, the target group being defined at a link of the subnet, the target group defining a bandwidth restriction;

directing two or more flows within the subnet, each of the two or more flows utilizing the link at which the target group is defined;

based upon the defined bandwidth restriction of the target group, limiting a flow rate for each of the two or more flows; and

defining a second target group within a subnet, the second target group being defined at a second link of the subnet, the second target group defining a second bandwidth restriction;

wherein the two or more flows are directed on a route within the subnet, the route traversing at least the link at which the target group is defined and the second link at which the second target group is defined;

wherein the target group and the second target group together comprise a hierarchical target group comprising a rate restriction comprising the bandwidth restriction and the second bandwidth restriction.

2 . The system of claim 1 ,

wherein a combined flow rate for the two or more flows is less than or equal to the defined bandwidth restriction.

3 . The system of claim 2 ,

wherein the flow rate for each of the two or more flows is limited based upon a respective priority level of each of the two or more flows.

4 . The system of claim 3 ,

wherein the respective priority level of each of the two or more flows comprises at least one of a quality of service level or a service level agreement.

5 . The system of claim 1 , the method further comprising:

querying, by a node of the subnet, the target group to determine the defined bandwidth restriction; and

wherein the node of the subnet limits the flow rate of at least one of the two or more flows.

6 . The system of claim 5 ,

wherein the node of the subnet limits the flow rate of the at least one of the two or more flows via a rate restriction component of the node.

7 . A method for providing congestion control in a high performance computing environment, comprising:

providing a computer comprising one or more microprocessors and a memory;

defining a target group within a subnet, the target group being defined at a link of the subnet, the target group defining a bandwidth restriction;

directing two or more flows within the subnet, each of the two or more flows utilizing the link at which the target group is defined;

based upon the defined bandwidth restriction of the target group, limiting a flow rate for each of the two or more flows; and

defining a second target group within a subnet, the second target group being defined at a second link of the subnet, the target group defining a second bandwidth restriction;

wherein the two or more flows are directed on a route within the subnet, the route traversing at least the link at which the target group is defined and the second link at which the second target group is defined; and

wherein the target group and the second target group together comprise a hierarchical target group comprised a rate restriction comprising the bandwidth restriction and the second bandwidth restriction.

8 . The method of claim 7 ,

wherein a combined flow rate for the two or more flows is less than or equal to the defined bandwidth restriction.

9 . The method of claim 8 ,

wherein the flow rate for each of the two or more flows is limited based upon a respective priority level of each of the two or more flows.

10 . The method of claim 9 ,

wherein the respective priority level of each of the two or more flows comprises at least one of a quality of service level or a service level agreement.

11 . The method of claim 7 , further comprising:

querying, by a node of the subnet, the target group to determine the defined bandwidth restriction;

wherein the node of the subnet limits the flow rate of at least one of the two or more flows.

12 . The method of claim 11 ,

wherein the node of the subnet limits the flow rate of the at least one of the two or more flows via a rate restriction component of the node.

13 . A non-transitory computer readable storage medium having instructions thereon for providing congestion control in a high performance computing environment, which when read and executed cause a computer to perform steps comprising:

defining a target group within a subnet, the target group being defined at a link of the subnet, the target group defining a bandwidth restriction;

directing two or more flows within the subnet, each of the two or more flows utilizing the link at which the target group is defined;

based upon the defined bandwidth restriction of the target group, limiting a flow rate for each of the two or more flows; and

defining a second target group within a subnet, the second target group being defined at a second link of the subnet, the target group defining a second bandwidth restriction;

wherein the two or more flows are directed on a route within the subnet, the route traversing at least the link at which the target group is defined and the second link at which the second target group is defined; and

wherein the target group and the second target group together comprise a hierarchical target group comprised a rate restriction comprising the bandwidth restriction and the second bandwidth restriction;

wherein the computer comprises one or more microprocessors.

14 . The non-transitory computer readable storage medium of claim 13 ,

wherein a combined flow rate for the two or more flows is less than or equal to the defined bandwidth restriction.

15 . The non-transitory computer readable storage medium of claim 14 ,

wherein the flow rate for each of the two or more flows is limited based upon a respective priority level of each of the two or more flows.

16 . The non-transitory computer readable storage medium of claim 15 ,

wherein the respective priority level of each of the two or more flows comprises at least one of a quality of service level or a service level agreement.

17 . The non-transitory computer readable storage medium of claim 13 , the steps further comprising:

querying, by a node of the subnet, the target group to determine the defined bandwidth restriction;

wherein the node of the subnet limits the flow rate of at least one of the two or more flows.

18 . The non-transitory computer readable storage medium of claim 17 ,

wherein the node of the subnet limits the flow rate of the at least one of the two or more flows via a rate restriction component of the node.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 24, 2023
From: JOHNSEN, BJØRN DAG; SIDDABATHUNI, AJOY; PANT, AVNEESH; JOLAD, AMARNATH
To: ORACLE INTERNATIONAL CORPORATION
Reel/Frame 065328/0006 →
Continuity (4)
Continuation 17744517 · May 13, 2022
Continuation 16872039 · May 11, 2020
Provisional Application 62937594 · Nov 19, 2019
Related Publication 20240056397A1 · Feb 15, 2024
References Cited (107)
US 6978300B1 · Beukema · 2005 [cited by applicant]
US 7107359B1 · Burton · 2006 [cited by applicant]
US 7283803B2 · Karaoguz · 2007 [cited by applicant]
US 8000323B2 · So · 2011 [cited by applicant]
US 8352953B2 · Bozek · 2013 [cited by applicant]
US 8593964B1 · Sinha · 2013 [cited by applicant]
US 8982688B2 · Froese · 2015 [cited by applicant]
US 9336166B1 · Faibish · 2016 [cited by applicant]
US 9444764B2 · Calvo · 2016 [cited by applicant]
US 9548872B2 · Ghanwani · 2017 [cited by applicant]
US 9979637B2 · Hughes · 2018 [cited by applicant]
US 9980179B2 · Coffman · 2018 [cited by applicant]
US 10148567B2 · Holen · 2018 [cited by applicant]
US 10237191B2 · Cheng · 2019 [cited by applicant]
US 10313272B2 · Moxnes · 2019 [cited by applicant]
US 10572509B2 · Song · 2020 [cited by applicant]
US 10756961B2 · Johnsen · 2020 [cited by examiner]
US 10846788B1 · Klein · 2020 [cited by examiner]
US 11005724B1 · Shpigelman · 2021 [cited by applicant]
US 11018945B2 · Lee · 2021 [cited by examiner]
US 11050675B2 · Holm · 2021 [cited by applicant]
US 11256655B2 · Johnsen · 2022 [cited by applicant]
US 11341082B2 · Johnsen · 2022 [cited by applicant]
US 11444881B2 · Johnsen · 2022 [cited by applicant]
US 11700206B2 · Johnsen · 2023 [cited by applicant]
US 11700207B2 · Johnsen · 2023 [cited by applicant]
US 11838217B2 · Johnsen · 2023 [cited by applicant]
US 20020010772A1 · Kusano · 2002 [cited by examiner]
US 20030067940A1 · Edholm · 2003 [cited by applicant]
US 20030174647A1 · Gutierrez · 2003 [cited by applicant]
US 20040025166A1 · Adlung · 2004 [cited by applicant]
US 20040054828A1 · Finch · 2004 [cited by applicant]
US 20050233728A1 · Karaoguz · 2005 [cited by applicant]
US 20070076598A1 · Atkinson · 2007 [cited by applicant]
US 20090324232A1 · So · 2009 [cited by examiner]
US 20110110241A1 · Atkinson · 2011 [cited by applicant]
US 20110138384A1 · Bozek · 2011 [cited by examiner]
US 20110267942A1 · Aybay · 2011 [cited by applicant]
US 20120044803A1 · Zhang · 2012 [cited by applicant]
US 20120163175A1 · Gupta · 2012 [cited by applicant]
US 20130107716A1 · Volpe · 2013 [cited by applicant]
US 20130135999A1 · Bloch · 2013 [cited by applicant]
US 20150074283A1 · Karthikeyan · 2015 [cited by examiner]
US 20150103755A1 · Cui et al. · 2015 [cited by applicant]
US 20150334716A1 · Coffman · 2015 [cited by examiner]
US 20160072817A1 · Makhervaks · 2016 [cited by applicant]
US 20160094450A1 · Ghanwani · 2016 [cited by applicant]
US 20160134544A1 · Roy · 2016 [cited by examiner]
US 20160212063A1 · Calvo · 2016 [cited by examiner]
US 20160246631A1 · Tsirkin · 2016 [cited by applicant]
US 20160267041A1 · Jose · 2016 [cited by applicant]
US 20160373362A1 · Cheng · 2016 [cited by examiner]
US 20170032011A1 · Song · 2017 [cited by examiner]
US 20170214580A1 · Moxnes · 2017 [cited by applicant]
US 20170257312A1 · Johnsen · 2017 [cited by applicant]
US 20170353383A1 · Hughes · 2017 [cited by examiner]
US 20170366460A1 · Friedman · 2017 [cited by applicant]
US 20180157445A1 · Gissin · 2018 [cited by applicant]
US 20190044832A1 · Deval · 2019 [cited by examiner]
US 20190068401A1 · Johnsen · 2019 [cited by applicant]
US 20190149473A1 · Zahid · 2019 [cited by applicant]
US 20190171584A1 · Kan · 2019 [cited by applicant]
US 20190222531A1 · Lin · 2019 [cited by examiner]
US 20190319848A1 · Johnsen · 2019 [cited by applicant]
US 20190372857A1 · Gandhi · 2019 [cited by applicant]
US 20200382435A1 · Holm · 2020 [cited by examiner]
US 20200389398A1 · Johnsen et al. · 2020 [cited by applicant]
US 20210149835A1 · Johnsen · 2021 [cited by applicant]
US 20210152477A1 · Johnsen · 2021 [cited by applicant]
US 20210152479A1 · Johnsen · 2021 [cited by applicant]
US 20210152494A1 · Johnsen · 2021 [cited by applicant]
US 20220147483A1 · Johnsen · 2022 [cited by applicant]
US 20220385595A1 · Han · 2022 [cited by examiner]
CN 102195752 · 2011 [cited by applicant]
CN 107852376 · 2018 [cited by applicant]
JP 2000041060 · 2000 [cited by applicant]
JP 2017184095 · 2017 [cited by applicant]
JP 2018537004 · 2018 [cited by applicant]
JP 2019503595 · 2019 [cited by applicant]
JP 2019507520 · 2019 [cited by applicant]
Japan Patent Office, Notice of Reasons for Refusal dated Mar. 15, 2024 for Japanese Patent Application No. 2022-529052 , 11 pages. [cited by applicant]
Escamilla, Jose V et al., “ICARO: Congestion Isolation in Networks-On-Chip”, 2014 Eighth IEEE/ACM International Symposium on Networks-on-Chip (NoCS), pp. 159-166, 8 pages. [cited by applicant]
“Understanding RoCEv2 Congestion Management”, published Dec. 3, 2018, retrieved from: <https://community.mellanox.com/s/article/understanding-rocev2-congestion-management>, 5 pages. [cited by applicant]
European Patent Office, International Search Report and Written Opinion of the International Searching Authority mailed Sep. 15, 2020 for PCT Application No. PCT/US2020/045273, 12 pages. [cited by applicant]
European Patent Office, International Search Report and Written Opinion of the International Searching Authority mailed Oct. 12, 2020 for PCT Application No. PCT/US2020/045271, 12 pages. [cited by applicant]
European Patent Office, International Search Report and Written Opinion of the International Searching Authority mailed Oct. 16, 2020 for PCT Application No. PCT/US2020/045272, 14 pages. [cited by applicant]
European Patent Office, International Search Report and Written Opinion of the International Searching Authority mailed Oct. 19, 2020 for PCT Application No. PCT/US2020/045270, 12 pages. [cited by applicant]
China National Intellectual Property Administration, Notification of the First Office Action dated Nov. 15, 2023 for Chinese Patent Application No. 202080080842.6 , 9 pages. [cited by applicant]
The State Intellectual Property Office of People's Republic of China, Notification of the First Office Action dated Jan. 6, 2025 for Chinese Patent Application No. 202080080798.9 , 4 pages. [cited by applicant]
Intellectual Property India, Examination report under sections 12 & 13 of the Patents Act, 1970 and the Patents Rules, 2003 dated Feb. 28, 2025 for India Patent Application No. 202247026232 , 8 pages. [cited by applicant]
Intellectual Property India, Examination report under sections 12 & 13 of the Patents Act, 1970 and the Patents Rules, 2003 dated Mar. 10, 2025 for India Patent Application No. 202247026458 , 5 pages. [cited by applicant]
Japan Patent Office, Decision to Grant a Patent dated Aug. 23, 2024 for Japanese Patent Application No. 2022-529054 , 5 pages. [cited by applicant]
Japan Patent Office, Decision to Grant a Patent dated Aug. 28, 2024 for Japanese Patent Application No. 2022-529052 , 5 pages. [cited by applicant]
European Patent Office, Communication under Rule 71(3) EPC dated Jul. 5, 2024 for European Patent Application No. 20760694.8 , 8 pages. [cited by applicant]
European Patent Office, Communication under Rule 71(3) EPC dated Nov. 11, 2024 for European Patent Application No. 20761390.2 , 8 pages. [cited by applicant]
Japan Patent Office, Decision to Grant a Patent dated Nov. 7, 2024 for Japanese Patent Application No. 2022-529078 , 5 pages. [cited by applicant]
The State Intellectual Property Office of People's Republic of China, Notification to Grant Patent Right for Invention dated Oct. 30, 2024 for Chinese Patent Application No. 202080080811.0 , 3 pages. [cited by applicant]
Japan Patent Office, Notice of Reasons for Refusal dated Mar. 15, 2024 for Japanese Patent Application No. 2022-529054 , 11 pages. [cited by applicant]
China National Intellectual Property Administration, Notification of the First Office Action dated Nov. 29, 2023 for Chinese Patent Application No. 202080080852.X , 4 pages. [cited by applicant]
The State Intellectual Property Office of People's Republic of China, Notification to Grant Patent Right for Invention dated Feb. 4, 2024 for Chinese Patent Application No. 202080080852.X , 6 pages. [cited by applicant]
China National Intellectual Property Administration, Notification of the First Office Action dated Jan. 4, 2024 for Chinese Patent Application No. 202080080811.0 , 11 pages. [cited by applicant]
The State Intellectual Property Office of People's Republic of China, Notification to Grant Patent Right for Invention dated Apr. 9, 2024 for Chinese Patent Application No. 202080080842.6 , 3 pages. [cited by applicant]
Japan Patent Office, Notice of Reasons for Refusal dated May 7, 2024 for Japanese Patent Application No. 2022-529078 , 17 pages. [cited by applicant]
Intellectual Property India, Examination report under sections 12 & 13 of the Patents Act, 1970 and the Patents Rules, 2003 dated Aug. 13, 2024 for Indian Patent Application No. 202247026474 , 7 pages. [cited by applicant]
European Patent Office, Communication pursuant to Article 94(3) EPC dated Apr. 11, 2025 for European Patent Application No. 20761031.2 , 8 pages. [cited by applicant]
Japan Patent Office, Notice of Reasons for Refusal dated Apr. 22, 2025 for Japanese Patent Application No. 2024-098702 , 6 pages. [cited by applicant]
China National Intellectual Property Administration, Notification to Grant Patent Right dated Jun. 2, 2025 for Chinese Patent Application No. 202080080798.9 , 4 pages. [cited by applicant]