IP Library Granted Patent US 12,192,109
Granted Patent B2
US 12,192,109 · App. 18/371,454 · Granted Jan 7, 2025

Virtualized QoS support in software defined networks

Inventors: Dexiang Wang (Sunnyvale, CA); Sreeram Kumar Ravinoothala (San Jose, CA); Yong Wang (San Jose, CA); Jerome Catrouillet (Palo Alto, CA)
Assignee: VMware LLC
H04L47/2416H04L43/0888H04L47/122H04L47/125H04L47/2433H04L47/781
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,192,109
App. No.
18/371,454
Granted
Jan 7, 2025
Kind
B2
Abstract

A network system that implements quality of service (QoS) by rate limiting at a logical network entity is provided. The logical network entity includes multiple transport nodes for transporting network traffic in and out of the logical network entity. The system monitors traffic loads of the multiple transport nodes of the logical network entity. The system allocates a local CR and a local BS to each of the multiple transport nodes. The allocated local CR and the local BS are determined based on the CR and BS parameters of the logical network entity and based on the monitored traffic loads. Each transport node of the logical network entity in turn controls an amount of data being processed by the transport node based on a token bucket value that is computed based on the local CR and the local BS of the transport node.

Claims (28)

1. A method performing distributed rate limiting on a plurality of host computers, the method comprising:

identifying a QoS (Quality of Service) value that is collectively assigned to a plurality of managed forwarding elements (MFEs) executing on the plurality of host computers;

based on the identified QoS value, assigning, to each host computer, an initial rate limit specifying an amount of packets that the MFE executing on the host computer is allowed to process;

monitoring amount of traffic processed by the MFEs;

based on the monitoring, adjusting the assigned rate limit on a set of two or more of the host computers to adjust how the MFEs on the set of host computers process the packets;

wherein assigning the initial rate limit comprises providing each host computer with a committed rate (CR) parameter and a burst size (BS) parameter that conjunctively control a rate at which the MFE on the host computer forwards packets.

2. The method of claim 1 , wherein the MFEs are configured to implement a logical forwarding element (LFE) for a logical network that is defined over a physical network that comprises the MFEs.

3. The method of claim 2 , wherein the QOS value is a QoS value associated with the LFE.

4. The method of claim 3 , wherein the LFE is a logical router, and the MFEs are managed software routers executing on the host computers.

5. The method of claim 1 , wherein the rate of packets forwarded by each host computer's MFE is based on a token bucket value that is computed based on the CR and BS parameters provided to each host computer.

6. The method of claim 1 , wherein adjusting the assigned rate comprises adjusting the CR and BS parameters for each of the two or more host computers.

7. The method of claim 1 , wherein assigning the initial rate limit further comprises allocating CR and BS parameters evenly among the plurality host computers.

8. The method of claim 1 , wherein the BS parameter for each host computer caps an amount of data allowed to be processed by the MFE of the host computer.

9. The method of claim 1 , wherein said monitoring comprises periodically collecting traffic load information from each host computer.

10. A non-transitory machine readable medium storing a program for performing distributed rate limiting on a plurality of host computers, the program for execution by at least one processing unit and comprising sets of instructions for:

identifying a QoS (Quality of Service) value that is collectively assigned to a plurality of managed forwarding elements (MFEs) executing on the plurality of host computers;

based on the identified QoS value, assigning, to each host computer, an initial rate limit specifying an amount of packets that the MFE executing on the host computer is allowed to process;

monitoring amount of traffic processed by the MFEs;

based on the monitoring, adjusting the assigned rate limit on a set of two or more of the host computers to adjust how the MFEs on the set of host computers process the packets;

wherein the set of instructions for assigning the initial rate limit comprises a set of instructions for providing each host computer with a committed rate (CR) parameter and a burst size (BS) parameter that conjunctively control a rate at which the MFE on the host computer forwards packets.

11. The non-transitory machine readable medium of claim 10 , wherein the MFEs are configured to implement a logical forwarding element (LFE) for a logical network that is defined over a physical network that comprises the MFEs.

12. The non-transitory machine readable medium of claim 11 , wherein the QoS value is a QOS value associated with the LFE.

13. The non-transitory machine readable medium of claim 12 , wherein the LFE is a logical router, and the MFEs are managed software routers executing on the host computers.

14. The non-transitory machine readable medium of claim 10 , wherein the rate of packets forwarded by each host computer's MFE is based on a token bucket value that is computed based on the CR and BS parameters provided to each host computer.

15. The non-transitory machine readable medium of claim 10 , wherein the set of instructions for adjusting the assigned rate comprises a set of instructions for adjusting the CR and BS parameters for each of the two or more host computers.

16. The non-transitory machine readable medium of claim 10 , wherein the set of instructions for assigning the initial rate limit further comprises a set of instructions for allocating CR and BS parameters evenly among the plurality host computers.

17. The non-transitory machine readable medium of claim 10 , wherein the BS parameter for each host computer caps an amount of data allowed to be processed by the MFE of the host computer.

18. The non-transitory machine readable medium of claim 10 , wherein said the set of instructions for monitoring comprises a set of instructions for periodically collecting traffic load information from each host computer.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2024
From: WANG, DEXIANG; RAVINOOTHALA, SREERAM KUMAR; WANG, YONG; CATROUILLET, JEROME
To: VMWARE, INC.
Reel/Frame 069184/0808 →
CHANGE OF NAME Recorded Feb 27, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 066692/0103 →
Continuity (3)
Continuation 17569276 · Jan 5, 2022
Provisional Application 63208394 · Jun 8, 2021
Related Publication 20240015105A1 · Jan 11, 2024
References Cited (100)
US 7516487B1 · Szeto et al. · 2009 [cited by applicant]
US 7734895B1 · Agarwal et al. · 2010 [cited by applicant]
US 8509068B2 · Begall et al. · 2013 [cited by applicant]
US 9378049B1 · Johnson · 2016 [cited by examiner]
US 9942144B1 · Ramalingam et al. · 2018 [cited by applicant]
US 10382329B1 · Thomas et al. · 2019 [cited by applicant]
US 10778721B1 · Holbrook et al. · 2020 [cited by applicant]
US 10785056B1 · Mathur et al. · 2020 [cited by applicant]
US 10833992B1 · Dickinson · 2020 [cited by applicant]
US 10880121B1 · Nirmala et al. · 2020 [cited by applicant]
US 10897420B1 · Pianigiani et al. · 2021 [cited by applicant]
US 11265292B1 · Leviseur · 2022 [cited by applicant]
US 11483246B2 · Wang et al. · 2022 [cited by applicant]
US 11539633B2 · Wang et al. · 2022 [cited by applicant]
US 11599395B2 · Wang et al. · 2023 [cited by applicant]
US 20030081546A1 · Agrawal et al. · 2003 [cited by applicant]
US 20030172145A1 · Nguyen · 2003 [cited by applicant]
US 20050047420A1 · Tanabe · 2005 [cited by examiner]
US 20050066166A1 · Chin et al. · 2005 [cited by applicant]
US 20050195964A1 · Hahn · 2005 [cited by examiner]
US 20050265376A1 · Chapman · 2005 [cited by examiner]
US 20070253439A1 · Iny · 2007 [cited by examiner]
US 20080222730A1 · Ford et al. · 2008 [cited by applicant]
US 20090016336A1 · LaVigne et al. · 2009 [cited by applicant]
US 20090119750A1 · Sembugamoorthy et al. · 2009 [cited by applicant]
US 20090129271A1 · Ramankutty et al. · 2009 [cited by applicant]
US 20090161682A1 · Johnson et al. · 2009 [cited by applicant]
US 20090300759A1 · Wang et al. · 2009 [cited by applicant]
US 20100135287A1 · Hosain et al. · 2010 [cited by applicant]
US 20110176418A1 · Gershinsky · 2011 [cited by examiner]
US 20120051218A1 · Mohandoss et al. · 2012 [cited by applicant]
US 20120081580A1 · Côtéet al. · 2012 [cited by applicant]
US 20130028228A1 · Nakayama · 2013 [cited by examiner]
US 20130058229A1 · Casado et al. · 2013 [cited by applicant]
US 20130125120A1 · Zhang et al. · 2013 [cited by applicant]
US 20130185436A1 · Carlin et al. · 2013 [cited by applicant]
US 20140052877A1 · Mao · 2014 [cited by applicant]
US 20140153422A1 · Nambiar et al. · 2014 [cited by applicant]
US 20140156720A1 · Janakiraman et al. · 2014 [cited by applicant]
US 20150016286A1 · Ganichev et al. · 2015 [cited by applicant]
US 20150016469A1 · Ganichev et al. · 2015 [cited by applicant]
US 20150113133A1 · Srinivas et al. · 2015 [cited by applicant]
US 20150244630A1 · Madem et al. · 2015 [cited by applicant]
US 20150256466A1 · Roitshtein · 2015 [cited by examiner]
US 20150263899A1 · Tubaltsev et al. · 2015 [cited by applicant]
US 20150271303A1 · Neginhal et al. · 2015 [cited by applicant]
US 20150281277A1 · May et al. · 2015 [cited by applicant]
US 20160014634A1 · Liu et al. · 2016 [cited by applicant]
US 20160057166A1 · Chesla · 2016 [cited by applicant]
US 20160080211A1 · Anand et al. · 2016 [cited by applicant]
US 20160105333A1 · Lenglet et al. · 2016 [cited by applicant]
US 20160164910A1 · Tang · 2016 [cited by applicant]
US 20160182255A1 · Liu et al. · 2016 [cited by applicant]
US 20160218918A1 · Chu et al. · 2016 [cited by applicant]
US 20160335129A1 · Behera · 2016 [cited by examiner]
US 20170118042A1 · Bhattacharya et al. · 2017 [cited by applicant]
US 20170149648A1 · Yang et al. · 2017 [cited by applicant]
US 20170317954A1 · Masurekar et al. · 2017 [cited by applicant]
US 20180048537A1 · Gaikwad · 2018 [cited by examiner]
US 20180091547A1 · Pierre · 2018 [cited by applicant]
US 20180157515A1 · Malloy et al. · 2018 [cited by applicant]
US 20180176181A1 · Fu et al. · 2018 [cited by applicant]
US 20180262599A1 · Firestone · 2018 [cited by applicant]
US 20180279161A1 · Chen et al. · 2018 [cited by applicant]
US 20180285151A1 · Wang et al. · 2018 [cited by applicant]
US 20180309640A1 · Nagarajan et al. · 2018 [cited by applicant]
US 20180359134A1 · Pech et al. · 2018 [cited by applicant]
US 20190007330A1 · Browne et al. · 2019 [cited by applicant]
US 20190014051A1 · Briscoe et al. · 2019 [cited by applicant]
US 20190044809A1 · Willis et al. · 2019 [cited by applicant]
US 20190081899A1 · Mundkur et al. · 2019 [cited by applicant]
US 20190108068A1 · Britkin et al. · 2019 [cited by applicant]
US 20190182367A1 · Kim et al. · 2019 [cited by applicant]
US 20190334868A1 · Tewari et al. · 2019 [cited by applicant]
US 20200278892A1 · Nainar et al. · 2020 [cited by applicant]
US 20200296139A1 · Fainberg et al. · 2020 [cited by applicant]
US 20200413283A1 · Shen et al. · 2020 [cited by applicant]
US 20210064429A1 · Stetter, Jr. et al. · 2021 [cited by applicant]
US 20210067489A1 · Jayawardena et al. · 2021 [cited by applicant]
US 20210119970A1 · Raphael et al. · 2021 [cited by applicant]
US 20210176168A1 · Eckert et al. · 2021 [cited by applicant]
US 20210218677A1 · Wang et al. · 2021 [cited by applicant]
US 20210227424A1 · Wang et al. · 2021 [cited by applicant]
US 20210255903A1 · Wang et al. · 2021 [cited by applicant]
US 20210297451A1 · Raphael et al. · 2021 [cited by applicant]
US 20210306354A1 · Raghuramu et al. · 2021 [cited by applicant]
US 20210399920A1 · Sundararajan et al. · 2021 [cited by applicant]
US 20210406255A1 · Raghuramu et al. · 2021 [cited by applicant]
US 20220070102A1 · Wang et al. · 2022 [cited by applicant]
US 20220158922A1 · Srivastava · 2022 [cited by applicant]
US 20220264360A1 · Chen · 2022 [cited by examiner]
US 20220393983A1 · Wang et al. · 2022 [cited by applicant]
US 20230041869A1 · Wang et al. · 2023 [cited by applicant]
US 20230130529A1 · Wang et al. · 2023 [cited by applicant]
US 20230168947A1 · Wang et al. · 2023 [cited by applicant]
US 20240031230A1 · Gaikwad · 2024 [cited by examiner]
CN 106059960A · 2016 [cited by applicant]
CN 109547502A · 2019 [cited by applicant]
EP 2015527 · 2018 [cited by examiner]
Seddiki, M. Said, et al., “FlowQoS: QoS for the Rest of US,” HotSDN '14, Aug. 22, 2014, 2 pages, ACM, Chicago, IL, USA. [cited by applicant]