IP Library Granted Patent US 12,388,743
Granted Patent B2
US 12,388,743 · App. 15/959,261 · Granted Aug 12, 2025

Transaction controls for supplying control plane data to managed hardware forwarding element

Inventor: Anupam Chanda (San Jose, CA)
Assignee: VMware LLC
H04L45/42H04L45/745H04L49/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,388,743
App. No.
15/959,261
Granted
Aug 12, 2025
Kind
B2
Abstract

Some embodiments provide novel methods for controllers to communicate with managed hardware forwarding elements (MHFEs) in a transactional manner. The transactional communication methods of some embodiments ensure that an MHFE receives the entirety of a control plane update that a controller supplies to it, before the MHFE starts to modify its data plane forwarding data and operations. The transactional communication methods of some embodiments provide one or more transactional boundary controls to the controllers to define complete control plane data set updates. In some embodiments, the transactional controls ensure that an MHFE receives all of a control plane update before it starts to modify its data plane forwarding data. Controllers use one transactional control in some embodiments when they define logical forwarding elements (e.g., logical switches or routers) on the MHFEs.

Claims (42)

1. A non-transitory machine readable medium storing a program for configuring a managed hardware forwarding element (MHFE) to implement a logical forwarding element (LFE) along with a plurality of other managed forwarding elements operating outside of the MHFE, and to communicatively couple the LFE with a private network, the program comprising sets of instructions for:

providing a plurality of data tuples for defining the LFE on the MHFE, without providing a data tuple for binding the LFE to a port of the MHFE to ensure that all data tuples of the plurality of data tuples are received at the MHFE before the MHFE creates forwarding records for the LFE;

after providing the plurality of data tuples for defining the LFE, receiving confirmation from the MHFE that the MHFE has processed the plurality of data tuples; and

after receiving the confirmation, providing the data tuple that binds the LFE to a port of the MHFE in order to couple the LFE with the private network through the MHFE.

2. The non-transitory machine readable medium of claim 1 , wherein the program further comprises a set of instructions for ordering the data tuples for the MHFE such that the data tuple for binding the LFE to the MHFE port is last.

3. The non-transitory machine readable medium of claim 1 , wherein:

the MHFE comprises (i) a module that retrieves the provided data tuples and uses the provided data tuples to configure dataplane forwarding records of the MHFE to implement the LFE and (ii) a forwarding engine that uses the dataplane forwarding records to process data messages that the MHFE receives; and

the module does not configure the dataplane forwarding records based on the provided data tuples until the data tuple for binding the LFE to the MHFE port is provided.

4. The non-transitory machine readable medium of claim 1 , wherein:

the MHFE comprises a database with a plurality of tables for storing the provided data tuples;

the database uses a hardware VTEP (VXLAN Tunnel End Point) schema and

the data tuples are provided to the database on the MHFE by using an OVSdb (open vswitch database) protocol.

5. The non-transitory machine readable medium of claim 1 , wherein the MHFE port is associated with multiple VLANs (virtual local area networks), and the binding data tuple binds the LFE to a VLAN associated with the MHFE port.

6. The non-transitory machine readable medium of claim 1 , wherein the other forwarding elements that implement the LFE with the MHFE comprise a plurality of software forwarding elements that execute on host computers on which data compute nodes also execute.

7. A method for configuring a managed hardware forwarding element (MHFE) to implement a logical forwarding element (LFE) along with a plurality of other managed forwarding elements operating outside of the MHFE, and to communicatively couple the LFE with a private network, the method comprising:

providing a plurality of data tuples to the MHFE to implement the LFE along with the plurality of other managed forwarding elements operating outside of the MHFE, said plurality of data tuples provided to the MHFE without providing a data tuple for binding the LFE to a port of the MHFE;

after providing the plurality of data tuples for implementing the LFE, receiving confirmation from the MHFE that the MHFE has processed the plurality of data tuples to implement the LFE on the MHFE; and

after receiving the confirmation, providing the data tuple that binds the LFE to a port of the MHFE in order to couple the LFE with the private network through the MHFE.

8. The method of claim 7 further comprising ordering the data tuples for the MHFE such that the data tuple for binding the LFE to the MHFE port is last.

9. The method of claim 7 :

wherein the MHFE comprises (i) a module that retrieves the provided data tuples and uses the provided data tuples to configure dataplane forwarding records of the MHFE to implement the LFE, and (ii) a forwarding engine that uses the dataplane forwarding records to process data messages that the MHFE receives; and

the module does not configure the dataplane forwarding records based on the provided data tuples until the data tuple for binding the LFE to the MHFE port is provided.

10. The method of claim 7 , wherein:

the MHFE comprises a database with a plurality of tables for storing the provided data tuples;

the database uses a hardware VTEP (VXLAN Tunnel End Point) schema; and

the data tuples are provided to the database on the MHFE by using an OVSdb (open vswitch database) protocol.

11. The method of claim 7 , wherein the MHFE port is associated with multiple VLANs (virtual local area networks), and the binding data tuple binds the LFE to a VLAN associated with the MHFE port.

12. The method of claim 7 , wherein the other forwarding elements that implement the LFE with the MHFE comprise a plurality of software forwarding elements that execute on host computers on which data compute nodes also execute.

13. A non-transitory machine readable medium storing a program for configuring a managed hardware forwarding element (MHFE) to implement a logical forwarding element (LFE) along with a plurality of other managed forwarding elements operating outside of the MHFE, and to communicatively couple the LFE with a private network, the program comprising sets of instructions for:

providing a first set of data tuples to define the LFE on the MHFE and to define a state of the LFE as incomplete;

providing a second set of data tuples to complete the definition of the LFE on the MHFE;

after providing the first and second sets of data tuples, receiving confirmation that the first and second sets of data tuples have been processed by the MHFE; and

after receiving confirmation, providing a third set of data tuples to change the LFE state to complete.

14. The non-transitory machine readable medium of claim 13 , wherein

the MHFE comprises (i) a module that retrieves the provided data tuples and uses the provided data tuples to configure dataplane forwarding records of the MHFE to implement the LFE and (ii) a forwarding engine that uses the dataplane forwarding records to process data messages that the MHFE receives; and

the module does not configure the dataplane forwarding records based on the provided data tuples until the state of the LFE is set to complete.

15. The non-transitory machine readable medium of claim 13 , wherein the MHFE comprises a database with a plurality of tables for storing the provided data tuples.

16. The non-transitory machine readable medium of claim 15 , wherein the database uses a hardware VTEP (VXLAN Tunnel End Point) schema, and the data tuples are provided to the database on the MHFE by using an OVSdb (open vswitch database) protocol.

17. The non-transitory machine readable medium of claim 16 , wherein:

the LFE is a logical switch;

the hardware VTEP schema specifies a logical switch table that comprises a record for the logical switch; and

the logical switch record includes a Boolean field for specifying whether the state of the logical switch is complete.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2025
From: CHANDA, ANUPAM
To: NICIRA, INC.
Reel/Frame 070911/0100 →
MERGER Recorded Jan 27, 2025
From: NICIRA, INC.
To: VMWARE LLC
Reel/Frame 070187/0487 →
Continuity (2)
Continuation 14970463 · Dec 15, 2015
Related Publication 20180241672A1 · Aug 23, 2018
References Cited (220)
US 5526484A · Casper et al. · 1996 [cited by applicant]
US 5751967A · Raab et al. · 1998 [cited by applicant]
US 6243394B1 · Deng · 2001 [cited by applicant]
US 6640251B1 · Wiget et al. · 2003 [cited by applicant]
US 6850529B1 · Wong et al. · 2005 [cited by applicant]
US 7463639B1 · Rekhter · 2008 [cited by applicant]
US 7933198B1 · Pan · 2011 [cited by applicant]
US 7937438B1 · Miller et al. · 2011 [cited by applicant]
US 8046456B1 · Miller et al. · 2011 [cited by applicant]
US 8161095B2 · Manion et al. · 2012 [cited by applicant]
US 8345688B2 · Zhou et al. · 2013 [cited by applicant]
US 8386642B2 · Elzur · 2013 [cited by applicant]
US 8589919B2 · Smith et al. · 2013 [cited by applicant]
US 8874876B2 · Bhadra et al. · 2014 [cited by applicant]
US 8897134B2 · Kern et al. · 2014 [cited by applicant]
US 8943490B1 · Jain et al. · 2015 [cited by applicant]
US 8964528B2 · Casado et al. · 2015 [cited by applicant]
US 9014181B2 · Lakshman et al. · 2015 [cited by applicant]
US 9032095B1 · Traina et al. · 2015 [cited by applicant]
US 9100285B1 · Choudhury et al. · 2015 [cited by applicant]
US 9130870B1 · Swierk et al. · 2015 [cited by applicant]
US 9154433B2 · Koponen et al. · 2015 [cited by applicant]
US 9178833B2 · Koponen et al. · 2015 [cited by applicant]
US 9306843B2 · Koponen et al. · 2016 [cited by applicant]
US 9319375B2 · Gross, IV et al. · 2016 [cited by applicant]
US 9331940B2 · Balus et al. · 2016 [cited by applicant]
US 9369426B2 · Koponen et al. · 2016 [cited by applicant]
US 9397946B1 · Yadav · 2016 [cited by applicant]
US 9455901B2 · Davie et al. · 2016 [cited by applicant]
US 9485149B1 · Traina et al. · 2016 [cited by applicant]
US 9577927B2 · Hira et al. · 2017 [cited by applicant]
US 9621461B2 · Sun · 2017 [cited by applicant]
US 9633040B2 · Lee · 2017 [cited by applicant]
US 9667541B2 · Song · 2017 [cited by applicant]
US 9699070B2 · Davie et al. · 2017 [cited by applicant]
US 9755965B1 · Yadav et al. · 2017 [cited by applicant]
US 9819581B2 · Chanda et al. · 2017 [cited by applicant]
US 9847938B2 · Chanda et al. · 2017 [cited by applicant]
US 9917799B2 · Chanda · 2018 [cited by applicant]
US 9923815B2 · Assarpour et al. · 2018 [cited by applicant]
US 9942058B2 · Chanda et al. · 2018 [cited by applicant]
US 9948577B2 · Chanda · 2018 [cited by applicant]
US 9967182B2 · Chanda et al. · 2018 [cited by applicant]
US 9979593B2 · Chanda et al. · 2018 [cited by applicant]
US 9992112B2 · Chanda · 2018 [cited by applicant]
US 9998324B2 · Chanda et al. · 2018 [cited by applicant]
US 9998375B2 · Chanda · 2018 [cited by applicant]
US 10153965B2 · Davie et al. · 2018 [cited by applicant]
US 10182035B2 · Basler · 2019 [cited by applicant]
US 10200343B2 · Chanda et al. · 2019 [cited by applicant]
US 10230576B2 · Chanda et al. · 2019 [cited by applicant]
US 10250553B2 · Chanda et al. · 2019 [cited by applicant]
US 10263828B2 · Chanda et al. · 2019 [cited by applicant]
US 10313186B2 · Wang et al. · 2019 [cited by applicant]
US 10411912B2 · Chanda et al. · 2019 [cited by applicant]
US 10447618B2 · Chanda · 2019 [cited by applicant]
US 10554484B2 · Chanda et al. · 2020 [cited by applicant]
US 10659431B2 · Basler · 2020 [cited by applicant]
US 10764111B2 · Chanda et al. · 2020 [cited by applicant]
US 10805152B2 · Chanda et al. · 2020 [cited by applicant]
US 10924386B2 · Davie et al. · 2021 [cited by applicant]
US 11005683B2 · Chanda et al. · 2021 [cited by applicant]
US 11032234B2 · Chanda et al. · 2021 [cited by applicant]
US 11095513B2 · Wang et al. · 2021 [cited by applicant]
US 11196682B2 · Chanda · 2021 [cited by applicant]
US 11245621B2 · Chanda et al. · 2022 [cited by applicant]
US 11368431B2 · Basler · 2022 [cited by applicant]
US 11502898B2 · Chanda et al. · 2022 [cited by applicant]
US 11522788B2 · Davie et al. · 2022 [cited by applicant]
US 20020001310A1 · Mai et al. · 2002 [cited by applicant]
US 20020065919A1 · Taylor et al. · 2002 [cited by applicant]
US 20020093952A1 · Gonda · 2002 [cited by applicant]
US 20030041170A1 · Suzuki · 2003 [cited by applicant]
US 20030163645A1 · Tremblay et al. · 2003 [cited by applicant]
US 20040037279A1 · Zelig et al. · 2004 [cited by applicant]
US 20040052216A1 · Roh · 2004 [cited by applicant]
US 20040267866A1 · Carollo et al. · 2004 [cited by applicant]
US 20050163115A1 · Dontu et al. · 2005 [cited by applicant]
US 20050262132A1 · Morita et al. · 2005 [cited by applicant]
US 20060092940A1 · Ansari et al. · 2006 [cited by applicant]
US 20070028039A1 · Gupta et al. · 2007 [cited by applicant]
US 20070115961A1 · Dorenbosch et al. · 2007 [cited by applicant]
US 20080189769A1 · Casado et al. · 2008 [cited by applicant]
US 20080215586A1 · Pruet · 2008 [cited by applicant]
US 20090135823A1 · Liu et al. · 2009 [cited by applicant]
US 20090006603A1 · Duponchel et al. · 2009 [cited by applicant]
US 20090150527A1 · Tripathi et al. · 2009 [cited by applicant]
US 20090161547A1 · Riddle et al. · 2009 [cited by applicant]
US 20110026521A1 · Gamage et al. · 2011 [cited by applicant]
US 20110075674A1 · Li et al. · 2011 [cited by applicant]
US 20110090911A1 · Hao et al. · 2011 [cited by applicant]
US 20110131361A1 · Takura · 2011 [cited by applicant]
US 20110158086A1 · Sakauchi et al. · 2011 [cited by applicant]
US 20110206047A1 · Donthamsetty et al. · 2011 [cited by applicant]
US 20110286326A1 · Awano · 2011 [cited by applicant]
US 20110299537A1 · Saraiya et al. · 2011 [cited by applicant]
US 20110317559A1 · Kern et al. · 2011 [cited by applicant]
US 20120011106A1 · Reid et al. · 2012 [cited by applicant]
US 20120084406A1 · Kumbalimutt · 2012 [cited by examiner]
US 20120147898A1 · Koponen et al. · 2012 [cited by applicant]
US 20120215876A1 · Ohtake · 2012 [cited by applicant]
US 20120236761A1 · Yang et al. · 2012 [cited by applicant]
US 20120278802A1 · Nilakantan et al. · 2012 [cited by applicant]
US 20120290694A9 · Marl et al. · 2012 [cited by applicant]
US 20120303835A1 · Kempf et al. · 2012 [cited by applicant]
US 20130044636A1 · Koponen et al. · 2013 [cited by applicant]
US 20130044641A1 · Koponen et al. · 2013 [cited by applicant]
US 20130051399A1 · Zhang et al. · 2013 [cited by applicant]
US 20130054773A1 · Onoue · 2013 [cited by applicant]
US 20130058208A1 · Pfaff et al. · 2013 [cited by applicant]
US 20130058225A1 · Casado · 2013 [cited by examiner]
US 20130058229A1 · Casado et al. · 2013 [cited by applicant]
US 20130058350A1 · Fulton · 2013 [cited by applicant]
US 20130058351A1 · Casado et al. · 2013 [cited by applicant]
US 20130060929A1 · Koponen et al. · 2013 [cited by applicant]
US 20130103817A1 · Koponen et al. · 2013 [cited by applicant]
US 20130103818A1 · Koponen et al. · 2013 [cited by applicant]
US 20130114466A1 · Koponen et al. · 2013 [cited by applicant]
US 20130121209A1 · Padmanabhan · 2013 [cited by examiner]
US 20130132533A1 · Padmanabhan · 2013 [cited by examiner]
US 20130223454A1 · Dunbar et al. · 2013 [cited by applicant]
US 20130287026A1 · Davie · 2013 [cited by applicant]
US 20130315246A1 · Zhang et al. · 2013 [cited by applicant]
US 20130322453A1 · Allan · 2013 [cited by applicant]
US 20130329548A1 · Nakil et al. · 2013 [cited by applicant]
US 20130336134A1 · Bao · 2013 [cited by examiner]
US 20140029451A1 · Nguyen · 2014 [cited by applicant]
US 20140029618A1 · Janardhanan · 2014 [cited by applicant]
US 20140071986A1 · Isobe · 2014 [cited by applicant]
US 20140101467A1 · Jubran et al. · 2014 [cited by applicant]
US 20140195666A1 · Dumitriu · 2014 [cited by examiner]
US 20140201738A1 · Choi et al. · 2014 [cited by applicant]
US 20140229605A1 · Besser · 2014 [cited by applicant]
US 20140247753A1 · Koponen · 2014 [cited by examiner]
US 20140269290A1 · Cors et al. · 2014 [cited by applicant]
US 20140269683A1 · Bhagavathiperumal et al. · 2014 [cited by applicant]
US 20140269709A1 · Benny et al. · 2014 [cited by applicant]
US 20140301391A1 · Krishnan et al. · 2014 [cited by applicant]
US 20140348161A1 · Koponen · 2014 [cited by examiner]
US 20140362775A1 · Steiner · 2014 [cited by examiner]
US 20150009992A1 · Zhang · 2015 [cited by applicant]
US 20150016287A1 · Ganichev et al. · 2015 [cited by applicant]
US 20150016469A1 · Ganichev et al. · 2015 [cited by applicant]
US 20150100560A1 · Davie et al. · 2015 [cited by applicant]
US 20150100675A1 · Davie et al. · 2015 [cited by applicant]
US 20150100704A1 · Davie et al. · 2015 [cited by applicant]
US 20150103838A1 · Zhang et al. · 2015 [cited by applicant]
US 20150124586A1 · Pani · 2015 [cited by applicant]
US 20150124809A1 · Edsall · 2015 [cited by examiner]
US 20150124821A1 · Chu et al. · 2015 [cited by applicant]
US 20150195178A1 · Bhattacharya et al. · 2015 [cited by applicant]
US 20150215189A1 · Lim · 2015 [cited by applicant]
US 20150281075A1 · Park et al. · 2015 [cited by applicant]
US 20150312117A1 · Behera · 2015 [cited by examiner]
US 20150326425A1 · Natarajan et al. · 2015 [cited by applicant]
US 20150326524A1 · Tankala et al. · 2015 [cited by applicant]
US 20150334696A1 · Gu · 2015 [cited by examiner]
US 20150372906A1 · Tirat · 2015 [cited by applicant]
US 20150379150A1 · Duda · 2015 [cited by examiner]
US 20150381428A1 · Ong · 2015 [cited by applicant]
US 20160014039A1 · Reddy et al. · 2016 [cited by applicant]
US 20160014073A1 · Reddy et al. · 2016 [cited by applicant]
US 20160173535A1 · Barabash · 2016 [cited by examiner]
US 20160197824A1 · Lin et al. · 2016 [cited by applicant]
US 20160212222A1 · Bultema et al. · 2016 [cited by applicant]
US 20160232019A1 · Shah et al. · 2016 [cited by applicant]
US 20160308690A1 · Chanda et al. · 2016 [cited by applicant]
US 20160337896A1 · Rajagopalan · 2016 [cited by examiner]
US 20160352613A1 · Narayanan · 2016 [cited by examiner]
US 20160352633A1 · Kapadia et al. · 2016 [cited by applicant]
US 20160380812A1 · Chanda et al. · 2016 [cited by applicant]
US 20170034002A1 · Sinn · 2017 [cited by applicant]
US 20170034051A1 · Chanda et al. · 2017 [cited by applicant]
US 20170034052A1 · Chanda et al. · 2017 [cited by applicant]
US 20170034053A1 · Chanda et al. · 2017 [cited by applicant]
US 20170063608A1 · Wang et al. · 2017 [cited by applicant]
US 20170085502A1 · Biruduraju · 2017 [cited by applicant]
US 20170093617A1 · Chanda et al. · 2017 [cited by applicant]
US 20170093618A1 · Chanda et al. · 2017 [cited by applicant]
US 20170093636A1 · Chanda et al. · 2017 [cited by applicant]
US 20170093646A1 · Chanda et al. · 2017 [cited by applicant]
US 20170093686A1 · Uttaro et al. · 2017 [cited by applicant]
US 20170093758A1 · Chanda · 2017 [cited by applicant]
US 20170126615A1 · Chanda et al. · 2017 [cited by applicant]
US 20170171055A1 · Wang et al. · 2017 [cited by applicant]
US 20170171077A1 · Chanda · 2017 [cited by applicant]
US 20170171078A1 · Chanda · 2017 [cited by applicant]
US 20170171113A1 · Chanda · 2017 [cited by applicant]
US 20170208097A1 · Kirby et al. · 2017 [cited by applicant]
US 20170272316A1 · Johnson et al. · 2017 [cited by applicant]
US 20170317928A1 · Gude et al. · 2017 [cited by applicant]
US 20170339054A1 · Yadav · 2017 [cited by examiner]
US 20170366446A1 · Davie et al. · 2017 [cited by applicant]
US 20180007004A1 · Basler · 2018 [cited by applicant]
US 20180007005A1 · Chanda et al. · 2018 [cited by applicant]
US 20180013687A1 · Jin · 2018 [cited by examiner]
US 20180026895A1 · Wang et al. · 2018 [cited by applicant]
US 20180183730A1 · Chanda · 2018 [cited by applicant]
US 20180219699A1 · Chanda et al. · 2018 [cited by applicant]
US 20180241622A1 · Chanda et al. · 2018 [cited by applicant]
US 20180248796A1 · Chanda et al. · 2018 [cited by applicant]
US 20190089622A1 · Davie et al. · 2019 [cited by applicant]
US 20190141011A1 · Basler · 2019 [cited by applicant]
US 20190207904A1 · Chanda et al. · 2019 [cited by applicant]
US 20190238393A1 · Chanda et al. · 2019 [cited by applicant]
US 20190260637A1 · Wang et al. · 2019 [cited by applicant]
US 20190356512A1 · Chanda et al. · 2019 [cited by applicant]
US 20200021541A1 · Chanda · 2020 [cited by applicant]
US 20200366553A1 · Chanda et al. · 2020 [cited by applicant]
US 20200396206A1 · Basler · 2020 [cited by applicant]
US 20210243112A1 · Davie et al. · 2021 [cited by applicant]
US 20220078111A1 · Chanda et al. · 2022 [cited by applicant]
EP 1154601A1 · 2001 [cited by applicant]
EP 1653688A1 · 2006 [cited by applicant]
JP 2003069609A · 2003 [cited by applicant]
WO 2005112390A1 · 2005 [cited by applicant]
WO 2008095010A1 · 2008 [cited by applicant]
Al-Fares, Mohammad, et al., “A Scalable, Commodity Data Center Network Architecture,” SIGCOMM '08, Aug. 17-22, 2008, 12 pages, Seattle, Washington, USA. [cited by applicant]
Graubner, Pablo, et al., “Cloud Computing: Energy-Efficient Virtual Machine Consolidation,” IT Professional, Mar. 2013, 7 pages, vol. 15, Issue 2, IEEE. [cited by applicant]
Pfaff, Ben, et al., “The Open vSwitch Database Management Protocol,” draft-pfaff-ovsdb-proto-00, Aug. 20, 2012, 34 pages, available at https://tools.ietf.org/html/draft-pfaff-ovsdb-proto-00. [cited by applicant]