IP Library Granted Patent US 9,847,934
Granted Patent B2
US 9,847,934 · App. 14/481,277 · Granted Dec 19, 2017

Reducing packet reordering in flow-based networks

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,847,934
App. No.
14/481,277
Granted
Dec 19, 2017
Kind
B2
Abstract

The present disclosure provides for methods, network devices, and computer readable storage media for packet reordering. In one embodiment, a method includes receiving a first packet of a first flow at a network device and determining whether flow-identifying information extracted from the first packet matches an existing flow entry. The method also includes, in response to a determination that the flow-identifying information does not match any existing flow entries, generating a new transient flow entry that includes the flow-identifying information and packet-in state. The method also includes forwarding the first packet to a controller via a packet-in stream.

Claims (98)

1. A method, the method comprising:

receiving a first packet of a first flow at a network device;

determining whether flow-identifying information extracted from the first packet matches any existing flow entries;

in response to a determination that the flow-identifying information does not match any existing flow entries, generating, by the network device, a new transient flow entry that includes the flow-identifying information and packet-in state, wherein the packet-in state indicates that packets of the first flow are presently being forwarded to a controller;

forwarding the first packet to the controller via a packet-in stream;

receiving flow policy for the first flow from the controller; and

in response to installation of the flow policy on the network device,

sending a marker packet to the controller via the packet-in stream, wherein the marker packet indicates an end to the packet-in stream, and

updating the new transient flow entry to include packet-out state, wherein the packet-out state indicates that packets of the first flow are no longer being forwarded to the controller.

2. The method of claim 1 , wherein

the existing flow entries comprises at least one of a group including:

an existing non-transient flow entry stored in a flow table, and

an existing transient flow entry stored in one of the flow table and a transient table,

wherein the existing non-transient flow entry includes flow policy that has been received from the controller and installed on the network device, and

wherein the existing transient flow entry includes transient status that indicates packets of a flow are presently being communicated between the network device and the controller.

3. The method of claim 1 , further comprising:

receiving a subsequent packet of the first flow at the network device from another network device before the flow policy has been installed on the network device; and

in response to a determination that second flow-identifying information extracted from the subsequent packet matches the new transient flow entry having packet-in state, forwarding the subsequent packet to the controller via the packet-in stream.

4. The method of claim 1 , further comprising:

generating, by the network device, the marker packet, wherein the marker packet includes the flow-identifying information extracted from the first packet.

5. The method of claim 1 , further comprising:

receiving a returning packet of the first flow from the controller via a packet-out stream, wherein the returning packet corresponds to a packet forwarded to the controller via the packet-in stream; and

in response to a determination that second flow-identifying information extracted from the returning packet matches the new transient flow entry having packet-out state, processing the returning packet according to the flow policy installed on the network device.

6. The method of claim 1 , further comprising:

receiving a subsequent packet of the first flow from another network device after the flow policy has been installed on the network device; and

in response to a determination that second flow-identifying information extracted from the subsequent packet matches the new transient flow entry having packet-out state, storing the subsequent packet in a FIFO (first in first out) buffer.

7. The method of claim 1 , further comprising:

receiving a returning marker packet for the first flow from the controller via a packet-out stream, wherein the returning marker packet corresponds to the marker packet sent to the controller via the packet-in stream, and the returning marker packet indicates an end to the packet-out stream; and

in response to a determination that second flow-identifying information extracted from the returning marker packet matches the new transient flow entry having packet-out state,

flushing a set of packets of the first flow from a FIFO (first in first out) buffer, wherein the set of packets were received at the network device after the marker packet was sent to the controller, and

processing the set of packets according to the flow policy installed on the network device.

8. The method of claim 7 , further comprising:

removing the new transient flow entry in response to receipt of the returning marker packet, wherein

the removing the new transient flow entry comprises one of

removing the new transient flow entry from a transient table, and

removing transient status from the new transient flow entry to result in a new non-transient flow entry stored in a flow table.

9. The method of claim 1 , further comprising:

receiving a subsequent packet of the first flow from another network device after the flow policy has been installed on the network device; and

in response to a determination that second flow-identifying information extracted from the subsequent packet does not match any transient flow entries, searching non-transient flow entries for the second flow-identifying information; and

in response to a determination that the second flow-identifying information matches a non-transient flow entry, processing the subsequent packet according to the flow policy indicated in the non-transient flow entry.

10. A network device comprising:

a port configured to receive a first packet of a first flow;

a first notifier configured to

detect a determination made by a flow match module whether flow-identifying information extracted from the first packet matches any existing flow entries;

a marker generator configured to

generate a new transient flow entry that includes the flow-identifying information and packet-in state, in response to a notification received from the first notifier that indicates the flow-identifying information does not match any existing flow entries, wherein

the packet-in state indicates that packets of the first flow are presently being forwarded to a controller; and

the flow match module is configured to forward the first packet to the controller via a packet-in stream; and

a second notifier configured to detect installation of a flow policy for the first flow received from the controller, wherein

the marker generator is further configured to

send a marker packet to the controller via the packet-in stream, in response to a second notification received from the second notifier that indicates the installation is detected, and

update the new transient flow entry to include packet-out state, in response to the second notification, wherein the packet-out state indicates that packets of the first flow are no longer being forwarded to the controller.

11. The network device of claim 10 , wherein

the existing flow entries comprises at least one of a group including:

an existing non-transient flow entry stored in a flow table, and

an existing transient flow entry stored in one of the flow table and a transient table,

wherein the existing non-transient flow entry includes flow policy that has been received from the controller and installed on the network device, and

wherein the existing transient flow entry includes transient status that indicates packets of a flow are presently being communicated between the network device and the controller.

12. The network device of claim 10 , wherein

the port is further configured to receive a subsequent packet of the first flow from another network device before the flow policy is installed on the network device,

wherein the flow match module is further configured to

forward the subsequent packet to the controller via the packet-in stream, in response to a determination that second flow-identifying information extracted from the subsequent packet matches the new transient flow entry having packet-in state.

13. The network device of claim 10 , wherein

the marker generator is further configured to generate the marker packet, wherein the marker packet includes the flow-identifying information extracted from the first packet.

14. The network device of claim 10 , further comprising:

a marker merger configured to

receive a returning packet of the first flow from the controller via a packet-out stream, wherein the returning packet corresponds to a packet forwarded to the controller via the packet-in stream, and

forward the returning packet to an action module for processing according to the flow policy installed on the network device, in response to a determination that second flow-identifying information extracted from the returning packet matches the new transient flow entry having packet-out state.

15. The network device of claim 10 , further comprising:

a marker merger configured to

receive a subsequent packet of the first flow from another network device after the flow policy is installed on the network device, and

store the subsequent packet in a FIFO (first in first out) buffer, in response to a determination that second flow-identifying information extracted from the subsequent packet matches the new transient flow entry having packet-out state.

16. The network device of claim 10 , further comprising:

a marker merger configured to

receive a returning marker packet for the first flow from the controller via a packet-out stream, wherein the returning marker packet corresponds to the marker packet sent to the controller via the packet-in stream, and the returning marker packet indicates an end to the packet-out stream, and

flush a set of packets of the first flow from a FIFO (first in first out) buffer to an action module for processing according to the flow policy installed on the network device, in response to a determination that second flow-identifying information extracted from the returning marker packet matches the new transient flow entry having packet-out state, wherein the set of packets were received at the network device after the marker packet was sent to the controller.

17. The network device of claim 16 , further comprising:

the marker merger further configured to remove the new transient flow entry in response to receipt of the returning marker packet, wherein

the marker merger is configured to perform one of

removal of the new transient flow entry from a transient table, and

removal of transient status from the new transient flow entry to result in a new non-transient flow entry stored in a flow table.

18. A non-transitory computer readable storage medium configured to store program instructions that, when executed on a processor, are configured to cause the processor to perform a method comprising:

receiving a first packet of a first flow at a network device;

determining whether flow-identifying information extracted from the first packet matches any existing flow entries;

in response to a determination that the flow-identifying information does not match any existing flow entries, generating a new transient flow entry that includes the flow-identifying information and packet-in state, wherein the packet-in state indicates that packets of the first flow are being forwarded to a controller; and

forwarding the first packet to the controller via a packet-in stream;

receiving flow policy for the first flow from the controller; and

in response to installation of the flow policy on the network device,

sending a marker packet to the controller via the packet-in stream, wherein the marker packet indicates an end to the packet-in stream, and

updating the new transient flow entry to include packet-out state, wherein the packet-out state indicates that packets of the first flow are no longer being forwarded to the controller.

19. The non-transitory computer readable storage medium of claim 18 , the method further comprising:

receiving a subsequent packet of the first flow from another network device after the flow policy has been installed on the network device; and

in response to a determination that second flow-identifying information extracted from the subsequent packet matches the new transient flow entry having packet-out state, storing the subsequent packet in a FIFO (first in first out) buffer.

20. The non-transitory computer readable storage medium of claim 18 , the method further comprising:

receiving a returning marker packet for the first flow from the controller via a packet-out stream, wherein the returning marker packet corresponds to the marker packet sent to the controller via the packet-in stream, and the returning marker packet indicates an end to the packet-out stream; and

in response to a determination that second flow-identifying information extracted from the returning marker packet matches the new transient flow entry having packet-out state,

flushing a set of packets of the first flow from a FIFO (first in first out) buffer, wherein the set of packets were subsequently received at the network device after the marker packet was sent to the controller, and

processing the set of packets according to the flow policy installed on the network device.

Assignments (16)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2021
From: NXP B.V.
To: FUTURE LINK SYSTEMS, LLC
Reel/Frame 055115/0491 →
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE APPLICATION 11759915 AND REPLACE IT WITH APPLICATION 11759935 PREVIOUSLY RECORDED ON REEL 040925 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE RELEASE OF SECURITY INTEREST. Recorded Feb 17, 2020
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP, B.V. F/K/A FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 052917/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE APPLICATION 11759915 AND REPLACE IT WITH APPLICATION 11759935 PREVIOUSLY RECORDED ON REEL 040928 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE RELEASE OF SECURITY INTEREST. Recorded Jan 17, 2020
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP B.V.
Reel/Frame 052915/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 10, 2019
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP B.V.
Reel/Frame 050744/0097 →
CORRECTIVE ASSIGNMENT TO CORRECT THE NATURE OF CONVEYANCE PREVIOUSLY RECORDED AT REEL: 040626 FRAME: 0683. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER AND CHANGE OF NAME EFFECTIVE NOVEMBER 7, 2016. Recorded Jan 12, 2017
From: NXP SEMICONDUCTORS USA, INC. (MERGED INTO); FREESCALE SEMICONDUCTOR, INC. (UNDER)
To: NXP USA, INC.
Reel/Frame 041414/0883 →
CHANGE OF NAME Recorded Nov 16, 2016
From: FREESCALE SEMICONDUCTOR INC.
To: NXP USA, INC.
Reel/Frame 040626/0683 →
RELEASE OF SECURITY INTEREST Recorded Nov 7, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP B.V.
Reel/Frame 040928/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 21, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP, B.V., F/K/A FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 040925/0001 →
SUPPLEMENT TO THE SECURITY AGREEMENT Recorded Jun 16, 2016
From: FREESCALE SEMICONDUCTOR, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 039138/0001 →
ASSIGNMENT AND ASSUMPTION OF SECURITY INTEREST IN PATENTS Recorded Jan 7, 2016
From: CITIBANK, N.A.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 037458/0460 →
ASSIGNMENT AND ASSUMPTION OF SECURITY INTEREST IN PATENTS Recorded Jan 7, 2016
From: CITIBANK, N.A.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 037458/0502 →
PATENT RELEASE Recorded Dec 21, 2015
From: CITIBANK, N.A., AS COLLATERAL AGENT
To: FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 037357/0921 →
SUPPLEMENT TO IP SECURITY AGREEMENT Recorded Nov 4, 2014
From: FREESCALE SEMICONDUCTOR, INC.
To: CITIBANK, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 034160/0351 →
SUPPLEMENT TO IP SECURITY AGREEMENT Recorded Nov 4, 2014
From: FREESCALE SEMICONDUCTOR, INC.
To: CITIBANK, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 034160/0370 →
SUPPLEMENT TO IP SECURITY AGREEMENT Recorded Nov 4, 2014
From: FREESCALE SEMICONDUCTOR, INC.
To: CITIBANK, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 034153/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2014
From: ANSARI, SHAD I.; ADDEPALLI, SRINIVASA R.
To: FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 033944/0079 →