OpenFlow v1.0

From OpenFlow Wiki

Jump to: navigation, search

Contents

Overview

A document describing the OpenFlow 1.0 release.

For details on the general release process, see the OpenFlow Releases page.

9/8/2009 Define a straw-man set of new features for the v1.0 release during the OpenFlow meeting. This list will be debated on openflow-discuss and openflow-spec for one week, at which point the barrier for new changes is much higher.

November 2009 Initial features implementation complete. All features must be submitted for public review at this time. Expect additional testing and debate.

Early December Release Candidates go out.

December 15, 2009 Final Release.

Current Status

Feature-related activities

Status of feature-related activities:

Feature Owner ETA
Ref
SW 		S
p
e
c 		R
e
f

S
W 		R
e
g
r
e
s
s 		W
i
r
e
s
h
a
r
k 		R
e
v
i
e
w 		M
e
r
g
e 		N
O
X 		ETA
NOX
New features/major changes
  3.1.1 Slicing Yiannis 11/15 Chk.png Chk.png Chk.png Chk.png Brandon Chk.png Rob
(FlowVisor) 		n/a
Enhancements
  3.2.1 Define Matching Capabilities for IP Fields in ARP Justin Dave Glen 10/31 Chk.png Chk.png Chk.png Chk.png Mikio Chk.png Chk.png Justin Chk.png
  3.2.2 Flow Cookies Justin 10/16 Chk.png Chk.png Chk.png Chk.png Dave Chk.png Chk.png KK Chk.png
  3.2.3 Specifying port number when requesting port stats Mikio 10/23 Chk.png Chk.png Chk.png Chk.png Bob Chk.png Chk.png KK Chk.png
  3.2.4 Adding a user specifiable datapath name Rob 10/16 Chk.png Chk.png Chk.png Chk.png KK Chk.png Chk.png KK Chk.png
  3.2.5 Support Matching on IP TOS bits Mikio 10/23 Chk.png Chk.png Chk.png Chk.png Glen Chk.png Chk.png KK Chk.png
  3.2.6 Improve resolution of flow duration in stats/expiry Dave 11/3 Chk.png Chk.png Chk.png Chk.png Glen Chk.png Chk.png KK Chk.png
3.3 Spec Clarifications (Major)
  3.3.1 Execution order of actions Brandon 11/10 Chk.png
  3.3.3 Port definition in Spec Brandon 11/10 Chk.png
  3.3.4 Simultaneous multiple controllers per switch Brandon 11/10 Chk.png
  3.3.7 Flow-mod to non-existing port Brandon Chk.png
  3.3.8 Overlap Questions Brandon Chk.png
  3.3.10 Proposed New Errors Justin Chk.png
3.4 Spec Clarifications (Minor)
  3.4.1 Clarify Counter Overflow Behavior Brandon 11/10 Chk.png
  3.4.2 Clarify Stats: note byte equals octet Brandon 11/10 Chk.png
  3.4.3 Clarify Required Action Combinations Brandon 11/10 Chk.png
  3.4.4 Add ICMP to Flow Matching diagram Brandon 11/10 Chk.png
  3.4.5 Reword Sentence on Async Messages Brandon 11/10 Chk.png
  3.4.6 Reword Sentence in Flow-removed section Brandon 11/10 Chk.png

Non-feature-related activities

Status of non-feature-related activities:

Task Owner Status Reviewer
Wire format compatible
  Reference switch Glen Chk.png
  NOX kk Chk.png
Test for Linux distros:
  Debian unstable Glen
  Ubuntu 9.10 -- 32-bit Glen
  Ubuntu 9.10 -- 64-bit Glen
  CentOS 5.4 Glen
  Fedora Core 12 Glen
Instructions on Wiki:
1-bit Glen
Final Packaging and Release:

0.9 features to be added to NOX

Status of 0.9 features to be added to NOX:

Task Owner ETA Status
Emergency flow cache Glen
Barrier command Glen 10/31 Chk.png
Matching VLAN priority Dan 10/31 Chk.png
Selected flow expiry KK Chk.png
Rewrite DiffServ/TOS Mikio 11/13 Chk.png
64 bit datapath ID Justin

Feature list

New Features / Major Changes

The list of proposed features describe changes that need some discussion about whether they belong in the 1.0 release.

Slicing

See the Slicing Wiki Page

Config Protocol

The Config Protocol has been moved to a separate future (tentatively 1.0.5) release.

Enhancements

This set of features are likely to require less change in the implementation.

Define Matching Capabilities for IP Fields in ARP

What: Allow flow matching rules to be defined over the IP fields in ARP packets.

Why: This allows the controller to specify more precise descriptions of flows. Further, it's required to properly implement L3-based in-band control.

How: For ARP packets that are for IP addresses, use the lower 8 bits of the ARP opcode field and place it into the OpenFlow "nw_proto" field. Place the ARP packet's ar_spa and ar_tpa fields and place them into the OpenFlow "nw_src" and "nw_dst" fields, respectively.

Flow Cookies

What: An opaque data field that is passed to a switch in a flow mod, and can be queried and returned with a flow expires

Why: It can be difficult/expensive to correlate flow mods and expiries across switches and correlating them as members of the same 'flow' without maintaining a lot of state in the controller. Having cookies will make this job easier, and allow other applications running in the controller to possibly take action based on what is in the cookie field.

How: Add an X length byte array to flow messages. This should not affect hardware implementations as the cookie can be simply stored in software, and in general should be an easy change.

Neighbor Discovery

Who(spec): KK Yap, Glen Gibb, Martin Casado, and David Underhill (currently absent)

What: Define a standard neighbor discovery protocol among OpenFlow devices and messages to inform controllers of changes.

Why: Link state changes (possibly loss of neighbors) can adversely affect routing tables and has to be discovered relatively quickly in the network for efficient routing. This has to be done on a switch, as the current controller-oriented LLDP approach is slow, and not scalable as the size of the network and/or number of controllers increases.

How: OpenFlow switches should supports neighbor discovery via Link Layer Discovery Protocol (LLDP) as a default mechanism for link discovery between OpenFlow switches. This includes the sending of LLDP packets/probes to discover new link and a disconnection. The probe should be of a reserved Ethernet type with the datapath id and switch port number in the payload. OpenFlow switches are expected to intercept these probes and trigger the appropriate link up and link down events. The intervals between probing for a new link or a disconnection should be independently configured. Alternative mechanisms to replace this default should ideally be user configurable, but is beyond the scope of the specification.

Specifying port number when requesting port stats

What: Provide port stats request with specified port number

Why: Since OpenFlow supports 2^16 ports, it would make sense to be able to specify a port number when requesting port statistics. This might be especially important if in the future the port space were extended to include virtual ports.

How:

Adding a user specifiable datapath description

What: Add the ability for a user to specify an easily identifiable datapath description that is paired with the dpid

Why: Developers working on controllers, applications, and network debugging frequently must identify switches which are uniquely identified by a 64 bit dpid. These are long human-unfriendly strings that are difficult to translate mentally into a tangible object, especially since the bits typically vary significantly between switches. Having a not-necessarily unique utf-8 string that can be specified by the switch configuration that is passed to the controller at the same time as the dpid would greatly ease this difficulty.

How: 

/* Body of reply to OFPST_DESC request.  Each entry is a NULL-terminated
 * ASCII string. */
struct ofp_desc_stats {
    char mfr_desc[DESC_STR_LEN];       /* Manufacturer description. */
    char hw_desc[DESC_STR_LEN];        /* Hardware description. */
    char sw_desc[DESC_STR_LEN];        /* Software description. */
    char serial_num[SERIAL_NUM_LEN];   /* Serial number. */
+   char dp_desc[DESC_STR_LEN];        /* Human readable description of datapath. */
};
OFP_ASSERT(sizeof(struct ofp_desc_stats) == 1056);

Support Matching on IP TOS bits

What: Turn the 11-tuple into a 12-tuple, and include IP ToS bits.

Why: If using OpenFlow in an L3 network, the VLAN bits cannot be used encode information, such as queue priority. OpenFlow already supports an action to set these bits.

How: Add another header field, with exact or wildcard matching, which applies when the ethertype corresponds to IP.

Improve resolution of flow duration in stats/expiry

What: Improve the resolution of flow stats/flow expiry messages. Extend to nanoseconds.

Why: Currently the flow duration field returned via stats messages and flow expiries has a granularity of seconds. This can be problematic if you are polling flows at a pretty fast pace, say every ~5-10s, attempting to determine a throughput rate, as your error can easily be >10%.

How: Add a nanoseconds field in addition to the seconds field for duration. (Done to mirror the timespec structure in Linux.)

Spec Clarifications (Major)

These are discussion-worthy clarifications.

Execution order of actions

What: Clarify execution order of actions.

Why: Action ordering is currently undefined in the OpenFlow spec. In short, two OpenFlow hardware switches may have inconsistent behavior yet be fully complaint. Simple examples include:

  • Example 1: Action list with two actions: (1) Send to port A; (2) Send to port A. Will 2 packets appear on port A? Note that this uses only required actions.
  • Example 2: Action list with one action: (1) Change VID. Will the packet be dropped? (The spec says an empty action list means to drop the packet.)
  • Example 3: The above two examples may be seen as bad action specifications. But consider an action list with four actions: (1) Change VID to X; (2) Send port A; (3) Change VID to Y; (4) Send to port B.

A sequential execution of Example 3 list would result in a packet with VID X on port A and a packet with VID Y on port B. The spec indicates that the order of actions cannot be specified, so there are a good number of possible results that would satisfy the list of actions (including packets on ports A and B with VID Z if Z were the original VID on the packet).

How:

  • Option 1: Declare that lists of actions are not performed sequentially.
    • Advantage: Simple
    • Disadvantages: (1) Lots of ambiguous cases now that may lead to different behavior in different implementations. (2) If platforms support sequential actions, they can't take advantage of it.
  • Option 2: Declare that lists of actions must be performed sequentially.
    • Advantage: Simple, ensures consistent behavior
    • Disadvantage: Forces flow mods with non-HW-supported action combinations to move to software, which may force all flow mods to move to software.
  • Option 3: Add a capability bit per table to indicate if the table is capable of sequential actions.
    • Advantage: Flexible.
    • Disadvantage: Not simple. Are there variations in what may be executed sequentially on different platforms? Requires change to implementation.

Discussion:

There are two philosophies coming into play here:

  • A network processor (NP) engine where the action list is essentially a program being executed by the processor;
  • A hardware pipeline (PIPE) in a switch ASIC where a packet may receive minor modifications before it is pushed out a port based on relatively simple switching logic.

The OpenFlow spec was likely conceived thinking of an NP, since the software implementation can do exactly this. In this case, having an ordering of actions makes sense, and arbitrarily complicated sequences of packet modifications, interspersed with forwarding of the intermediate results, may be achieved.

The implementation on current switch ASICs is often much more limited. A sequence of packet modifications may be carried out, followed by a specification of the destination port. Some hardware may support arbitrary multicast of any packet, but some may not.

  • Question 1: Is it a requirement that the OpenFlow spec support arbitrary ordered action sequences?
  • Question 2: Should the spec be modified so that the following action list guarantees the packet gets sent to port A with VID X: (1) Modify VLAN to X; (2) Send to port A? How can we ensure this and still provide a reasonable spec for existing switch ASICs?

What if a table could signal "send to port is a final action" meaning that other actions would be carried out non-sequentially prior to the output action?

Resolution

Modified spec Actions section: " Each flow entry is associated with zero or more actions that dictate how the switch handles matching packets. If no actions are present, the packet is dropped. Action lists for \emph{inserted} flow entries MUST be processed in the order specified. However, there is no packet output ordering guaranteed within a port. For example, an action list may result in two packets sent to two different VLANs on a single port. These two packets may be arbitrarily re-ordered, but the packet bodies must match those generated from a sequential execution of the actions.

A switch may reject a flow entry if it cannot process the action list in the order specified, in which case it should immediately return an unsupported flow error (see \ref{unsupported_flow}). Ordering within a port may vary between vendor switch implementations. ... "

The referenced-later text in the Flow Table Modification Messages section: " \label{unsupported_flow}If a switch cannot process the action list for any flow mod message in the order specified, it MUST immediately return an \verb|OFPET_FLOW_MOD_FAILED| : \verb|OFPFMFC_UNSUPPORTED| error and reject the flow. "

Pushed commit with text and openflow.h changes to devel/brandonh/spec/execution_order

Added Non-deterministic Behavior wiki section for stuff like this, noting single-port ordering first.

Port definition in Spec

What: The port definition of OpenFlow should be clarified because the spec does not allow to carry both production and OpenFlow traffic over trunk/dot1q-turnking like below. 

App#1 trunking - Which is the OpenFlow port? eth0 or eth0.200?

                        OpenFlow enabled L2/L2.5 switch
                     +---------------------------------------
                     |  VID:100-(eth0.100)- production switching domain
                     | /
--(trunk)---------(eth0)
                     | \
                     |  VID:200-(eth0.200)- OpenFlow switching domain
                     +----------------------------------------

App#2 Q-in-Q or 802.1ad - Which is the OpenFlow port? eth0 or eth0.10 or eth0.200?

                        OpenFlow enabled L2/L2.5 switch
                     +---------------------------------------
                     |            VID:100-(eth0.100)- production switching domain
                     |            /
--(dot1q-tunnel)--(eth0)-VID:10-(eth0.10)
                     |            \
                     |            VID:200-(eth0.200)- OpenFlow switching domain
                     +----------------------------------------

App#3 link aggr (802.3ad) - Which is the OpenFlow port? both of ethX or lag0?

                        OpenFlow enabled L2/L2.5 switch
                     +---------------------------------------
--(link#1)--------(eth0)
                     | \
                     |  +-(lag0)- OpenFlow switching domain
                     | /
--(link#2)--------(eth1)
                     +----------------------------------------

Why: The spec describes that the OpenFlow port is physical switch port.

How: First of all, the term "virtual port" is used in the spec to refer to OpenFlow specific port references: "All", "Controller", "Local", "Table" and "In-Port". This terminology should be changed because "virtual port" is used in too many other contexts already. I suggest virtual changes to OpenFlow special in these instances.

Second, the spec refers to physical port whenever it is talking about non-special ports. I suggest this be changed in all cases to port.

Third, in Section 2: Switch Components, add a short paragraph explaining about ports. Something like: 

An OpenFlow port is a device on which packets can be sent or received.  In
general, this is a physical port, but the protocol does not preclude abstractions
like port aggregations or VLAN traffic on a port appearing as an OpenFlow port.
OpenFlow ports have limited state such as "up", "down" and whether flooded packets
should be forwarded.  Additional configuration of ports is handled by the 
configuration protocol.  There are several "OpenFlow special" ports used by
OpenFlow to indicate, for example, flooding or the ingress port.

In 5.2.1, there is some discussion of STP. This probably deserves a different RFC, but if STP is to remain in the spec, we shold mention it in the sentence about port-state above.

Discussion: There are several issues here. One is the fact that vendors currently want to differentiate between OpenFlow and non-OpenFlow (production above) traffic. This is a sort of virtualization that really needs to be done outside (or prior) to OpenFlow. We need to decide whether this needs to be addressed in either the OpenFlow spec or in the OpenFlow configuration spec (where it might be reasonable to assume such configuration would be controlled).

A second issue is about how OpenFlow should view ports. Consider the first diagram, App#1, above (but ignore production and assume OpenFlow controls all the traffic on the port). It would be best if both models could be supported: (1) OpenFlow sees one port and sees packets with different VLAN tags on that port; (2) OpenFlow sees two ports, possibly only untagged packets on each port. You can see how different rules would be used in these two models to get the various sorts of forwarding you might want, but either would work. I don't think the spec should decide how that should be implemented.

Pushed commit 4c378f6cf2 to branch devel/brandonh/spec/next

Simultaneous multiple controllers per switch

What: Clarify behavior in case of multiple controllers per switch.

Why: Undefined behavior.

How:

Make multiple controller support explicitly undefined.

Pushed commit 527fc62 to devel/brandonh/spec/next.

Flow-mod to non-existing port

What: What's the expected behavior when you add a flow which forwards packets to a non-existing port?

Why:

The OpenFlow reference implementation accepts the flow and inserts it at the table, and then silently drops the packets.

Open vSwitch accepts OpenFlow port numbers that Open vSwitch can possibly use, rejecting only those that are outside the range that it ever uses (currently 1024 ports for the Linux kernel datapath, but adjustable).

However, the spec does not define the expected behavior.

How: One option is to throw an error, such as OFPET_FLOW_MOD_FAILED, or OFPET_BAD_ACTION:OFPBAC_BAD_OUT_PORT.

Another is to accept this behavior as typical and fail silently. In the case of replacing or upgrading a fabric or port module on a chassis switch, you might want existing entries to remain.

For example, packets to module A go to module B and C. If module C's ports go away, it seems reasonable not to touch the flow table and disturb A to B connectivity. The controller can always respond to the port status change message. A linecard-upgrade-aware controller might leave the entries alone, while a naive controller might either flush the switch entries or just those forwarding to failed ports.

Resolution:

Added this text to the "Flow Table Modification Messages" subsection:

"If the action list in a flow mod message references a port that will never be valid on a switch, the switch must return an \verb|ofp_error_msg| with \verb|OFPET_BAD_ACTION| type and \verb|OFPBAC_BAD_OUT_PORT| code. If the referenced port may be valid in the future, e.g. when a linecard is added to a chassis switch, or a port is dynamically added to a software switch, the switch may either silently drop packets sent to the referenced port, or immediately return an \verb|OFPBAC_BAD_OUT_PORT| error and refuse the flow mod."

Pushed bc8ba1ab to devel/brandonh/spec/next

Overlap Questions

What: Ambiguities in the definition of the OFPFF_CHECK_OVERLAP flag, pointed out by Justin Pettit:

  • What is the correct behavior if the overlap flag is set and the flow

is identical? Possible error: OFPFMFC_OVERLAP

  • Why are exact match entries explicitly excluded from checking for

overlap? I'd think for predictability that they would both behave in the same manner.

Pushed fix to devel/brandonh/spec/next

Proposed New Errors

See openflow-spec mailing list thread

Spec Clarifications (Minor)

Clarify Counter Overflow Behavior

What: What do counters do upon overflow?

Why: Undefined behavior.

How: Clarified that counters wrap around on overflow, with no explicit indicator. 64-bit counters should not overflow too often, even with 10 Gb links.

Pushed commit fd3a78a to devel/brandonh/spec/next.

Clarify stats: note byte equals octet

What: The meaning of the word byte is ambiguous to old-school telecom types.

Why: From Wikipedia byte definition:

"The size of a byte is typically hardware dependent, but the modern de facto standard standard is 8 bits, as this is a convenient power of 2. No formal definition exists however, and other sizes have been used in various computers historically.

The term octet is widely used as a precise synonym of the 8-bit byte where ambiguity is undesirable, such as in communications protocol definitions."

How: Keep using the word byte, but add a brief note to clarify in the spec:

"In this document, the phrase byte refers to 8-bit octets."

Pushed commit aa0fa8a to branch devel/brandonh/spec/next

Clarify Required Action Combinations

What: Ambiguity about required action combinations.

Why:

Bottom of spec p5:

"Therefore, the requirement is that the switch support sending to any combination of physical ports and the Controller virtual port simultaneously. "

Then, just below:

"The controller will only ask the switch to send to multiple physical ports simulta- neously if the switch indicates it supports this behavior in the initial handshake (see section 5.3.1)."

Practically, are there platforms incapable of sending to multiple physical ports? What does simultaneously mean here - would packets sent serially by a software path out physical ports be considered simultaneous?

MUST and SHOULD could be used to clarify the language here.

How:

This seems like another example where universal support requires an assumption of a software datapath (just like action ordering restrictions), and where hardware pipeline support will have a drastic effect on an action's forwarding rate. Maybe this assumption should be made explicit, and can help unify/explain some interface choices.

  • Option 1: Remove the top text.
  • Option 2: For simplicity and determinism, remove this capability bit and require support for arbitrary port combinations.
  • Option 3: Clarify that such support is optional. Controllers must beware the capability bit and not send unsupported port combinations, which may result in an error.

Pushed fix to branch devel/brandonh/spec/clarify_action_combinations

Add ICMP to flow matching diagram

page8 of spec: The last two bullets at top of page ICMP and fragments need to be added to the p. 7 flowchart with another couple of boxes for those two cases.

Fixed, and all flowcharts converted to vector graphics format; should print much nicer.

pushed commit b1f271c5 to devel/brandonh/spec/next

Reword Sentence on Async Messages

page9, sec 4.1.2: "sent without solicitation from the switch"

Awkward ambiguous english - needs rewording to mean "without solicitation from the controller"

Wording is now:

"Asynchronous messages are sent without the controller soliciting them from a switch. Switches send asynchronous messages to the controller to denote a packet arrival, switch state change, or error."

Pushed commit 366d6da6 to devel/brandonh/spec/next.

Reword Sentence in Flow-removed section

page10, sec 4.1.2: flow-removed section

Refers to "Delete message" without defining it.

Changed:

"Delete messages may also cause flow removed messages."

to:

"Flow modify messages which delete flows may also cause flow removed messages."

Pushed commit 90f66ac to devel/brandonh/spec/next.

Testing Plan and Schedule

Moved to OpenFlow v1.0 testing page

Contributor acknowledgement

It has been suggested that we have a contributors section in the spec to acknowledge contributions from industry.

Retrieved from "http://www.openflow.org/wk/index.php/OpenFlow_v1.0"