GEC8Tutorial

From OpenFlow Wiki

Jump to: navigation, search

NOTE: THIS TUTORIAL IS OBSOLETE. Please use the newer one at HOTITutorial2010

Welcome to the OpenFlow tutorial!

OpenFlow is an open interface for remotely controlling the forwarding tables in network switches, routers, and access points. Upon this low-level primitive, researchers can build networks with new high-level properties. For example, OpenFlow enables more secure default-off networks, wireless networks with smooth handoffs, scalable data center networks, host mobility, more energy-efficient networks and new wide-area networks – to name a few.

This tutorial is your opportunity to gain hands-on experience with the platforms and debugging tools most useful for developing network control applications on OpenFlow.

Innovate in your network!

After completing this tutorial, please fill out the feedback form.


Contents


Overview

Tutorial flow.png

Following an introduction, you'll turn the provided hub controller into a controller-based learning switch, then a flow-accelerated learning switch, and finally a firewall or router - you get to choose. Along the way, you'll learn the full suite of OpenFlow debugging tools. You will:

  • view flow tables with dpctl
  • dissect OpenFlow message with Wireshark
  • simulate a multi-switch, multi-host network with Mininet
  • benchmark your controller with cbench

After the tutorial, you can apply what you've learned to physical networks based on software switches, NetFPGAs, or even hardware switches at line rate.

To get you started quickly, we provide a preconfigured virtual machine with the needed software.

Pre-requisites

You will need a computer with at least 1GB (preferably 2GB+) of RAM and at least 5GB of free hard disk space (more preferred). A faster processor may speed up the virtual machine boot time, and a larger screen may help to manage multiple terminal windows.

These instructions consider Linux, OS X, and Windows. Linux and OS X are preferred - there's less to install.

You will need administrative access to the machine.

The tutorial instructions require no prior knowledge of OpenFlow. The OpenFlow Learn More page is a concise introduction, and worth reading if you're not running this at a tutorial session.

Stuck? Found a bug? Questions?

Email [brandonh - at - stanford.edu] if you're stuck, think you've found a bug, or just want to send some feedback.

If reporting a bug, try to include as many details about your setup as possible. Include your OS, virtualization software info, memory size, and the step you're on.

Install Required Software

At the Tutorial Only: Flash drives and DVDs are passed at the tutorial, and should include all of the software you will need. Please save the available wireless bandwidth for other traffic at the tutorial! If preparing for a tutorial, follow the instructions for web users below to pre-install the software.

Web Users + Tutorial Preparers: You will need to download these files individually.

The files include virtualization software, a SSH-capable terminal, an X server, and the VM image.

The tutorial image is distributed as a compressed VMware virtual disk image (.vmdk). Both VMware and VirtualBox enable you to run a virtual machine inside a physical machine, but they differ by the interface, features and price. They're both good choices, but we recommend VMware as it saves a few steps by setting up a transparent bridged network on the host. VirtualBox requires a few commands to set up networking, but is free for all platforms. VMware Player is free for Windows and Linux, but the Mac version (Fusion) and the more full-featured Windows and Linx version (Workstation) are not free. That's OK, you can use a free 30-day trial for the tutorial. For Mac, there are other choices, like Parallels Workstation. For Linux, there are plenty of other choices, including Qemu, KVM, and Xen.

The following instructions assume the use of VMware virtualization; if you insist on VirtualBox, please see the VirtualBox-specific instructions at the bottom of the page.

[At Tutorial Only] Copy Files From Flash Drive

All the files you might need should be on these flash drives and DVDs.

The first step is to create a folder on your computer, say 'oft', and start copying the relevant portions of the provided flash drive to this directory. This folder includes OpenFlowTutorial-052310.zip (the compressed virtual machine image), and a directories for each OS with installer files for required programs. Only copy the files needed for your platform that you haven't already installed. This may take a few minutes.

When you've copied the files, please pass a drive to your neighbor, and install the programs specific to your OS.

[Web Users + Tutorial Preparers] Download Files

You'll need to download the files corresponding to your OS, plus the tutorial VM.

Start now with downloading the OpenFlowTutorial.zip compressed VM image. This will take awhile - it's on the order of 1GB in size.

Then move on to the other stuff:

OS Type OS Version Virtualization Software X Server Terminal
Windows 7 VMware Player or VMware workstation Xming PuTTY
Windows XP VMware Player or VMware workstation Xming PuTTY
Mac OS X 10.6 Snow Leopard VMware Fusion X11 (install from OS X main system DVD, preferred), or download XQuartz Terminal.app (built in)
Mac OS X 10.5 Leopard VMware Fusion X11 (install from OS X main system DVD, preferred), or download XQuartz Terminal.app (built in)
Mac OS X 10.4 Tiger VMware Fusion X11 (install from OS X main system DVD, preferred), or download XQuartz Terminal.app (built in)
Mac OS X 10.3 Panther VMware Fusion download Mac X11 server or download XQuartz Terminal.app (built in)
Linux Ubuntu 10.04 VMware player or VMware workstation X server already installed gnome terminal + SSH built in

You'll need to have each column item (X server, Virtualization software, and SSH terminal) installed for your platform, plus the VM image (OpenFlowTutorial-MMDDYY.vmwarevm.zip).

Setup Virtual Machine

Get Virtual Machine Disk Image

GEC8 Tutorial Attendees: Copy the file OpenFlowTutorial-052310.vmwarevm.zip from the DVD or flash drive.

Web users ONLY: download the VM from here (~1.3GB). Remember where you save this file, as you'll need it shortly. Warning This a large file, and the download will take awhile.

Unpack Virtual Machine Disk Image

The virtual machine image ships compressed. We'll need to unzip it first.

Warning This step will take awhile - the unpacked .vmdk file is around 2 GB.

OS X and Linux Users: from a terminal, unzip the virtual machine image: 

unzip OpenFlowTutorial.zip

Windows Users: unzip the image in Windows Explorer.

Start Virtual Machine

Double-click on the vmx file (not the vmdk), and it should open in VMware. Click the play button to start it. Do not change the kernel choice during boot.

DO NOT INSTALL VMware Tools NOW.

If VMware asks if you'd like to install the VMware tools, decline, for now. VMware tools speed up the networking connection between the guest and host, enable copy/paste from the VM console, speed up graphics, and do a few other things. We don't need any of these, so decline. Feel free to install the VMware tools after the tutorial.

At the boot prompt, enter the username and password, which are both mininet. Note that the mininet user is a sudoer.

Choose Preferred Editor

Nano, Vim, Emacs, and Gedit come installed on the OpenFlowTutorial VM. Brief instructions for each:

Nano: You can immediately modify a file. When you're done, hit 'ctrl-x', then say 'Yes' to the prompt, to save and quit.

Vim: to modify a file, type 'i' to enter Insert mode, then use the arrow keys to navigate and edit. When you're done, hit 'esc', type ':wq', then press enter, to save and quit.

Emacs: you can immediately modify a file. When you're done, hit 'ctrl-x', 'ctrl-s', then hit 'ctrl-x', 'ctrl-c' to exit.

Gedit: a graphical text editor, no instructions needed.

Eclipse: Eclipse and its dependencies would require about 500MB extra space on the VM image, so it's not shipped by default. If you have Eclipse installed on the host VM, using the Remote Systems Explorer can be a convenient way to access and modify text files on the VM, with many of the advantages of Eclipse, such as syntax highlighting.

[Web Users Only] If you have another preferred text editor, feel free to install it now: 

sudo apt-get install <editor>

Setup Network Access

The tutorial VM is shipped without a desktop environment, to reduce its size. All the exercises will be done through X forwarding, where programs display graphics through an X server running on the host OS.

To start up the X forwarding, you'll first need to find the guest IP address.

VMware

VMware sets up a bridge between the host and the guest by default, with external access through the host via NAT. No additional config should be necessary to contact the VM from the host, as well as access Internet resources from within the guest.

Other

For VirtualBox, see the instructions at the bottom. For anything else, you'll need to consult its documentation.

Access VM via SSH

In this step, you'll verify that you can connect from the host PC (your laptop) to the guest VM (OpenFlowTutorial) via SSH.

First, from the virtual machine console - the one that appeared when you started up VMware - login as user mininet with password mininet, then enter: 

ifconfig eth0

Note this IP; you'll need it later. Next, log in, which will depend on your OS.

Mac OS X and Linux

Open a terminal (Terminal.app in Mac, Gnome terminal in Ubuntu, etc). In that terminal, run: 

ssh -Y mininet@[Guest IP Here]

Replace [Guest IP Here] with the IP you just noted.

Login with password mininet. Test the X11 forwarding by running on the local machine: 

xterm

and a new terminal window should appear. If you have succeeded, you are done with the basic setup. Close the xterm. If you get a 'Display not set error', verify your X server installation from above.

Windows XP

Start Xming with X Forwarding enabled. To enable X-Forwarding, click puTTY->Connection->SSH->X11, then click on Forwarding->"Enable X11 Forwarding", as shown below:


Putty x11 fwding.PNG


Nothing exciting will happen after clicking OK, but you should see an icon on the right of the toolbar.

Open a terminal: click the Windows 'Start' button, 'run', then enter 'cmd'.

Change to the directory where you saved putty. 

cd <dir>

Run: 

putty.exe -X mininet@[Guest IP Here]

Replace [Guest IP Here] with the IP you just noted.

A new window will pop up; type in the password 'mininet'. Now, type: 

xterm

A white terminal window should appear. If you have succeeded, you are done with the basic setup. Close the xterm.

Learn Development Tools

In this section, you'll bring up the development environment. In the process, you'll be introduced to tools that will later prove useful for turning the provided hub into a learning switch. You'll cover both general and OpenFlow-specific debugging tools.

Let's define some terminology, starting with terminal types:

  • VMware/VirtualBox console terminal: connects to OpenFlowTutorial. This is the one created when you started up the VM. You can't copy and paste from this tutorial page to the console terminal, so it's a bit of a pain. Minimize this NOW, if you haven't already done so. Once you've used it to set up networking, it won't be needed.
  • SSH terminal: connects to OpenFlowTutorial. Created by using putty on Windows or SSH on OS X / Linux, as described in the previous section. Copy and paste should work on this terminal.
  • xterm terminal: connects to a host in the virtual network. Created in the next section when you start up the virtual network. Will be labeled at the top with the name of the host.

The OpenFlowTutorial VM includes a number of OpenFlow-specific and general networking utilities pre-installed. Please read the short descriptions:

  • OpenFlow Controller: sits above the OpenFlow interface. The OpenFlow reference distribution includes a controller that acts as an Ethernet learning switch in combination with an OpenFlow switch. You'll run it and look at messages being sent. Then, in the next section, you'll write our own controller on top of NOX (a platform for writing controllers).
  • OpenFlow Switch: sits below the OpenFlow interface. The OpenFlow reference distribution includes both a user-space and a kernel-based software switch. One other software switch is supported (Open vSwitch) and a number of hardware switches are available from Broadcom (Stanford Indigo release), HP, NEC, and others.
  • dpctl: command-line utility that sends quick OpenFlow messages, useful for viewing switch port and flow stats, plus manually inserting flow entries.
  • Wireshark: general (non-OF-specific) graphical utility for viewing packets. The OpenFlow reference distribution includes a Wireshark dissector, which parses OpenFlow messages sent to the OpenFlow default port (6633) in a conveniently readable way.
  • iperf: general command-line utility for testing the speed of a single TCP connection.
  • Mininet: command-line utility for emulating an OpenFlow network on a single machine. Mininet enables virtual hosts with their own IP addresses to communicate through virtual Ethernet links with OpenFlow software switches, by using the network namespace virtualization built into recent versions of Linux. More Mininet details can be found at the Mininet web page.
  • cbench: utility for testing the flow setup rate of OpenFlow controllers.

From here on out, make sure to copy and paste as much as possible! For example, manually typing in ‘sudo dpctl show n1:0’ may look correct, but will cause a confusing error; the 'nl' is short for NetLink, not n-one.

Let's get started...

Start Network

The network you'll use for the first exercise includes 3 hosts and a switch:


Three switch layout.png


To create this network in the VM, in an SSH terminal, enter: 

sudo mn --topo single,3 --mac

This tells Mininet to start up a 3-host, single-switch topology and set the MAC address of each host equal to its IP.

Here's what Mininet just did:

  • Created 3 virtual hosts, each with a separate IP address.
  • Created a single OpenFlow software switch in the kernel with 3 ports.
  • Connected each virtual host to the kernel switch with a virtual ethernet cable.
  • Started up the OpenFlow reference controller.
  • Set the MAC address of each host equal to its IP.

Mininet Brief Intro

Since you'll be working in Mininet for the whole tutorial, it's worth learning a few Mininet-specific commands:

To see the list of nodes available, in the Mininet console, run: 

nodes

To see a list of available commands, in the Mininet console, run: 

help

To run a single command on a node, prepend the command with the name of the node. For example, to check the IP of a virtual host, in the Mininet console, run: 

h2 ifconfig

The alternative - better for running interactive commands and watching debug output - is to spawn an xterm for one or more virtual hosts. In the Mininet console, run: 

xterm h2 h3

You can close these windows now, as we'll run through most commands in the Mininet console.

Mininet has loads of other commands and startup options to help with debugging, and this brief starter should be sufficient for the tutorial. If you're curious about other options, follow the Mininet Walkthrough after the main tutorial.

Ping Test

Your network should now be up. Run a ping to verify host connectivity through the OpenFlow switch. In the Mininet console: 

h2 ping -c3 h3

Note that the name of host h3 is automatically replaced when running commands in the Mininet console with its IP address (10.0.0.3).

The ping should complete. Note that the first pings take a bit longer, due to OpenFlow messages setting up switch state.

View Startup Messages in Wireshark

The VM image includes the OpenFlow Wireshark dissector pre-installed. Wireshark is extremely useful for watching OpenFlow protocol messages, as well as general debugging.

Start a new SSH terminal and connect to the VM w/X forwarding:

If you're using MAC OS X or Linux, enter: 

ssh -X mininet@[guest ip address]

If you're using putty.exe, enter: 

putty.exe -X mininet@[guest ip address]


Now open Wireshark: 

sudo wireshark &

Click on Capture->Interfaces in the menu bar. Click on the Start button next to 'lo', the loopback interface. You may see some packets going by.

Now, set up a filter for OpenFlow control traffic, by typing 'of' in Filter box near the top: 

of

Press the apply button to apply the filter to all recorded traffic. You should only see Echo Request and Echo Reply messages; these messages allow the controller to verify that the switch is alive, and vice versa.

To view the OpenFlow startup messages, we're going to restart the network.

First, kill your Mininet network. In the Mininet console, enter: 

exit

Alternately, this can be done by typing: 

sudo mn -c

If for whatever reason you're experiencing errors with Mininet, you should run the command 'sudo mn -c' to wipe out its state.

Now, with the Wireshark dissector listening, we'll start the network up again. This time, look for the OpenFlow control messages between the switch and the controller. Clear out the previous messages, by going to Capture->Restart.

Also, we're not so interested in seeing Echo Request (type 2) and Echo Reply (type 3) messages. To remove these from display, replace the previous filter, 'of', with the following one: 

of && !(of.type == 2 || of.type == 3)

Then restart the network. In the SSH terminal being used for Mininet: 

sudo mn --topo single,3 --mac

You should see a bunch of messages displayed, from the Hello exchange onwards. As an example, click on the Features Reply message. Click on the triangle by the 'OpenFlow Protocol' line in the center section to expand the message fields. Click the triangle by Switch Features to display datapath capabilities - feel free to explore.

These messages include:

Message Type Description
Hello Controller->Switch following the TCP handshake, the controller sends its version number to the switch.
Hello Switch->Controller the switch replies with its supported version number.
Features Request Controller->Switch the controller asks to see which ports are available.
Set Config Controller->Switch in this case, the controller asks the switch to send flow expirations.
Features Reply Switch->Controller the switch replies with a list of ports, port speeds, and supported tables and actions.
Port Status Switch->Controller enables the switch to inform that controller of changes to port speeds or connectivity. Ignore this one, it appears to be a bug.


Since all messages are sent over localhost when using Mininet, determining the sender of a message can get confusing when there are lots of emulated switches. However, this won't be an issue, since we only have one switch. The controller is at the standard OpenFlow port (6633), while the switch is at some other user-level port.

View OpenFlow Messages for Ping

Now, we'll view messages generated in response to packets.

Run a ping to view the OpenFlow messages being used. In the Mininet console: 

h2 ping -c1 h3

In the Wireshark window, you should see a number of new message types:

Message Type Description
Packet-In Switch->Controller a packet was received and it didn't match any entry in the switch's flow table, causing the packet to be sent to the controller.
Packet-Out Controller->Switch controller send a packet out one or more switch ports.
Flow-Mod Controller->Switch instructs a switch to add a particular flow to its flow table.
Flow-Expired Switch->Controller a flow timed out after a period of inactivity.


First, you see an ARP request miss the flow table, which generates a broadcast Packet-Out message. Next, the ARP response comes back; with both MAC addresses now known to the controller, it can push down a flow to the switch with a Flow-Mod message. The switch does then pushes flows for the ICMP packets. Subsequent ping requests go straight through the datapath, and should incur no extra messages; with the flows connecting h2 and h3 already pushed to the switch, there was no controller involvement.

Re-run the ping, again from the Mininet console (hitting up is sufficient - the Mininet console has a history buffer): 

h2 ping -c1 h3

If the ping takes the same amount of time, run the ping once more; the flow entries may have timed out while reading the above text.

This is an example of using OpenFlow in a reactive mode, when flows are pushed down in response to individual packets.

Alternately, flows can be pushed down before packets, in a proactive mode, to avoid the round-trip times and flow insertion delays.

dpctl Example Usage

dpctl is a utility that comes with the OpenFlow reference distribution and enables visibility and control over a single switch's flow table. It is especially useful for debugging, by viewing flow state and flow counters. Most OpenFlow switches can start up with a passive listening port, from which you can poll the switch, without having to add debugging code to the controller.

Create a second SSH window if you don't already have one, and run: 

sudo dpctl show nl:0

You won't see the messages used by dpctl in Wireshark, since these are sent directly to the kernel switch over a netlink socket. If we were using the user-space switch or a physical switch, you'd see these. The 'show' command connects to the switch and dumps out its port state and capabilities.

Here's a more useful command: 

sudo dpctl dump-flows nl:0

We'll assume a minute has passed since the last ping you did, and that the flows were evicted. In that case, the flow table is empty.

Now go back to the Mininet console and send 10 pings, once every 200ms: 

h2 ping -c10 -i0.2 h3

Back in the other SSH terminal, dump the flow state again: 

sudo dpctl dump-flows nl:0

Check the flow output; do you understand why each flow matched the number of packets shown? Note that flows time out by default when using the reference controller - so the stats shown by dpctl may change over time.

Regression Tests

The OpenFlow reference distribution includes a set of tests to verify that an OpenFlow switch correctly sends and receives packets. In this section, you'll run through some of the "Black Box" tests, where the regression switch acts as a controller and verifies that a switch responds properly to OpenFlow messages, as well as properly forwards packets sent to the input ports of the switch. The Black Box regression tests use this layout:


Regression tests.png


First, exit from the Mininet console: 

exit

To run the tests, log in as root, from the same SSH terminal: 

sudo su

Run a script to start up virtual ethernet pairs locally. 

openflow/regress/bin/veth_setup.pl

Veth0 connects to veth1, for example, and anytime a packet is sent to veth0, it'll pop out veth1, and vice versa. The regression suite will allocate four veth pair halves to a software OpenFlow switch and the other four to the test suite, which sends packets and checks the contents and locations of responses.

In an SSH terminal: 

ifconfig -a

You should see 8 new virtual interfaces, veth0..veth7.

Configure Wireshark to show OpenFlow messages and TCP resets in red (which delimit individual tests). In the Wireshark filter box, enter: 

of || tcp.flags.reset == 1

Start running the tests, with Wireshark running on lo (loopback interface): 

openflow/regress/bin/of_kmod_veth_test.pl

You may see OpenFlow messages for each test - if not, hit Ctrl-C to stop the tests, and clean up any leftover state: 

openflow/regress/bin/of_kmod_veth_teardown.pl

Sometimes on the first run, the tests fail; we're not sure why this is happening. Restart after tearing down and they should work.

Then, re-run the tests: 

openflow/regress/bin/of_kmod_veth_test.pl

When you get bored, stop the tests and check the messages. Then clean up any state: 

openflow/regress/bin/of_kmod_veth_teardown.pl

The regression suite is useful for verifying new switch features, as well as diagnosis divergent behavior between different switches. The suite is generally used with 4 physical ports on hardware switches. We won't do anything more with it in this tutorial, but feel free to peruse the code. See the openflow/regress/projects/black_box directory, which has a file for each test.

To teardown virtual ethernet pairs, run: 

openflow/regress/bin/veth_teardown.pl

A replacement test suite is in the making, and is expected for mid 2010. The new suite will be based on Python and reorganized to make switch-specific tests and port configurations much easier to define and debug.

Exit back to regular username: 

exit

Run NOX

For the first exercise, the controller you program will be built on top of NOX and written in the Python language. From noxrepo.org:

NOX is an open-source platform that simplifies the creation of software for controlling or monitoring networks. Programs written within NOX (using either C++ or Python) have flow-level control of the network. This means that they can determine which flows are allowed on the network and the path they take.

For more details, see the NOX overview.

You do no need to install NOX - it comes ready to run on the VM.

We're not going to be using the reference controller anymore, which is running in the background (do 'ps -A | grep controller' if you're unsure). To re-start Mininet and tell it to point to a different controller: 

sudo mn --topo single,3 --mac --controller remote

The switches are all trying to connect to a controller, and will increase the period of their attempts to contact the controller, up to a maximum of 15 seconds. Since the OpenFlow switch has not connected yet, this delay may be anything between 0 and 15 seconds. If this is too long to wait, the switch can be configured to wait no more than N seconds using the --max-backoff parameter.

Make sure the reference controller used before is not running, so that NOX can use port 6633: 

sudo killall controller

Go to the directory holding the built NOX executable (~/noxcore/build/src) in the other SSH window: 

cd ~/noxcore/build/src

Then, in the same window, start the base Python hub code: 

./nox_core -v -i ptcp: pytutorial

This command told NOX to start the 'tutorial' application, to print verbose debug information, and to passively listen for new switch connections on the standard OpenFlow port (6633).

Wait until the application indicates that the OpenFlow switch has connected. When the switch connects, NOX will print something like this: 

00039|nox|DBG:Registering switch with DPID = 1

Benchmark Hub Controller w/iperf

iperf is a command-line tool for checking speeds between two computers.

Here, you'll benchmark the provided hub; later, when you've implemented your flow-based switch, you should see a nice performance increase.

First, verify reachability. Mininet should be running, along with the NOX tutorial in a second window. In the Mininet console, run: 

pingall

This is just a sanity check for connectivity. Now, in the Mininet console, run: 

iperf

This Mininet command runs an iperf TCP server on one virtual host, then runs an iperf client on a second virtual host. Once connected, they blast packets between each other and report the results.

Now, compare your number with the reference controller. Close the NOX tutorial controller, with Ctrl-c. Then start up the reference controller in the SSH terminal: 

controller -v ptcp:

Run another iperf test in the Mininet console: 

iperf

For comparison, a test with a Core i5 2.4 Ghz laptop yielded about 3 Gbps internal bandwidth. Since iperf is only using one flow, one core, and TCP, this test actually under-reports the bandwidth available for Mininet; we've seen upwards of 10 Gbps with multiple switches and multiple connections.

Once the reference controller has pushed down a flow entry, the controller gets out of the way, and the iperf test becomes more a test of the speed of the OpenFlow switch used and the virtual ethernet pair implementation.

Now compare with the user-space switch. In the mininet console: 

exit

Start the same Mininet with the user-space switch: 

sudo mn --topo single,3 --mac --controller remote --switch user

Run one more iperf test with the reference controller: 

iperf

See a difference? With the user-space switch, packets must cross from user-space to kernel-space and back on every hop, rather than staying in the kernel as they go through the switch. The user-space switch is easier to modify (no kernel oops'es to deal with), but slower for simulation.

Exit Mininet: 

exit

Bring back the kernel OpenFlow switch: 

sudo mn --topo single,3 --mac --controller remote

Close the reference controller and re-start the NOX tutorial. In the other SSH terminal: 

./nox_core -v -i ptcp: pytutorial

Verify Hub Behavior with tcpdump

Now we verify that hosts can ping each other, and that all hosts see the exact same traffic - the behavior of a hub. To do this, we'll create xterms for each host, and view the traffic in each. In the Mininet console, start up three xterms: 

xterm h2 h3 h4

Arrange each xterm so that they're all on the screen at once. This may require reducing the height of to fit a cramped laptop screen.

In the xterms for h3 and h4, run tcpdump, a utility to print the packets seen by a host: 

sudo tcpdump -XX -n

In the xterm for h2, send a ping: 

ping -c1 10.0.0.3

The ping packets are now going up to the controller, which then floods them out all interfaces except the sending one. You should see identical ARP and ICMP packets corresponding to the ping in both xterms running tcpdump.

Now, see what happens when a non-existent host doesn't reply. From h2 xterm: 

ping -c1 10.0.0.5

You should see three unanswered ARP requests in the tcpdump xterms. If your code is off later, three unanswered ARP requests is a signal that you might be accidentally dropping packets.

Create Learning Switch

Next, you'll modify the provided hub to act as an L2 learning switch. In this application, the switch will examine each packet and learn the source-port mapping. Thereafter, the source MAC address will be associated with the port. If the destination of the packet is already associated with some port, the packet will be sent to the given port, else it will be flooded on all ports of the switch.

Later, you'll turn this into a flow-based switch, where seeing a packet with a known source and dest causes a flow entry to get pushed down.

Open Hub Code and Begin

The file you'll modify is ~/noxcore/src/nox/coreapps/tutorial/pytutorial.py. Open this file in your favorite editor.

Most of the code will go in one function, learn_and_forward(). There isn't much code here, yet it's sufficient to make a complete hub, and an example of a minimal NOX app. You'll need to add roughly 10 lines to make a learning switch.

Each time you change and save this file, make sure to restart NOX, then use pings to verify hub or Ethernet learning switch behavior. Hosts that are not the destination should display no tcpdump traffic after the initial broadcast ARP request.

Learning Python

This section introduces Python, giving you just enough to be dangerous.

Python:

  • is a dynamic, interpreted language. There is no separate compilation step - just update your code and re-run it.
  • uses indentation rather than curly braces and semicolons to delimit code. Four spaces denote the body of a for loop, for example.
  • is dynamically typed. There is no need to pre-declare variables and types are automatically managed.
  • has built-in hash tables, called dictionaries, and vectors, called lists.
  • is object-oriented and introspective. You can easily print the members variables and functions of an object at runtime.

Common operations:

To initialize a dictionary: 

mactable = {}

To add an element: 

mactable[0x123] = 2

To check for dictionary membership: 

if 0x123 in mactable:
    print 'element 2 in mactable'

To print a debug message in NOX: 

log.debug('saw new MAC!')

To print an error message in NOX: 

log.error('unexpected packet causing system meltdown!')

To print all member variables and functions of an object: 

print dir(object)

To comment a line of code: 

# Prepend comments with a #; no // or /**/

More Python resources:

Learning NOX

The subsections below give details about NOX APIs that should prove useful in the exercise.

Sending OpenFlow messages

The main files for the NOX Python API are at: 

~/noxcore/src/nox/lib/core.py

Some functions that will be useful include: 

Component.install_datapath_flow( ... ) # push a flow to a switch
Component.send_openflow( ... ) # send a packet out a port

See core.py for more API details.

Note that install_datapath_flow() takes in an attributes dictionary with parts of the OpenFlow match. Here's an example: 

attrs = {}
attrs[core.IN_PORT] = inport
attrs[core.DL_DST] = packet.dst

install_datapath_flow also requires a list of actions. Here's another example: 

actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]

This code creates a list of actions, where the first action is defined as a forward-port action, with a max_len field of 0 (meaning, send entire packet to controller) and output port 'outport'. Yes, the NOX documentation could make that more clear. Sometimes you need to check the API code.

For more information about OpenFlow constants, see the main OpenFlow types/enums/structs file, openflow.h, in 

~/openflow/include/openflow/openflow.h

Parsing Packets with the NOX packet libraries

The NOX packet parsing libraries are automatically called to parse a packet and make each protocol field available to Python.

The parsing libraries are in: 

~/noxcore/src/nox/lib/packet/

Each protocol has a corresponding parsing file.

For the first exercise, you'll only need to access the Ethernet source and destination fields. To extract the source of a packet, use the dot notation: 

packet.src

The Ethernet src and dst fields are stored as arrays, so you'll probably want to to use the mac_to_str() function or mac_to_int. I suggest mac_to_str, and it avoids the need to do any hex conversion when printing. The mac_to_str() function is already imported, and comes from packet_utils in ~noxcore/src/nox/lib/packet/packet_utils.py

To see all members of a parsed packet object: 

print dir(packet)

Here's what you'd see for an ARP packet: 

['ARP_TYPE', 'IP_TYPE', 'LLDP_TYPE', 'MIN_LEN', 'PAE_TYPE', 'RARP_TYPE', 
 'VLAN_TYPE', '__doc__', '__init__', '__len__', '__module__', '__nonzero__',
 '__str__', 'arr', 'dst', 'err', 'find', 'hdr', 'hdr_len', 'msg', 'next', 'parse', 
 'parsed', 'payload_len', 'prev', 'set_payload', 'src', 'tostring', 'type',
 'type_parsers']

Many fields are common to all Python objects and can be ignored, but this can be a quick way to avoid a trip to a function's documentation.

Testing Your Controller

To test your controller-based Ethernet switch, first verify that when all packets arrive at the controller, only broadcast packets (like ARPs) and packets with unknown destination locations (like the first packet sent for a flow) go out all non-input ports. You can do this with tcpdump running on an xterm for each host.

Once the switch no longer has hub behavior, work to push down a flow when the source and destination ports are known. You can use dpctl to verify the flow counters, and if subsequent pings complete much faster, you'll know that they're not passing through the controller. You can also verify this behavior by running iperf in Mininet and checking that no OpenFlow packet-in messages are getting sent.

Let the instructor know when your learning switch works!

Support Multiple Switches

Your controller so far has only had to support a single switch, with a single MAC table. In this section, you'll extend it to support multiple switches.

Start mininet with a different topology. In the Mininet console: 

exit
sudo mn --topo linear --controller remote

Your created topology looks like this:


Linear2.png


This will create a 2-switch topology where each switch has a single connected host.

Now, modify your switch so that it stores a MAC-to-port table for each DPID. This strategy only works on spanning tree networks, and the delay for setting up new paths between far-away hosts is proportional to the number of switches between them. Other modules with NOX, including routing, act smarter. NOX routing, for example, maintains an all-pairs shortest path data structure and will immediately push down all flow entries needed to previously-seen sources and destinations.

After the mods, to verify that your controller works, in the Mininet console, run: 

pingall

Benchmark Your Controller

cbench is a program for testing OpenFlow controllers by generating packet-in events for new flows. More info is available at the oflops/cbench page. Cbench is alpha-level software that only works with OpenFlow v0.8.9, so beware.

Cbench emulates a bunch of switches which connect to a controller, send packet-in messages, and watch for flow-mods to get pushed down.

To benchmark your (flow-based) controller, first close down Mininet from the Mininet console: 

exit

Kill your controller, too, with Ctrl-c, then re-start it without the verbose option - this is a performance test after all. 

./nox_core -i ptcp: pytutorial

Start up cbench in the previously-Mininet terminal: 

cd ~/oflops/cbench
./cbench -l5

This command runs 5 loops with 16 virtual switches, then reports the number of flow setups per second. The value shown under-reports the real number, and in effect, measures latency. Cbench by default only sends additional packet-ins when one has been received.

To run cbench in throughput mode, with many simultaneous in-flight packet-ins, run: 

./cbench -l5 -t

To compare with the reference controller, close down your controller with Ctrl-c, and start up the reference controller: 

controller ptcp:

Re-run cbench: 

./cbench -l5 t

Is it a little faster? Straight C generally wins against interpreted Python. What about C++? To run the built-in NOX C++ switch, run: 

./nox_core -i ptcp: switch

Compare cbench both with and without -t; how do the numbers compare?

You can improve these numbers by expanding your controller to use multiple cores or by compiling NOX with the --ndebug option, but these are out of scope for the tutorial.

Congratulations on getting this far. You have two options:

  • add layer-3 forwarding capabilities to your switch
  • add firewall capabilities to your switch

Proceed to the section you prefer.

Create Router

In this exercise, you'll make a static layer-3 forwarder/switch. It's not exactly a router, because it won't decrement the IP TTL and recompute the checksum at each hop (so traceroute won't work). Actions to do TTL and checksum modification are expected in the upcoming OpenFlow version 1.1. However, it will match on masked IP prefix ranges, just like a real router.

From RFC 1812:

An IP router can be distinguished from other sorts of packet switching devices in that a router examines the IP protocol header as part of the switching process. It generally removes the Link Layer header a message was received with, modifies the IP header, and replaces the Link Layer header for retransmission.

To simplify the exercise, your "router" will be completely static. With no BGP or OSPF, you'll have no need to send or receive route table updates.

Each network node will have a configured subnet. If a packet is destined for a host within that subnet, the node acts as a switch and forwards the packet with no changes, to a known port or broadcast, just like in the previous exercise. If a packet is destined for some IP address for which the router knows the next hop, it should modify the layer-2 destination and forward the packet to the correct port.

Our hope is that this exercise will show that with an OpenFlow-enabled forwarding device, the network is effectively layerless; you can mix switch, router, and higher-layer functionality.

Create Topology

You'll need a slightly different topology, something like this:


Router topo.png


… which will need to be described in a way that Mininet will understand.

There's an example custom topology at: 

~/mininet/custom/topo-2sw-2host.py

First, copy the example to a new file: 

cp ~/mininet/custom/topo-2sw-2host.py mytopo.py

To run a custom topology, pass Mininet the custom file and pass in the custom topology: 

sudo mn --custom mytopo.py --topo mytopo --mac

Then, in the Mininet console, run: 

pingall

Now, modify your topology file to match the picture and verify full host connectivity with pingall in the Mininet console.

Set up hosts

Next, set up IP configuration on each virtual host to force each one to send to the gateway for destination IPs that are outside of their configured subnet.

You'll need to configure each host with a subnet, IP, gateway, and netmask.

Testing your router

If the router works properly:

  • attempts to send from 10.0.1.2 to an unknown address range like 10.99.0.1 should yield an ICMP destination unreachable message.
  • packets sent to hosts on the same subnet should be treated like before.
  • packets sent to hosts on a known address range should have their MAC dst field changed to that of the next-hop router.
  • the router should be pingable, and should generate an ICMP echo reply in response to an ICMP echo request.

The exercise is meant to give more practice with packet parsing and show how to use OpenFlow to modify packets.

Let the instructor know when you get stuck or when you complete the exercise. This one is more open, and you'll need different actions (MAC dst modify) and matching (IP dst/prefix combinations), for which NOX core.py should help.

Create Firewall

In this exercise, you'll modify your switch to reject connection attempts to specific ports, just like a firewall.

This exercise is meant to show how OpenFlow can even do basic layer-4 tasks. Other uses could even extend into layer 7; cookie-based load balancing, for example.

Testing your Firewall

To run iperf on xterms, on the xterm for h2, start up a server: 

iperf -s

Then on the xterm for h3 start up a client: 

iperf -c 10.0.0.2

Your task is to prevent this from happening, and to block all flow entries with this particular port.

You can change the port used by iperf with the -p option. Both server and client will need this option specified. Again, NOX core.py will be useful for figuring out how to specify a flow entry that matches on wildcard ports. Also note that an empty action list will drop a packet; there's no explicit drop action.

Learn More

OpenFlow

To learn more about OpenFlow in general, consult the main OpenFlow page. There are videos, blog entries, and more. Check the wiki for link to OpenFlow-based projects and demos.

Additional Tools

Credits

This tutorial was written by Brandon Heller and beta-tested by Bob Lantz and KK Yap.

Notes

Questions

  • How does one scroll an xterm?
    • possible answer: start it with the flag -sl 500 to store 500 lines of output

Frequently Asked Questions (FAQ)

VirtualBox-specific Instructions

VirtualBox Networking

VirtualBox requires a few extra commands to set up networking.

To allow network access in the VM, execute: 

sudo dhclient eth1

which grabs an IP address. Else, you can change the default interface as shown below and execute sudo dhclient eth0.

Now we need to provide SSH access from your local machine into the VM.

OS X / Linux Users Run the VBoxManage commands below from a local terminal (not the VM window).

Windows Users If you are using Windows Vista, first close Virtual box, to ensure that config files aren't locked. Then open a command line prompt, by click on 'Start', then 'Run', then typing 'cmd' and hitting enter. In the terminal that appears, change to the VirtualBox directory: 

cd "C:\Program Files\Sun\xVM VirtualBox"

All OSes: From a terminal on your local machine (NOT the VM/VirtualBox), run: 

VBoxManage setextradata OpenFlowTutorial "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/HostPort" 2222
VBoxManage setextradata OpenFlowTutorial "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/GuestPort" 22
VBoxManage setextradata OpenFlowTutorial "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/Protocol" TCP

Verify the settings: 

VBoxManage getextradata OpenFlowTutorial enumerate

The three values you added above should be shown.

Save your network settings for future reboots, by powering down the VM, from the VM window: 

sudo poweroff

Restart the OpenFlowTutorial VM once again, by clicking 'Start' at the top of VirtualBox. Login again as user openflow with password openflow.

Windows Users You may need to change the VM configuration. Edit /etc/network/interfaces. Check that it says: 

allow hot-plug eth1
iface eth1 inet dhcp

If it says eth0, change it to eth1; note that to write the file you'll need to use sudo. If you changed the interfaces file, poweroff the machine (sudo poweroff) and then start it again.

VirtualBox SSH Connections

Mac OS X and Linux

To SSH from the local machine to the VM, on the local machine run: 

ssh -Y -l openflow -p 2222 localhost

with password openflow. Test the X11 forwarding by running on the local machine: 

xterm

and a new terminal window should appear. If you have succeeded, you are done with the basic setup. Close the xterm.

Windows XP

Start Xming; nothing exciting will happen but it will show an icon on the right of the toolbar.

Open a terminal: click 'Start', 'run', then enter 'cmd'.

Change to the directory where you saved putty. 

cd <dir>

Run: 

putty.exe -X -P 2222 -l openflow localhost

A new window will pop up; type in the password 'openflow'. Now, type: 

xterm

A white terminal window should appear. If you have succeeded, you are done with the basic setup. Close the xterm.

Retrieved from "http://www.openflow.org/wk/index.php/GEC8Tutorial"