An OpenFlow Programmable Patch Panel

Kudos to Dan Hersey, a systems engineer at Big Switch Networks, who recently set up a programmable patch panel using OpenFlow.  Its a creative and very useful way of taking advantage of open source tools like Floodlight and Indigo. He even went all the way and wrote an IPhone app that can push static flows to help manage the patch panel he built.  Awesome! Dan’s Article follows below.

When I started exploring the OpenFlow protocol I was struck by how fundamentally it could change networking.  OpenFlow truly provides a clean slate to correct the network mess we have created over the last 20 years. Solutions such as network virtualization, lawful intercept, network service insertion and other super cool (and highly desirable) concepts based on OpenFlow have garnered the most press but there is limitless possibilities.

Over the last couple of months I’ve had a number of friends and former colleagues ask ‘How can I start using OpenFlow?’, so I though I would write a tutorial on something interesting, practical, and simple to get them off the ground. A programmable patch panel using OpenFlow seemed like a good start.

There are a number of commercial Layer 1 switches on the market today and they do a very good job but I thought I’d have a little fun with some switches laying around and do a little OpenFlow education at the same time.

An Open Source Programmable Patch Panel

One of the characteristics of a OpenFlow switch is that all packets are (supposed to be) forwarded based on how the switch is programmed. In the case of a patch panel we want all packets, including link layer packets such as CDP/LLDP or BPDUs, to be forwarded between two ports. The devices attached to a passive patch panel have no idea the panel is in the middle.  This is the behavior I want to replicate.

From an OpenFlow standpoint the concept behind a programmable patch panel is very simple.  Determine the two ports on a switch you want to interconnect and insert two static flows that directs any incoming traffic on one port to the other port.  There is some interesting logic you may want to build into a more complete solution but I’ll stick with the very basics.  The flow table would look something like this:

To get started we need an OpenFlow capable switch and optionally an OpenFlow controller.  I say the controller is option because many of the OpenFlow capable switches allow you to insert static flows without a controller.  I’m going to use a controller in this design because over time I want to deploy multiple patch panels and the controller will offer a common interface for all the switches, regardless of brand.

The switch I’m going to use is the Pronto 3290, which is a 48 x 1Gb + 4 x 10Gb switch.  The list price for the Pronto is $2750, which makes it a fairly cheap platform to build this on. You should be able to use any OpenFlow enabled switch for this. I’ve seen some of the HP OpenFlow capable switches on eBay for under $800.  You might even have a switch laying around that is capable of OpenFlow with a firmware, almost every switch vendor is showing OpenFlow capability at this point.

I replaced the native firmware on the Pronto with an open source firmware called Indigo. Indigo is arguably the most complete implementation of the OpenFlow 1.0 spec and it is very easy to set up.  Indigo is a breeze to get connected to an OpenFlow controller.  If you have one of the platforms supported by Indigo I highly recommend you give it a try.

For the controller I’m going to use Floodlight, an open source controller with a RESTful API. I recommend you download the prepackaged VM (the nightly build), which is what I’ll be using in my examples. Floodlight is designed to be programmed via API so it does not have a CLI.  To interact with the controller I’ll be using curl which is supported natively from the Floodlight VMs shell.

Bugs, Bad Cables and Bad Luck

What I hoped to be a 30 minute project of documenting and testing turned into hours in the lab troubleshooting the setup.  Interestingly enough OpenFlow was the easiest part to get working.  The environment consisted of two switches attached to the OpenFlow switch with the intent to connecting them together.

When I built the topology I started to see strange behavior right off the bat.  The most basic of items are sometimes that hardest to troubleshoot.  In this case, two of the four Cat5e cables I used to build the topology were bad.  To make matters worse, one of the two cables I grabbed as replacement was also bad. Needless to say there was time wasted on ensuring the physical layer worked.

Once my cable nightmare was worked out I ran into a genuine networking issue.  It was easy to see that something was wrong because before any flows were inserted into the Pronto switch it was forwarding packets.  This was obviously not what I expected to see and it produced some interesting network behavior.  The most telling sign was the output of ‘show cdp neighbor’ one of the Cisco switches.

3750-1#show cdp neighbor
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
 S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone
Device ID Local Intrfce Holdtme Capability Platform   Port ID
3750-1    Gig 1/0/4     139     S I        WS-C3750E  Gig 1/0/3
4948-1    Gig 1/0/4     85      R S I      WS-C4948   Gig 1/40
4948-1    Gig 1/0/3     85      R S I      WS-C4948   Gig 1/40
4948-1    Gig 1/0/4     25      R S I      WS-C4948   Gig 1/39
4948-1    Gig 1/0/3     25      R S I      WS-C4948   Gig 1/39

The Cisco 3750 was seeing CDP from five interfaces, which included one from itself!  It didn’t take long to figure out that even without flows installed the Pronto was forwarding broadcast traffic and CDP, being a broadcast based protocol, was being flooded to all interfaces.  I’m surprised the Cisco switch didn’t complain in some way.

This appears to be a bug in the Indigo firmware.  While this would appear to squash the usefulness of the switch as a ‘patch panel’ OpenFlow actually came to the rescue here.

Once a port on the Pronto switch is explicitly programmed with a flow it begins to treat traffic as you would expect, based on the flow criteria.  For example, if a flow is inserted to forward all traffic from port 3 to port 40 all packets, including Layer 2 broadcast (like CDP), were forwarded to port 40 only.  This is the behavior we want and expected.  There was still a problem to overcome which is that ports that don’t have explicit flows attached may still broadcast and even ports 3 and 40 (in the above example) would still receive the broadcast.  To eliminate this a flow was inserted on each interface, that was broadcasting traffic, to drop the traffic until I was ready to patch that specific interface.  Not a pretty solution but an effective one for sure.

I’ve asked the folks that support Indigo to look into the broadcast issue.  When a root cause is found and the problem is resolved I’ll post an update.

Let The Connectivity Begin!

After fighting to get to a stable starting point it was time to connect Floodlight to the Pronto switch. The Indigo and Floodlight websites have great instructions on getting started so I’ll assume you can get both to the point where they have IP addresses assigned. If all else fails they both use DHCP by default.

Indigo has both a CLI and a web-based GUI that is designed to get you up and running in just a few minutes.  The only step we need on the switch side is to tell Indigo the IP address of the Floodlight controller.  Using a browser, connect to the switch and navigate to the ‘Daemon Settings’ tab within the ‘Configuration’ screen.  Enter the IP address of your Floodlight controller and make sure the ‘OpenFlow Controller TCP port’ is set to 6633.

Take note of the switches datapath ID (DPID).  This is how Floodlight will identify the switch and how we’ll refer to it in our API calls.  You may set it to an arbitrary hex value or leave the default.  As long as it’s unique to the controller you are fine.  Click on the ‘Save Configuration’ button.  Indigo will ask you restart the daemons, which is required for the switch to connect.

Once the switch has Floodlight’s IP we jump over to Floodlight and confirm that it can see the switch. To do this I’ll ask Floodlight for a list if switches it sees.  It should be noted that Floodlight responds to API calls on port 8080.  If Floodlight returns a 404 error, more than likely forgetting to insert the port number is the cause. Floodlight should return the DPID you set within Indigo as shown below.

floodlight@localhost:~$ curl [{“dpid”:”00:00:12:34:56:ab:cd:ef”}]

Now that Floodlight and Indigo are good friends we can start patching devices together. Inserting flows through Floodlight is fairly straight forward. Floodlight includes an API for inserting static flows (flows that won’t time out) which we can call via curl.  The API requires a few items to be present for our flows.

Switch DPID – The switch to install the flow on.

Flow Name – This is an arbitrary value used to identify the flow.  It must be unique so we will use the number of the ingress port.

Ingress Port – The ingress port traffic is coming in on.

Action – What do we want to do with the traffic?  In our case we simply want to forward it to a destination port.

Floodlight and OpenFlow support a variety of flow matches and output actions but port based match and output will meet our basic needs.

For a patch panel we want to direct all traffic on two selected ports to each other.  A single flow is unidirectional so to do this we need two flows per ‘patch’.  In my topology I started by connecting the 4948 port 40 to the 3750 port 3. Using Floodlight I made two API calls would look like this:

floodlight@localhost:~$ curl -d ‘{“switch”:”00:00:12:34:56:ab:cd:ef”, “name”:”flow-40″, “ingress-port”:”40″, “actions”:”output=3″}’

{“status” : “Entry pushed”}

floodlight@localhost:~$ curl -d ‘{“switch”:”00:00:12:34:56:ab:cd:ef”, “name”:”flow-3″, “ingress-port”:”3″, “actions”:”output=”40″}’

{“status” : “Entry pushed”}

You can see the Floodlight verifies the entries have been pushed to the switch but you can verify them within the switch using either the GUI or CLI.  You can find them in the GUI under the ‘Flowtable’ tab in the monitoring section.  To view them via the CLI in Indigo you can SSH into the switch (root/OpenFlow) and issue a ‘cmd flowtable’.

IndigoSwitch-1# cmd flowtable
Flow table show
port      L3src     L3dst  L4src   L4dst  Out_P  packets
  40      0       0      3        5
   3      0       0     40        5

I quickly patched together ports 39 and 4 by making two more API calls against Floodlight:

floodlight@localhost:~$ curl -d ‘{“switch”:”00:00:12:34:56:ab:cd:ef”, “name”:”flow-39″, “ingress-port”:”39″, “actions”:”output=4″}’

{“status” : “Entry pushed”}

floodlight@localhost:~$ curl -d ‘{“switch”:”00:00:12:34:56:ab:cd:ef”, “name”:”flow-4″, “ingress-port”:”4″, “actions”:”output=”39″}’

{“status” : “Entry pushed”}

Once these two flows were inserted I finally saw the behavior I expected, which is two switches talking together as if they were connected via a patch panel.  The only difference was that patch panel in this case was a switch using OpenFlow, meaning I can re-patch cables via API instead of running down to the lab to move them around.  Sweet!

3750-1#show cdp neighbor
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
 S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone 
Device ID   Local Intrfce   Holdtme   Capability   Platform   Port ID
4948-1      Gig 1/0/4       128       R S I        WS-C4948   Gig 1/39
4948-1      Gig 1/0/3       128       R S I        WS-C4948   Gig 1/40

3750-1#show spanning-tree
 Spanning tree enabled protocol ieee
 Root ID Priority 32801
         Address  001e.4a73.1aa1
         Cost     4
         Port     3  (GigabitEthernet1/0/3)
         Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32801 (priority 32768 sys-id-ext 33)
          Address 0023.eac3.f660
          Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
          Aging Time 300
Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- ------------------------------
Gi1/0/3          Root FWD 4         128.31   P2p
Gi1/0/4          Altn BLK 4         128.32   P2p

4948-1#show cdp neighbor
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone
Device ID   Local Intrfce   Holdtme   Capability   Platform      Port ID
3750-1      Gig 1/39        134       S I          WS-C3750E-Gig 1/0/4
3750-1      Gig 1/40        146       S I          WS-C3750E-Gig 1/0/3

4948-1#show spanning-tree
 Spanning tree enabled protocol rstp
 Root ID Priority 32801
         Address  001e.4a73.1aa1
         This bridge is the root
         Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32801 (priority 32768 sys-id-ext 33)
          Address 001e.4a73.1aa1
          Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
          Aging Time 300
Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- ------------------------------
Gi1/39           Desg FWD 20000     128.31   P2p Peer(STP)
Gi1/40           Desg FWD 20000     128.32   P2p Peer(STP)

This is just one of many applications for OpenFlow and SDN.  It may be trivial compared to other use cases but a programmable patch panel is something you can use today, it’s a pretty cool project, and an easy way to say you know a thing or two about OpenFlow.

Give it a try and let me know how you make out.  You may even find yourself writing a full-fledged application on top of Floodlight to manage your new lab wide patch panel project. I’d love to see someone write an application for Floodlight that patches together two devices based on the CDP/LLDP information. Who wants to take up the challenge?

Bonus For iPhone Users (sorry Android fans)

I haven’t written a useful line of code in over a decade but I’ve managed to slap together an ugly (visually and code wise) iPhone App that allows you to ‘patch’ two interfaces together using Floodlight. You can download the project here.  You will need Apples Xcode on a Mac to compile and push it to your iPhone, or you can run it right from the iOS simulator provided with Xcode.  This code has no error checking or status reporting and does what you ask blind to any configuration consequences.  No warranty is provided and tech support is via Western Union Telegram only :-)  Bottom line, use at your own risk!  Feel free to take this basic code and turn it into something more complete.

Comments are closed.