Archive for the ‘switching’ tag
The wife and I had a romantic day driving several hours to a small town to take Cisco exams. If this doesn’t get me some action, I don’t know what else to try.
I’ve already used the phrases “skin of my teeth” and “a pass is a pass” on Twitter today for good reason. Passing is a score of 790, and I blew that away with a 790. One more lapse in concentration and I would have been making up more excuses instead of smiling. I think I’ve mentioned this before, but I have this weird reaction to taking exams where I don’t get nervous at all until after I’m finished. Walking into the testing center, I was fine. Walking out, I was shaking like Northern Virginia. It was so bad that I could barely hold on to the door knob when trying to leave, so I guess that I’m really prouder than I thought I was.
The exam was a total piece of crap. Nearly every diagram was so compressed and blurry that I couldn’t see them at all. Most of the time I can infer what the diagram is showing, but, when your bridge priorities are listed there, it’s pretty hard to find root ports. Absolutely horrible. There were the inevitable spelling errors in there, too. Most of those are fine, but STP and SPT are two different topics that are both covered on this exam. I had no problems figuring out which one they were talking about, but it’s pretty unacceptable to have spelling errors in this exam. Of course, there were also the questions that I feel are unanswerable. Switches in VTP transparent mode behave differently from version 1 to version 2, eh?
After being recommended at Cisco Live this year, I added the Boson ExSIM-Max to the pile of prep materials. It not only helped teach a few new things, but it cleared up a bunch of foggy details. I’m sure that using any other study materials will do the same to some extent, but the Boson stuff provided something else – it helped to teach me to take the exams. I was able to go through the questions and practice figuring out what was being asked, which choices were completely wrong, and how to not get utterly frustrated with the questions. Practice makes perfect, right?
The wife came with me to take her ICND1 exam. She did better than she thought she would, but, alas, no dice this time. She says that she’s glad she’s been through it now and knows exactly what to study. I’m trying to convince her to start her own blog since she’s starting up her cert journey from such a unique place. We’ll see how that works out.
What’s next? I have to find a company to help me prep for the lab now. I’m sure that’s not cheap at all. Maybe I should just blindly sit the lab and see what happens. Maybe not. :)
I’m scheduled to take the CCIE R&S Written exam on 10 July at Cisco Live, and I’ve been asked by a handful of people on Twitter exactly what materials I’m using. I figured it would be a good idea to let everyone know so that we all can determine whether or not I’m on the right track. I may get to the exam and find out that the books I’ve been reading aren’t even close. It’s happened before.
CCNP Routing and Switch Quick Reference, 2nd Edition : This doesn’t have the required details, but I read it to get the rust off the more-basic topics. A good read-through at lunch does wonders for the memory.
CCIE Routing and Switching Certification Guide, 4th Edition : I’m using this book as the cornerstone of my studies, and my study schedule is based off of the topics in here. I’m not expecting all the details to be in this book, but it seems to be going along with the blueprint pretty well. We shall see, shan’t we?
Routing TCP/IP Volume I, 2nd Edition, Routing TCP/IP Volume 2 : I’m not reading the Doyle Bibles cover-to-cover; instead, I’m using them for cross reference. If I’m reading anything that makes no sense or that is worded awkwardly, I just read that section of the Doyle Bible to clear it up. It’s been working great so far.
RFCs : Any RFC mentioned in the text goes into my pile of reading materials. A lot of these are fairly short, but some of them are a huge struggle to work through. I would find it hard to believe if you told me you enjoyed RFC 2328.
Obviously, I hope this will be enough to get me through the exam. We’ll know in less than a month, won’t we? This would put me halfway through my goals for the year with many months to spare…months I’ll need for a lab attempt.
$1400 any questions my way.
Did you catch the article on setting up fault tolerance on the CSM? In that article, I mentioned that Cisco recommends a dedicated trunk for the FT VLAN if you have two HA CSMs in two chassis. Discuss amongst yourselves while I drone on.
Why should you set up a dedicated trunk for this stuff? The most obvious reason is to be sure that normal traffic doesn’t step on the syncing traffic. Since we’re syncing state information as well as configuration, the frames need to arrive in a timely manner. Any errors could potentially disrupt the FT process, which is bad. You surely don’t want the primary to fail only to find out that the standby doesn’t have the complete or current config.
Another reason is to keep the syncing traffic from stepping on normal traffic. The CSM is a pretty robust box and can handle a pretty good chunk of data. If you had a 100Mbps trunk between your chassis, there is the potential for the link to get flooded if the CSM ever starts sending some real data. All things being equal, though, your trunks are probably sized properly for your network, and the addition of the syncing traffic probably won’t affect much.
Let’s review our configuration from the other article.
vlan 83 name CSM-Sync ! module csm 3 ft group 1 vlan 83 priority 100 alt 90 preempt
This snippet creates VLAN 83 and tells the CSM to use it for syncing, but how do we dedicate a trunk for that VLAN? We use the switchport trunk allowed vlan directive. We’ll assume that G1/1 on your primary switch is connected to G1/1 on your standby.
interface GigabitEthernet1/1 description CSM Syncing switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 83 switchport mode trunk
This sets G1/1 up to only allow VLAN 83 across it. If you do a show int G1/1 trunk, you’ll see that this VLAN is the only one allowed, the only one active, and the only one one forwarding on that link. Of course, you’ll need to do the same on the other side to keep traffic flow sane, but it’s fairly easy.
What if G1/1 goes down, though? You’d lose sync, so you probably want to look at a solution for that little problem. You could put in multiple links and let Spanning Tree do the work. You could even turn those links into an EtherChannel for redundancy and throughput. If you have more than two chassis, you could full mesh them with trunks dedicated to VLAN 83. There are a number of ways around the problem. Be creative.
Be sure to send turkey questions my way.
We have a location that’s a few blocks down from the main office here, and we were reviewing the circuit size to make sure it was sized properly. Since not one person knows what’s going on and the trending graphs gave us conflicting details, one of our network dudes took me down to the site to do a physical survey to see what’s going on. Well, besides the fact that no one was there, we discovered a hodgepodge of routers and switches that were cross-connected to one another on multiple floors of the building (I really wish I could post pics to emote the effect). It’s kind of hard to figure out what’s going on when you can’t see both ends of the cable, so we had to abandon all hope.
What are our options, then, to see how things are uplinked and connected? In this case, CDP is the answer. The Cisco Discovery Protocol (CDP), if you don’t know, tells you what other Cisco devices a particular Cisco device in plugged into. So, if you have a 2811 plugged into a 2960, you can see what ports they’re connect on along with some other details. Here’s an example.
Switch1#sh cdp neighbors
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater, P – Phone
Device ID Local Intrfce Holdtme Capability Platform Port ID
Router1 Gig 0/10 122 R 3640 Fas 0/1
Switch2 Gig 0/9 141 S I WS-C2950G-Fas 0/48
As you can see from the output, Switch1′s G0/10 is plugged into Router1′s F0/1 interface, and Switch1′s G0/9 is plugged into Switch2′s F0/48. You can also see that Router1 is a 3640 and a router (the “R” under Capability). Switch2 is a 2950G switch.
So, today, I’m going to start at the head end of my frazzled location and try to figure out where everything is connected. I’ll get all the CDP neighbors for that device, document it, then repeat for the next hop until I’m all the way through. When I’m done, I should have a nice physical map.
Beware, my friends, that the “C” stands for Cisco. It doesn’t stand for Juniper or Nortel or anybody else. The rule is that CDP only shows your Cisco devices that are connected together and won’t show any other devices in the path, but there are exceptions. Since it’s broadcast-based, a lot (maybe all?) non-Cisco switches just pass along the packet to the next hop on layer 2, so you may see CDP neighbor adjacencies between switches that aren’t connected to one another. CDP will think they are, and I don’t know of any way to detect that, so be careful.
Send me money Halloween candy comments if you feel inclined.
We ran into this today, and, though I knew it existed, I never actually saw it in the wild. I’m talking about MAC access-lists.
In the example setup, we have a DMZ off of a firewall that contains a whole mess of servers — email, web, ftp, etc. These should all be in the DMZ for sure, but they shouldn’t talk to each other. If a bad guy was able to own my FTP server, he would have a nice platform to use to attack my email server. That’s not cool, so we’ve put in MAC access-lists to help out.
MAC access-lists do just what you think they do; they put access controls on what MAC addresses can talk to what on a particular port. You build a list of permit and deny lines that you want to use to control access and apply them to a port. Sound familiar? Yeah…it does sound a lot like IP ACLs, doesn’t it? Here’s some technical detail for those that care.
- MAC ACLs can be numbered or named. The range for numbered ACLs is 700-799. The naming conventions follow the same rules as an IP list.
- A port must be a in access mode before you can apply a MAC ACL. You can’t apply them to trunks.
- You can use the any and host directives instead of using MAC/mask combos.
- If you’re feeling froggy, you can actually do a MAC/mask combo, but, since the MACs on your hosts probably aren’t sequential, I don’t see the point.
Let’s build one. Suppose we have a host with the MAC 1111.2222.3333 plugged into a switch on port F0/1. It is a web server and needs access to the mail server on the same network (4444.5555.6666) and the firewall (9999.8888.7777) as a gateway. How do we set it up so that the web server can only speak to those guys?
mac access-list extended WEBSERVER
permit host 1111.2222.3333 host 4444.5555.6666
permit host 1111.2222.3333 host 9999.8888.7777
mac access-group WEBSERVER in
Now that host can only speak to those two MACs on layer 2. Pretty simple yet again. There are some things to note, though.
First of all, this just keeps this host from talking to other MACs on the same network and does nothing to keep packets from other host from reaching our webserver. Though the webeserver won’t be able to respond, one could argue that it’s best practice to apply an outbound MAC ACL that mirrors the inbound.
There’s also an issue of broadcast. Say that the webserver is now trying to send a packet to the mail server to send an alert. One of the first things that it will do is to send a broadcast (ffff.ffff.ffff) asking for an ARP reply. Guess what? That MAC isn’t in the ACL. You could put static ARP entries on the boxes, but it may be easier just to allow the host to talk to the broadcast.
Don’t get layer-2 security mixed up with layer-3 security. This restricts access on (usually) a single IP network where you don’t have routers or firewalls between hosts. Use the old-fashioned IP ACL for between networks and MAC ACLs for between hosts on a network.
Questions? Comments? Bribes? Free money? Send them all my way.
At the office, we reprovision servers like it’s going out of style. It happens so often that my cabling documentation rarely matches what’s actually out in field, which is a pretty big problem when you’re trying to find to what switch port a server is connected. I finally relegated myself to asking for the MAC address of the server, having the admin ping something, and then tracing it down through the CAM table entries of the switches. It works, but the guys really don’t know how a switch populates its CAM table, so they always say “Why can’t you just look on the switch? I shouldn’t have to ping anything.” Here’s one just for the aspiring system admin.
The Content Addressable Memory (CAM) table on a switch keeps track of MAC addresses and on what port they appear, along with some other stuff like age. When a device that’s plugged into a particular port sends a frame to the switch, the switch makes note of the source MAC and the port and checks the CAM table. If it’s a new MAC, it adds an entry in the CAM table; if it’s an existing on a different port, it removes the old entry and adds a new one; if it’s an old MAC on the same port, it updates the age. By default, Cisco switches keep CAM entries for 300 seconds, so they don’t stay there forever.
What about the destination MAC? Good question. That’s a pretty important field when sending a packet, but, when generating a CAM entry, the destination MAC is ignored. If a host talks, a switch knows exactly from where the frame came, but there’s no way to know exactly where it should go without the destination first speaking up.
Let’s set up an example. You have a Cisco 2950 switch that you’ve just powered on with nothing plugged into it except the console cable. If you do a show mac-address-table, you’ll see the CAM table — a table of MAC addresses that the switch knows; you would think that it would be empty since nothing’s plugge din, but the switch has its own MACs, so it always knows those guys. There’s not much to see here yet, though, since we haven’t hooked anything up to the switch yet.
Switch#sh mac-address-table Mac Address Table ------------------------------------------- Vlan Mac Address Type Ports ---- ----------- -------- ----- All 000a.f43b.ddc0 STATIC CPU All 0100.0ccc.cccc STATIC CPU All 0100.0ccc.cccd STATIC CPU All 0100.0cdd.dddd STATIC CPU Switch#
Next, let’s plug a Linux desktop up to it. Once that box has booted, what should you see in the CAM table? If you guessed the MAC of the Linux box, you may be right; it all depends on if the server sent a frame or not. There’s lots of things that run on a Linux box that could send frames on startup — DHCP requests, multicast services, network-based storage — so, more than likely, a frame did get sent. The only way to know it to take a gander.
Switch#sh mac-address-table Mac Address Table ------------------------------------------- Vlan Mac Address Type Ports ---- ----------- -------- ----- ... 1 001c.0cbb.ada2 DYNAMIC Fa0/1 Switch#
Ah…it worked. That’s good, but it’s boring with only a single device. Let’s plug in a simply-configured and fully-booted Cisco router and see what happens. More than likely the router won’t speak until spoken to, so the CAM table won’t update, and the switch won’t know where to send the frame, right? Yes, but the frame still gets sent. If the switch doesn’t know where the destination MAC lives (i.e., it’s not in the CAM table), then it floods the frame out every port except the one on which it was received.
When I first learned that this is how it worked, I immediately wondered why LANs weren’t flooded out constantly. I didn’t think long enough to realize that the host being sent the packet will actually respond within several milliseconds (hopefully), so the CAM table will then have an entry for that guy. In reality, it’s even simpler than that. Since most of the world runs TCP/IP, we have this wonderful thing called ARP. When a host needs to talk to another host on the same network segment (IP subnet), it checks its ARP table, and, if it doesn’t know a MAC for that IP, it will actually ask what MAC it should use via a broadcast message. In a well-behaved network, the mystery host will answer with a “Here I am!” type message, which causes the switch to generate a CAM entry. In a “perfect world”, you should only have a few floods on a switch per day/week/month/year/decade.
Here’s a couple items of note.
- A trunk interface will have a whole bunch of MACs listed as attached to that port. This is quite normal, so don’t freak out.
- If someone plugs a switch or hub into one of your ports, you will see multiple CAM entries for the same port. This is a good way to see who brought in their Linksys hub from home.
- If a host hasn’t sent a frame in more than 5 minutes, it disappears from the CAM table, so the whole discovery process starts over again.
- There’s a limit to the size of a CAM table, so it’s possible to fill it up and then every new destination gets flooded. Wow, I can see your packets.
Have fun. Be safe. Practice safe computing. Lock down your network.
My friend Josh over at blindhog.net has found a collection of cheat sheet gems for the network dude(tte). There’s sheets on BGP, OSPF, Subnetting, QoS, connector types, and more. Check it out.
We run a large number of LANs all over the country that are “controlled” by the particular business unit. We manage the gear, but, since they have the money and have to pay for anything we do, they make the final decision on what gets put in. Sometimes that gets out of hand, as you can well imagine.
good terrible example came up a few months ago. It seems that, at some time in the past, one site needed some more LAN ports, but, instead of calling us and having us send them another switch, one of the “technical people” there brought in a hub from home. It really irks me to see a hub on the switched LAN, but we really have no control over those decisions. They plugged the hub into one of the existing drops somewhere in the building and plugged everyone in. It worked…until somebody moved one of the machines. The machine was at a desk near the hub, and the network cable, still with one end plugged into the hub, was just left lying there. A good Samaritan came by, saw that the hub was not plugged into the network (though it was through another path), and plugged it back in for us — providing a nice second link from the hub to the switch stack in the closet. Take one switch stack, add a hub, insert a switching loop, bake at 350F for a few milliseconds, and you have a broadcast storm. If you don’t know already, broadcast storms are bad and eat switch CPU like the yummy cookies we baked. In this case, several 3750s were taken completely down.
How does one prevent such from happening again? Well, the first thing to do is to get the CTO to tell everyone that they can’t plug hubs into the network. That works about 0% of the time, though, so we had to find a solution that was enforceable. One of my coworkers found the traffic storm control mechanism built into Cisco switches. This mechanism allows you to set thresholds based on broadcast, multicast, and unicast traffic and take action when those are reached.
Here are the gory details. I need to mention, though, that storm-control is configured very differently across platforms and IOS versions. I would say your mileage may vary, but it’s probably more accurate to say that this won’t work on your switch. A 6500 is configured differently than a 4500. A 2900XL is different from a 2950. This will get you going, but you’re going to have to do some research on your own to find out what works on your platform.
interface FastEthernet 0/1
storm-control broadcast level 50
storm-control action shutdown
What just happened? Good question to ask. If broadcast traffic on F0/1 utilizes 50% of available bandwidth, the port is shutdown. That means that if broadcast traffic takes up 50Mbps of bandwidth on this port, the port is admined down just as if you did a shutdown on it.. You should probably do the same for multicast or unicast as well to make sure you don’t get bitten by those. If you don’t want to shut down the port, you can also use the trap action to just send an SNMP trap with the port and information, but that doesn’t prevent very much; the storm will probably wreak havoc before an email for the trap lands on your Crackberry.
Here’s another big disclaimer. Finding a good level for a port can be very, very difficult. A linux box is going to have very different broadcast/multicast/unicast traffic than a Windows box which is different than a Mac. You may have to spend a lot of time analyzing SNMP counters to find out what a good level is. God help you if you have a hub like we did with mixed computer platforms on it.
In my professional life at some point, I came across someone who had a stack of Catalyst 2950 switches all trunked together with their Internet routers connected to the top of the stack. This was all well and good until they kept adding hosts to the “middle” of the stack, then they had all sorts of latency and packet loss.
The old adage of your chain only being as strong as your weakest length holds true in this case. Here, the weakest link is actually the most-congested trunk, though. Let’s step through to see. A 2950 is a 10/100 switch, so a single trunk can handle 100Mbps of traffic. We have 10 of these guys, Switch1 to Switch10, all trunked to the one above and below. If a server in the center of the stack on Switch5 is sending a lot of data to the Internet routers on Switch1, the trunks off of Switch5 will start to get saturated. Switch4 has a few hosts doing the same thing, so traffic from both Switch4 and Switch5 heads towards Switch1, further filling the trunks. Same for Switch3. Same for Switch2. Next thing you know, there’s 184Mbps or so trying to go across a 100Mbps link.
I fixed the problem using EtherChannel. EtherChannel, sometimes called trunking (not Cisco trunks…don’t get confused) or bonding, takes up to 8 interfaces of a switch and binds them logically as one Port-channel interface. The switch then load-balances (not really, but can be pretty close) the traffic across all the links, so, in essence, if you have 2 100Mbps ports in an EtherChannel, you get 200Mbps of bandwidth. If you put in 8 ports, you’ll get 800Mbps. Or, at least, close to it.
There are, as usual, stipulations for using it.
- All ports on the EtherChannel must terminate to the same device on both ends. You can’t have one port go to one switch and another to another switch.
- The ports must use identical media. You can’t have a copper port and a fiber port in the same group.
- The ports must have the same capabilities. You can’t put a 100Mbps port and a 1Gbps port in the same group.
- The ports must be configured identically or it won’t work. This is actually pretty easy to maintain since configuring the Port-channel actually sets the config on all participating ports.
- Both ends of the ports must be EtherChannel. You can’t run it on one switch but not the other.
Enough of that. Let’s configure. The scenario is that you have SwitchA and you want to bind F0/1 and F0/2 into an EtherChannel. Of course, you want to carry traffic from all VLANs, so let’s make it a trunk.
int Port-channel 1
switchport trunk encapsulation dot1q
switchport mode trunk
Easy as pie. Do the same thing on the other end and you have a 200Mbps, bonded trunk.
VLAN Trunk Protocol (VTP) is a little gem on Cisco switches that allows you configure VLANs in one place and have them appear on all of your switches. This is great for large enterprises with 8457839 switches all trunked together because who wants to configure the new VLAN for that one-off application on all 8457839 switches?
VTP works by having designated VTP servers (not real servers like your Linux box, but a switch) tell the rest of the switches in the network with what VLANs they should be configured. All the designated VTP clients say “OK” and configure themselves with those VLANs. When you take a VLAN out of the server, all the clients take it out; when you add a new VLAN, all the clients add it as well. The server and client designation is known as the VTP mode, and there’s one more to mention. When a switch is in VTP transparent mode, he will see VTP from the servers but will ignore them and pass them on to the next switch as if nothing ever happened.
To configure VTP, you have to define the VTP domain, which is a name for a group of switches that exchange VTP information. In general, in an enterprise, all your switches will probably be in the same VTP domain, but there are situations where you have multiple VTP domains on a LAN (like production and test networks). Next, you set the mode on the switch — server, client, transparent. I would suggest you configure a password as well to at least appear as though you’re practicing good security practices (I’m not sure it’s actually secure or not since every switch has the same password and the hash is sent in every packet…somebody enlighten me, please).
Get on your configuring gloves while I give you some more detail. VTP only works over 802.1q or ISL trunks. Also, VTP packets are Ethernet frames with the VTP stuff shoved into the data section. A switch can only be in one VTP domain at a time and can only be in one mode at a time. You can have more than one VTP server for a domain if you’d like (via a revisioning mechnism where the latest update always wins).
Here’s your scenario. You have SwitchA’s port G1/1 plugged into SwitchB’s G2/1. You want SwitchA to be the VTP server and SwitchB to be the client. Let’s get the config in place.
SwitchA(config-if)#switchport trunk encapsulation dot1q
SwitchA(config-if)#switchport mode trunk
SwitchA(config)#vtp mode server
SwitchA(config)#vtp domain VTP-DOMAIN
SwitchA(config)#vtp password VTP.PASS
SwitchB(config-if)#switchport trunk encapsulation dot1q
SwitchB(config-if)#switchport mode trunk
SwitchB(config)#vtp mode client
SwitchB(config)#vtp domain VTP-DOMAIN
SwitchB(config)#vtp password VTP.PASS
Both switches are now in VTP domain “VTP-DOMAIN” and share the password “VTP.PASS”. When you add a VLAN to SwitchA, it automatically gets added to SwitchB. When you add a VLAN to SwitchB, it actually errors out and says something like “Dude, I’m a VTP client. You can’t update me.”
At work, we have a large number of ports but a small number of switches. We also make very few changes to VLANs, so the work needed to maintain the VLANs is very low. Because of that, we actually do use transparent mode on all the switches. I’m also a bit of a purist, and configuring VLANs is something I like to do by hand; in my opinion, some things, especially mechanisms that can take whole networks down, are better left to the human. That’s just me, though.