Aaron’s Worthless Words

We run a large number of LANs all over the country that are “controlled” by the particular business unit. We manage the gear, but, since they have the money and have to pay for anything we do, they make the final decision on what gets put in. Sometimes that gets out of hand, as you can well imagine.

A good example came up a few months ago. It seems that, at some time in the past, one site needed some more LAN ports, but, instead of calling us and having us send them another switch, one of the “technical people” there brought in a hub from home. It really irks me to see a hub on the switched LAN, but we really have no control over those decisions. They plugged the hub into one of the existing drops somewhere in the building and plugged everyone in. It worked…until somebody moved one of the machines. The machine was at a desk near the hub, and the network cable was just unplugged and left lying there. A good Samaritan came by, saw that the hub was not plugged into the network, and plugged it back in for us — providing a nice second link from the hub to the switch stack in the closet. Take one switch stack, add a hub, insert a switching loop, bake at 350F for a few milliseconds, and you have a broadcast storm. If you don’t know already, broadcast storms are bad and eat switch CPU like the yummy cookies we baked. In this case, several 3750s were taken completely down.

How does one prevent such from happening again? Well, the first thing to do is to get the CTO to tell everyone that they can’t plug hubs into the network. That works about 0% of the time, though, so we had to find a solution that was enforceable. One of my coworkers found the traffic storm control mechanism built into Cisco switches. This mechanism allows you to set thresholds based on broadcast, multicast, and unicast traffic and take action when those are reached.

Here are the gory details. I need to mention, though, that storm-control is configured very differently across platforms and IOS versions. I would say your mileage may vary, but it’s probably more accurate to say that this won’t work on your switch. A 6500 is configured differently than a 4500. A 2900XL is different from a 2950. This will get you going, but you’re going to have to do some research on your own to find out what works on your platform.

interface FastEthernet 0/1
storm-control broadcast level 50
storm-control action shutdown

What just happened? Good question to ask. If broadcast traffic on F0/1 utilizes 50% of available bandwidth, the port is shutdown. That means that if broadcast traffic takes up 50Mbps of bandwidth on this port, the port is admined down just as if you did a shutdown on it.. You should probably do the same for multicast or unicast as well to make sure you don’t get bitten by those. If you don’t want to shut down the port, you can also use the trap action to just send an SNMP trap with the port and information, but that doesn’t prevent very much; the storm will probably wreak havoc before an email for the trap lands on your Crackberry.

Here’s another big disclaimer. Finding a good level for a port can be very, very difficult. A linux box is going to have very different broadcast/multicast/unicast traffic than a Windows box which is different than a Mac. You may have to spend a lot of time analyzing SNMP counters to find out what a good level is.  God help you if you have a hub like we did with mixed computer platforms on it.

• Leave a comment... (2)

Josh over at Blindhog.net has an article linking to a bunch of Cisco IP Phone videos — from the 7906 to the 7975. These are Cisco videos and a good place to start if you don’t know anything about their IP phones.

• Leave a comment... (0)

I was doing an HSRP lab the other day, and a project from the past popped into my head. A customer had a host on a network that was separated from the rest of the network by a 1700 with a couple of FEs. They wanted that host to be NATted to a local address so that they didn’t have to do any routing, which makes sense, I guess. This is just your standard 1-to-1 NAT, so we plunked down a quick config.

The setup had two networks with 192.168.0.0/24 on the customer’s local network and 192.168.1.0/24 on the “DMZ” (Yes, I know it’s not really a DMZ). We want to NAT 192.168.1.100 to 192.168.0.100.

interface FastEthernet0/0
ip address 192.168.0.10 255.255.255.0
ip nat inside
!
interface FastEthernet0/1
ip address 192.168.1.10 255.255.255.0
ip nat outside
!
ip nat inside source static 192.168.1.100 192.168.0.100

This works great, but then the customer asked for some redundancy. He wanted a second router inline that could take over in case of failure on the original. “Piece of cake,” I thought and went about setting up the HSRP stuff on both the original and new second routers.

interface FastEthernet0/0
standby ip 192.168.0.1
standby preempt
standby name MYHSRP

But what about that NAT thing? If the primary router goes down, the NAT goes down with it. If I configure the secondary router for the NAT, we get an IP conflict. I looked around for a while and found a feature introduced in IOS version 12.2(4) that allows a NAT to follow an HSRP or VRRP active member. If a router is the active member of an HSRP or VRRP cluster, the NAT is with that box; if it fails over, the standby IP and the NAT move. Isn’t that cool? The only caveat is that it keys off the name/description of the HSRP/VRRP cluster instead of the ID, so you have to have that configured as we do above.

ip nat inside source static 192.168.1.100 192.168.0.100 redundancy MYHSRP

Let’s test to be sure it works. On the active member (a router called NAT0), we can look at the ARP table and see that this router is speaking for the NAT address of 192.168.0.100 as well as the HSRP IP of 192.168.0.1. The secondary member (NAT1) only has ARP entries for its interface IPs. Looks good so far.

NAT0#show standby brief
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Active local 192.168.0.11 192.168.0.1
NAT0#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.0.100 - c800.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.0.10 - c800.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.1.10 - c800.12bc.0001 ARPA FastEthernet0/1
Internet 192.168.0.1 - 0000.0c07.ac00 ARPA FastEthernet0/0
...
NAT1#show standby brief
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Standby 192.168.0.10 local 192.168.0.1
NAT1#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.11 - c801.12bc.0001 ARPA FastEthernet0/1
Internet 192.168.0.11 - c801.12bc.0000 ARPA FastEthernet0/0


When I shut down F0/0 on NAT0, the HSRP and NAT both roll over to NAT1.

AT0(config)#int f0/0
NAT0(config-if)#shut
*Mar 1 00:30:25.248: %HSRP-5-STATECHANGE: FastEthernet0/0 Grp 0 state Active -> Init
*Mar 1 00:30:27.263: %LINK-5-CHANGED: Interface FastEthernet0/0, changed state to administratively down
*Mar 1 00:30:28.265: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down
NAT0#
NAT0#sh stand b
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Init unknown unknown 192.168.0.1
NAT0#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.10 - c800.12bc.0001 ARPA FastEthernet0/1
...
NAT1#sh stand b
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Active local unknown 192.168.0.1
NAT1#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.0.100 - c801.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.1.11 - c801.12bc.0001 ARPA FastEthernet0/1
Internet 192.168.0.11 - c801.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.0.1 - 0000.0c07.ac00 ARPA FastEthernet0/0

Now we have redundancy and NAT on the same cluster. Sweet. I also did the same lab for VRRP, but I’ll spare you the innards. When you configure VRRP, you can give it a description, which is the same as the name in HSRP. You use that string as the redundancy name to have a NAT move with VRRP. Yes, you can do both an HSRP- and VRRP-based NAT at the same time.

*I labbed this out on a Dyanmips/Dynagen instance of two 2651XM routers running 12.4(19) Advanced Enterprise. Like all the configs here, your mileage may vary if you have different versions or feature sets on your routers.

Edit:  I wanted to make a not on GLBP and NAT.  Since all the nodes in a GLBP cluster actually route traffic and are routed to, you can’t use this type of NAT solution with it.  I don’t know what the best practice would be, but I imagine it would involve running HSRP on a few routers as we talked about above.

• Leave a comment... (0)