Aaron’s Worthless Words

I was doing an HSRP lab the other day, and a project from the past popped into my head. A customer had a host on a network that was separated from the rest of the network by a 1700 with a couple of FEs. They wanted that host to be NATted to a local address so that they didn’t have to do any routing, which makes sense, I guess. This is just your standard 1-to-1 NAT, so we plunked down a quick config.

The setup had two networks with 192.168.0.0/24 on the customer’s local network and 192.168.1.0/24 on the “DMZ” (Yes, I know it’s not really a DMZ). We want to NAT 192.168.1.100 to 192.168.0.100.

interface FastEthernet0/0
ip address 192.168.0.10 255.255.255.0
ip nat inside
!
interface FastEthernet0/1
ip address 192.168.1.10 255.255.255.0
ip nat outside
!
ip nat inside source static 192.168.1.100 192.168.0.100

This works great, but then the customer asked for some redundancy. He wanted a second router inline that could take over in case of failure on the original. “Piece of cake,” I thought and went about setting up the HSRP stuff on both the original and new second routers.

interface FastEthernet0/0
standby ip 192.168.0.1
standby preempt
standby name MYHSRP

But what about that NAT thing? If the primary router goes down, the NAT goes down with it. If I configure the secondary router for the NAT, we get an IP conflict. I looked around for a while and found a feature introduced in IOS version 12.2(4) that allows a NAT to follow an HSRP or VRRP active member. If a router is the active member of an HSRP or VRRP cluster, the NAT is with that box; if it fails over, the standby IP and the NAT move. Isn’t that cool? The only caveat is that it keys off the name/description of the HSRP/VRRP cluster instead of the ID, so you have to have that configured as we do above.

ip nat inside source static 192.168.1.100 192.168.0.100 redundancy MYHSRP

Let’s test to be sure it works. On the active member (a router called NAT0), we can look at the ARP table and see that this router is speaking for the NAT address of 192.168.0.100 as well as the HSRP IP of 192.168.0.1. The secondary member (NAT1) only has ARP entries for its interface IPs. Looks good so far.

NAT0#show standby brief
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Active local 192.168.0.11 192.168.0.1
NAT0#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.0.100 - c800.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.0.10 - c800.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.1.10 - c800.12bc.0001 ARPA FastEthernet0/1
Internet 192.168.0.1 - 0000.0c07.ac00 ARPA FastEthernet0/0
...
NAT1#show standby brief
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Standby 192.168.0.10 local 192.168.0.1
NAT1#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.11 - c801.12bc.0001 ARPA FastEthernet0/1
Internet 192.168.0.11 - c801.12bc.0000 ARPA FastEthernet0/0


When I shut down F0/0 on NAT0, the HSRP and NAT both roll over to NAT1.

AT0(config)#int f0/0
NAT0(config-if)#shut
*Mar 1 00:30:25.248: %HSRP-5-STATECHANGE: FastEthernet0/0 Grp 0 state Active -> Init
*Mar 1 00:30:27.263: %LINK-5-CHANGED: Interface FastEthernet0/0, changed state to administratively down
*Mar 1 00:30:28.265: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to down
NAT0#
NAT0#sh stand b
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Init unknown unknown 192.168.0.1
NAT0#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.10 - c800.12bc.0001 ARPA FastEthernet0/1
...
NAT1#sh stand b
Interface Grp Prio P State Active Standby Virtual IP
Fa0/0 0 100 P Active local unknown 192.168.0.1
NAT1#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.0.100 - c801.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.1.11 - c801.12bc.0001 ARPA FastEthernet0/1
Internet 192.168.0.11 - c801.12bc.0000 ARPA FastEthernet0/0
Internet 192.168.0.1 - 0000.0c07.ac00 ARPA FastEthernet0/0

Now we have redundancy and NAT on the same cluster. Sweet. I also did the same lab for VRRP, but I’ll spare you the innards. When you configure VRRP, you can give it a description, which is the same as the name in HSRP. You use that string as the redundancy name to have a NAT move with VRRP. Yes, you can do both an HSRP- and VRRP-based NAT at the same time.

*I labbed this out on a Dyanmips/Dynagen instance of two 2651XM routers running 12.4(19) Advanced Enterprise. Like all the configs here, your mileage may vary if you have different versions or feature sets on your routers.

Edit:  I wanted to make a not on GLBP and NAT.  Since all the nodes in a GLBP cluster actually route traffic and are routed to, you can’t use this type of NAT solution with it.  I don’t know what the best practice would be, but I imagine it would involve running HSRP on a few routers as we talked about above.

• Leave a comment... (0)

I use Google Analytics to track the 2 or 3 hits I get a day, and sometimes I see some interesting search terms. Yesterday, some googled up the term “does the ASA 5505 run HSRP”; I think that deserves a short article.

The ASA and PIX firewalls don’t actually run any of the usual HA solutions you use on routers. They don’t do HSPR, VRRP, or GLBP at all. Since firewalls have all sorts of state tables, connection tables, translation tables, blah, blah, blah, they need to share more information than just if they’re alive or not, so they use different methods to provide HA.

Cisco uses two different methods to handle this issue - a failover cable or a failover interface.  On a PIX (above the 501s and 506s), you’ll see a DB-15 interface labelled…wait for it…”failover”.  When a very expensive Cisco cable is placed between the failover ports of two PIXes, the boxes do some election stuff over it and decide on an active and a standby state for each (it does all sorts of stuff, but I won’t go into it).  The other method is basically the same, but, instead of an expensive, proprietary Cisco cable, it uses one of the interfaces to connect the two.  You basically connect a crossover cable between two of them, do some configuration, and we’re good.   The ASA doesn’t come with a failover cable so it uses the interface method, but you need a license for that (imagine that).

Just for the record, if an ASA or PIX becomes the active member of the cluster, it takes over both the IP and MAC address of the primary.  There’s no election and configuration of a standby address.  The use of the real addresses, along with the constant sync of the state data, means a very fast and seamless failover.  I actually SSHed through an FWSM (think of it as a PIX on a blade) and pulled the power plugs from it; I actually only lost one packet.

• Leave a comment... (0)

HSRP (Hot Standby Router Protocol) is a Cisco-proprietary method for supplying a highly-available gateway for hosts to use. GLBP (Gateway Load Balancing Protocol) does the same thing. So, what’s the difference?

HSRP works on layer 3 and provides a standby IP address for hosts on that network to use as their gateway (or other routers to use as a next-hop for a route). Two or more routers are configured with the standby IP on a broadcast interface (usually an Ethernet of some kind), and a passive election is held to determine the active router. This router answers ARP requests for the standby IP with a virtual MAC address, so every host that sends packets to the standby IP winds up sending it to the active router. If the active router dies, another election is held, and a new king is crowned who answers for the virtual MAC; the hosts never know anything happened.

GLBP is a little different and runs on layer 2. Instead of one router taking all the traffic all the time, GLBP provides a mechanism to load-balance the standby IP. I’m sure you figured that out by the name, though. When configured, GLBP provides a standby IP just as HSRP does, but it also provides multiple virtual MAC addresses. When a host on the connected network sends an ARP request, one of the routers answers with the virtual MAC address. The next time a host ARPs, a different router answers with a different virtual MAC address. After all is said and done in our perfect world, you have an equal number of hosts sending traffic to each router doing GLBP via the virtual MACs (this never pans out due to the way machines ARP). If a router dies, one of the other participating routers takes over for that virtual MAC, and the host is none-the-wiser.

If you’re having problems deciding on which one to use, it really all boils down to how many hosts you have on that particular network that you want to be HA. For example, if I had a network that only had two routers as the gateway and a single firewall, I’d use HSRP; there will only be the one firewall ARPing and sending packets the standby IP any traffic so you wind up only using one anyway. If you have a network with a thousand hosts on it (say a web farm), then go with GLBP to balance the traffic across your routers.

• Leave a comment... (0)

We’ve done some tracking with [tag]HSRP[/tag] in other articles, but there are lots and lots of ways to use [tag]object[/tag] [tag]tracking[/tag] on an HSRP device. In our example network, we tracked the interface, and, if it went down, we decremented the standby priority. What if just the line protocol goes down? How about if the BGP peer on the other end stops sending you routes? If you don’t know that object tracking is the answer, you didn’t read the title.

In doing any type of object tracking, the first thing you is…wait for it…create the object. Let’s do the line protocol object first.

track 100 interface S0/0 line-protocol

This creates an object with the object number of 100 that tracks the line protocol of interface S0/0. Now what? If we look back to the HSRP setup we have two routers with HSRP running on each of the FastEthernets. If we add an interface S0/0 for Internet access (or corporate access or POS access or access to your toilet), we probably want to track the line protocol of those interfaces to make sure the interface is still healthy. Here’s the new configuration on the FastEthernet interfaces.

interface FastEthernet 0/0
ip address 10.1.1.11 255.255.255.0
[tag]standby[/tag] 75 ip 10.1.1.1
standby 75 priority 50
standby 75 preempt
standby 75 track 100 decrement 55

Now, when the line protocol of S0/0 goes down, the priority of standby group 75 goes down by 55. Sweet. How about if S0/0 is to an Internet circuit, and the BGP peer stops providing routes? It’s just as easy to set up.

First, you need to find an route in your BGP table that’s going to be stable. I like Google or Yahoo, but it doesn’t really matter. Let’s say the route you want is 1.2.0.0/20, so let’s build the object.

track 101 ip route ip route 1.2.0.0/20 reachability

We put this in the config, and we end up with this.

interface FastEthernet 0/0
ip address 10.1.1.11 255.255.255.0
standby 75 ip 10.1.1.1
standby 75 priority 50
standby 75 preempt
standby 75 track 101 decrement 55

If your router doesn’t have the exact route in object 101, the priority of standby group 75 goes down by 55. Notice I said exact — if you have a bigger or smaller route, it won’t match. You knew that, though. And, yes, you can have more than one track statement in each standby group, so you can track the route and the line protocol at the same time if you want. Good stuff.

If you implement HSRP anywhere, you should probably do tracking of some kind. Check out Cisco’s page on Enhanced Object Tracking for a list of the tracking objects you can use.

• Read comments... (1)

Remember the article on router-on-a-stick? And the one on HSRP? Let’s add to that example network, shall we? Let’s make those routers into edge routers so they connect your internal network to the Internet with some size circuit. Let’s just say they each terminate DS3s to different providers.

Here’s our network now (I’m experimenting with Visio alternatives, so excuse the diagram footer there). Let’s assume that we have [tag]HSRP[/tag] set up like the HSRP article and that we have many sub-interfaces on the Ethernet side of the routers like the ROAS article. Also, Router1 is the HSRP active peer and each router has a default route pointing to the upstream ISP through interface Serial 0/0.

This looks pretty good, but what happens if the DS3 on Router1 goes down? We won’t be able to pass traffic to the Internet at all since all the hosts are using the HSRP IP as their gateways. Oh, God…that sucks. What can we do? HSRP has a tracking feature, and we can use it to monitor the DS3 and decrement [tag]priority[/tag] if something happens to it.

Like everything in the network world, this is a piece of cake. All we have to do is one single line to each of our HSPR standby groups to set this all up. Remember to do each standby group on both routers.

standby 1 track Serial 0/0 decrement 55

Now, when the [tag]interface[/tag] goes down, the HSRP priority will be decremented by 55. If Router1 gets decremented by 55, Router2 will be the active peer since Router1’s new priority will be 45. If Router2 loses the interface and gets decremented by 55, nothing will really happen since Router2 is already the standby peer.

—-

The note: An interface has to be down for this to take place. If you lose line protocol but the interface stays up (the interface is up/down), HSRP won’t decrement the priority. Look out for an article on object tracking later to fix this problem.

• Leave a comment... (0)

In the article describing a router-on-a-stick, I mentioned that I would use two routers that run HSRP for availability, so I figured that I would write up a short post on what it is and how it works.

HSRP (Hot Standby Router Protocol) is a Cisco-proprietary protocol for establishing two or more layer-3 devices as a fault-tolerant gateway. Please note that it is not a routing protocol like OSPF or BGP. HSRP provides availability and fault-tolerance…it does not advertise routes. I actually found several Google results that said it was a routing protocol. Those were on the first page of the results, so be careful when searching! Webopedia.com is terrible.

I’m sure you would like to know how it works, so let’s walk through the process. Each router (we’ll just assume its a router, but you can run HSPR on any Cisco layer-3 device) is configured with a standby group, priority, and standby address. Each advertises its configuration to the others, and, after everyone knows what the other routers’ settings are, each looks at the list of priorities and figures out which one is the lowest. If a router thinks that it’s the lowest priority, it becomes the active router and will start answering for the standby address. If a router isn’t configured with the lowest priority, it becomes the standby router and just chills. Every few seconds, everyone sends hello packets to let everyone know that they’re still alive. If the active router doesn’t answer in a certain amount of time, another internal election occurs, and the router with the lowest priority becomes the new active router. This whole process takes less than 10 seconds and is automatic. As long as at least one router is configured for the standby group, the standby ip is available.

That was awfully technical, so let’s look at an example. Here’s another terrible diagram to show what I’m talking about. I can’t afford Visio. Anyway, both routers have their FastEthernet0/0 on the same network, and we want to configure them as HSRP pairs.

Let’s do the configuring. We’ll use standby group 75 for our configuration. It’s just a number so you can use multiple HSPR configurations on the same interface, so it doesn’t really matter. Router 1 and Router 2 have IP addresses of 10.1.1.11 and 10.1.1.12, respectively. We’ll use 10.1.1.1 as the standby IP. We’ll also say that the priority of Router 1 should be lower just so we can get an example going.

Router 1

interface FastEthernet 0/0
ip address 10.1.1.11 255.255.255.0
standby 75 ip 10.1.1.1
standby 75 priority 50
standby 75 preempt

Router 2

interface FastEthernet 0/0
ip address 10.1.1.12 255.255.255.0
standby 75 ip 10.1.1.1
standby 75 priority 100
standby 75 preempt

The only thing I haven’t noted yet is the preempt command. This tells the router that it can take over the standby IP if its priority says so. Everything else is pretty straightforward and should work like a champ. If you’re using a router-on-a-stick setup, you configure the sub-interfaces instead of the physical interfaces (like F0/0.1 instead of F0/0).

Have fun and let me know if you have any questions.

A note as usual: These are just the basics of HSRP. It can do all sorts of stuff like interface tracking, object tracking, load sharing (it’s a workaround, really), and authentication.

• Leave a comment... (1)