ASA and Proxy ARP
Wow. A new entry. Everyone sit down before you pass out.
I’ve got a real-world example for you today. We have an ASA 5540 installed at a business unit with interfaces in multiple networks, including one containing the production servers and another containing the accounting servers. The production network sits on a 7600 that’s not ours, so, to avoid IP conflicts, we are statically NATting connections into that network. The 7600 has with many, many VLANs, and, since the firewall production servers are on different VLANs, there’s an interface VLAN between us. Sounds pretty straightforward, but it just wasn’t working when we try to connect between the interfaces.
When we tried to connect from the accounting servers to the production gear, the firewall saw the SYN, built the outbound connection, sent the packet on, and waited. Nothing back. SYN timeout. The vendor on the production side checked the routing. Fine. Checked the ACLs. None installed. When the (other) vendor ran TCPDump on the production servers, they saw the SYN landing and the SYN-ACK leaving, but it never got to the ASA. We even looked at the inline IDS and still didn’t see the SYS-ACK hitting the firewall. It was simply not getting passed on.
I got tired of walking people through stuff over the phone, so I drove up there to see what I could find. When I checked the ARP table on the 7600, I noticed that the statically NATted IP we were serving was conveniently incomplete. For those who don’t know, that means that the 7600 was ARPing for the address, but nothing was answering for it. Obviously, our ASA should be answering, right? To make the situation a little more dire, I did a debug arp (or something close) on the firewall and generated an ARP request; the firewall saw the request but just ignored it. Ugh!
If you couldn’t tell by the title, it turns out that the solution was to enable proxy ARP. It’s off by default for good reason, but here’s how to enable it.
no sysopt noproxyarp PRODUCTIONINTERFACE
Enabling proxy ARP, however, could be a security issue. Any time you use the word “proxy”, there is a potential to spoof addresses, and, in this case, an attacker could (potentially) use the firewall to discover hosts that are on the other side of it. That wouldn’t be good.
A more-secure solution is to use static ARP entries. In our case, we added a static ARP entry on the 7600 that points our NATted IP to the MAC address of the firewall. Now, when you ping the IP, the 7600 doesn’t ARP; it already has the MAC in the ARP table, so it just sends the packet on. Since we only have one static translation in this case, it’s no big deal, but, if we had a whole class-C of addresses to NAT, there would be a management problem.
A part of me wants to do the simple thing and enable proxy ARP, but the vast majority of article, blogs, forums, lists, etc., that I’ve ready say to turn it off for security and efficiency purposes. The more I think about it, though, the more Iwonder why proxy ARP needs to be enabled to make staic NATs work. I looked back at an old PIX running 6.x, and proxy ARP is on by default. The same holds true for an FWSM running 2.x. I’m going to have to ask Cisco what’s up with that.
Send any misconfigured subnet masks questions my way.
https://masto.ai/@aconaway
- Generating Network Diagrams from Netbox with Pynetbox - August 23, 2023
- Out-of-band Management – Useful Beyond Catastrophe - July 13, 2023
- Overlay Management - July 12, 2023
Hi,
Good to know about the ARP Reply issue with the ASA. What ASA model and code is running on the appliance?
So what type of Static NAT policy did you have on the ASA? Sounds like a Static NAT that covers an entire subnet (1-1, host-host).
I do not seem to get the network topology from your description above. Can you email me one? I would like to see the VLAN layout to understand the traffic flow from the production server to/fro accounting server (no ip addresses please…:-)).
Thanks,
netninja11
Hi, NN11:
This particular instance was a 5540 running 8.2(1). The NAT was a 1-to-1 host NAT, so all traffic to/from the billing server is NATted.
I’d rather not share too much information on the topology, but it’s quite simple. The billing network is on an interface with security level of 0. The production network is off an interface with security level of 50. In this case, the productions servers were another layer-3 hop away, but the same would hold true if it were directly attached to the network with the production interface. So, simplistically, you have
<- Static NAT -> Billing -- 5540 -- ProductionI hope that clears it up for you. If you have questions on using VLANs on the 7600, I can see what I can drag up for you.
I ran across this post while researching the same issue as you describe, however in this case proxy arp is turned on. The ASA is running the latest 8.3 code, so I suspect a bug.
On another note, the Cisco web site indicates that proxy arp is *on* by default on the ASA’s. Do you have a Cisco source that says it isn’t?
~~~~~~~~
To disable proxy ARP for NAT global addresses or VPN client addresses on an interface, use the sysopt noproxyarp command in global configuration mode. To reenable proxy ARP, use the no form of this command.
sysopt noproxyarp interface_name
no sysopt noproxyarp interface_name
Syntax Description
interface_name
The interface name for which you want to disable proxy ARP.
Defaults
Proxy ARP is enabled by default.
~~~~~~~~~~
Thanks,
Pulsarav
According to what you submitted and what I found, proxy ARP is indeed enabled by default (no sysopt proxyarp X), but it surely wasn’t on our 5540. I can’t find any other documentation that says it should be that way, though. I’ve got no answer. 🙁
@Pulsarav – I think Cisco bug CSCti38867 is what you’re looking for. I believe I just tripped over that myself.
Interestingly enough I just did an upgrade from 8.2 to 8.4 and got hosed by this, but in the opposite way.
Because it’s a small site, the two main purposes of ASA was internet firewall and routing between two VLANs.
Keeping in mind that the only uplink through the ASA was at the core switch (no routing on it – don’t ask me, just part of the nightmare I walked into), I somehow had stuff on Subnet A showing the ASA’s MAC as the MAC for about half the equipment on that subnet. I was able to ping those addresses of course, because it was actually the ASA responding, but nothing else would communicate with those hosts correctly.
Eventually the Google led me to a post on Proxy Arp and how to disable it. With no better ideas I just did it and hoped.
Solved my problem though.
I’m guessing the default went from proxy ARP being off to on at the big 8.3 switch. Thanks, Cisco!
Default Settings
•(Routed mode) The default real and mapped interface is Any, which applies the rule to all interfaces.
•(8.3(1), 8.3(2), and 8.4(1)) The default behavior for identity NAT has proxy ARP disabled. You cannot configure this setting. (8.4(2) and later) The default behavior for identity NAT has proxy ARP enabled, matching other static NAT rules. You can disable proxy ARP if desired. See the “Routing NAT Packets” section for more information.
•If you specify an optional interface, then the ASA uses the NAT configuration to determine the egress interface. (8.3(1) through 8.4(1)) The only exception is for identity NAT, which always uses a route lookup, regardless of the NAT configuration. (8.4(2) and later) For identity NAT, the default behavior is to use the NAT configuration, but you have the option to always use a route lookup instead. See the “Routing NAT Packets” section for more information.
Take car about the new command in 8.4.5+
no arp permit-nonconnected by default. It sucks me with a particular NAT of an address that was not part of the subnet interface.