Archive for the ‘vlan’ tag
Migrating CSM Serverfarms to Other Server VLANs
A coworker brought an interesting problem to me the other day. He wanted to move a serverfarm from one server VLAN to another without taking an outage. Since I didn’t want to have to come into the office late at night to do work, I decided to see what we could do.
It turned out to be pretty easy. We tend to think of CSM VLANs as pairs — you have the client VLAN for the web servers where the vserver sits and the server VLAN where the serverfarm sits. The CSM doesn’t know about these relationships; all it cares about is whether the servers are in a server VLAN, and we can use that to our advantage here.
Here’s a snippet of what the original config looked like (not really since I’m not telling you how my company’s network is set up). The original serverfarm included a serverfarm called SFARM-ORIG that included 192.168.0.10[12]. That farm is used by the vserver VSERV-ORIG that listens to 1.1.1.1 on HTTP. The probe is in there, too.
probe HTTP tcp port 80 ! serverfarm SFARM-ORIG real 192.168.0.101 inservice real 192.168.0.102 inservice probe HTTP ! vserver VSERV-ORIG virtual 1.1.1.1 tcp http vlan 100 serverfarm SFARM-ORIG inservice
To make the move, we start by creating a new vserver and serverfarm that contains all the IPs invovled — both the original IPs that are already in service as well as the new IPs to which the servers will migrate. The new vserver listens for 2.2.2.2. In this case, we’re moving the servers to 10.10.1.10[12].
serverfarm SFARM-NEW real 192.168.0.101 inservice real 192.168.0.102 inservice real 10.10.1.101 inservice real 10.10.1.102 inservice probe HTTP ! vserver VSERV-NEW virtual 2.2.2.2 tcp http vlan 200 serverfarm SFARM-NEW inservice
When you first drop in the config, the original RIPs should come up as operational, and the new ones should fail since they don’t exist yet (duh!). When everyone’s ready, you then move the service over to the new VIP and run off of that for a while to make sure everything’s working as expected. When all the parties involved are happy, you can then start moving over the servers one at a time. The probe should fail out a server pretty quickly, then, when the server is reconfigured and put on the right VLAN, the CSM should eventually see the new RIP come up and put it back in the available server pool for the farm.
Configured like that, you can move the servers whenever you would like, and the CSM will automatically detect the changes and act accordingly. You just have to remember to remove the old IPs out of the serverfarm when a server moves.
Send any alternative study techniques questions my way.
An Interesting Problem with Multiple DCs on a Stick
We talked about running multiple data centers on a stick back in August, which is where you have multiple logical pairs of client and server VLANs on a single CSM for different tiers or functions. The big point of the article was that you had to do some fancy forwarding to get a server-initiated connection from one server VLAN to appear out the appropriate client VLAN. Well, we ran into an interesting issue with the given solution.
Let’s set up a scenario. Check the diagram for an overview. We have many pairs of client and server VLANs each with a firewall interface as the gateway into the DCOAS. Let’s just focus on just one, though — client VLAN 101 and server VLAN 102. In VLAN 101 is ServerA (not pictured) with an IP of 1.1.101.45; in VLAN 102 is our web farm that needs to connect to ServerA to drop off some data. We add a static route on ServerA pointing traffic for 1.1.102.0/24 back through the CSM.
When you try to connect from the web farm, though, it just times out. Why is that?
Remember that weird forwarding vserver that we had to use to get traffic to come out of the right client VLAN? Well, that’s stabbing you in the eye right now. When the web server initiates a connection, it sends traffic to the server VLAN IP of the CSM. The forwarding vserver grabs the new connection and load balances it to its only RIP, which is the IP of the firewall. What happens when any good firewall accepts traffic destined on an interface destined for a host out of the same interface? It drops the packet, and, eventually, the server times out.
What’s the fix, then? There are a few that come to mind. The first may be to just move ServerA to another network segment. Another may be to change the process around a bit by having ServerA pull the data instead of it being pushed since client-initiated connections will work like a champ.
A really outrageous one would be to set up another forwarding vserver that has only ServerA as it’s serverfarm. You would then add a static route in the web servers pointing to ServerA through that VIP, which would foward it over.
On the CSM, you’d do something like this.
serverfarm SERVERA-SF no nat server no nat client real 1.1.101.45 <--- ServerA inservice vserver SERVERA-VS virtual 1.1.102.5 any vlan 102 serverfarm SERVERA-SF inservice
On the server, you would add a static route to ServerA through 1.1.102.5. If you’re using some brand of Linux, you’d do this.
route add 1.1.101.45 gw 1.1.102.5
Don’t forget the static route on ServerA.
Send any Peeps questions my way.
RSPANs on Cisco Switches
We discussed SPANs earlier, but let’s talk about RSPANs for a bit.
Can anyone guess what the “R” means? You guessed it — “Remote”. An RSPAN is a way to get traffic from a SPAN source on one switch to a SPAN destination on another switch that’s connected via a trunk.
The basic premise is that a special VLAN is created on all the switches and allowed to traverse the trunks. You then set up a SPAN session that copies your traffic to this special VLAN. This VLAN then gets the traffic to the other switches through some voodoo magic to be used as source for a SPAN on another switch.
Let’s work through the steps. In our example, we want to copy traffic from G2/18 on SwitchA to G3/38 on SwitchB.
First, on both switches, we need to create the new RSPAN VLAN. We’ll assume you’ve already got it set up to allow this VLAN over your trunks.
vlan 2000 name RSPAN remote-span
Notice the nice keyword remote-span. This designates the VLAN to be used in an RSPAN. Easy so far.
Now, let’s create the session to copy traffic to the RSPAN. The source port is G2/18, and the destination is the RSPAN VLAN.
switchA(config)# monitor session 1 source interface Gi 2/18 switchA(config)# monitor session 1 destination remote vlan 2000
Now the traffic is being copied to the RSPAN, so let’s copy that traffic from the RSPAN to the sniffer. In this case, the source is the RSPAN, and the destination is the sniffer’s port. Let’s use session 8 to avoid confusion.
switchB(config)# monitor session 8 source remote vlan 2000 switchB(config)# monitor session 8 destination interface Gi 3/38
There are always things to look out for, aren’t there? The first that comes to mind is the fact that you’re copying traffic from a port onto one or more trunks. If the port is sending enough traffic and your trunk is close to capacity, you may wind up crushing the trunk link. That would suck.
If you have a fully-meshed switch environment, you’ll see the additional traffic across all your trunks if you’re set up that way. If you have four trunks that transport all VLANs, you may have four copies of the data coming out of the switch. Let’s say the box being monitored is compromised and sending out 600Mbps of data. That means that every switch will have to deal with that much traffic. This sounds to me like a CPU/memory issue waiting to happen.
Don’t expect RSPANs to work on your 2950 like this. On the lower-end switches, you have to use a reflector port to copy the traffic to the RSPAN. I don’t get into that here, but Google is your friend.
Send any Cadbury Creme Eggs questions my way.
VLANs on Linux
My home network has a Linux box running IPTables as it’s center point, and, since there are four networks, it has 4 NICs and 4 cables into the switch. I kept running into problems with the NICs (they would reorder depending on what flavor of Linux was installed), so I wanted to consolidate the NICs down to 2 — one for the Internet link and one for the LAN segments with 802.1q tagging.
Disclosure: I have only labbed this stuff out, and it seems to work, but I have yet to implement it. Use at your own risk in the wild.
Configuring VLAN tagging on Linux is pretty simple, actually. One way to do it is to use the vconfig command to add and remove VLANs from interfaces. As a demonstration, say you want to run VLANs 20 and 30 on eth0. You would just do something like this. Note that the interface you mention here has to be in an UP state, so do an ifconfig eth0 up if you need to get it into a good state.
vconfig add eth0 20
vconfig add eth0 30
Now, whenever eth0 comes up, you’ll have the interfaces eth0.20 and eth0.30. You can give them IP addresses through the command line with ifconfig.
ifconfig eth0.20 192.168.20.1 netmask 255.255.255.0 up
ifconfig eth0.30 192.168.30.1 netmask 255.255.255.0 up
I didn’t expect them to do so, but the IP addresses actually stay across network restarts; as long on the physical interface comes up, the VLANs come up with IPs and everything. Speaking of network restarts, the “downfall” with using vconfig is that VLAN interfaces don’t show as coming up or going down during network restarts; I don’t like that at all.
Another way to configure the VLANs is through the old-fashioned network-scripts directory. Copy your interface config (ifcfg-eth0) to the same name but with the VLAN extension (ifcfg-eth0.20) and edit it. Change the device field appropriately along with the IP address subnet mask info. For the final touch, at the end of the file, add this line.
VLAN=yes
Personally, this is the way I would do it. It lets you change configurations through the configuration files just like physical interfaces instead of trusting the configuration that resides out in the ether that is the Linux kernel. Also, when you restart network, the interface itself actually goes up and down, so you can see what’s going on with it. If you need some help with this, check out Redhat’s manual on it. Let me know if you’re still having problems with it.
Remember to set the box’s switch port to a matching 802.1q trunk. You’ve seen that before, but here’s a refresher, assuming the Linux box is plugged into f0/1 on the switch.
int f0/1
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
To check the status of your VLANs, look in /proc/net/vlan. You’ll see the config file, which lists all your VLANs. You’ll also see a device file (like eth0.20) with the statistics for that VLAN device (interface).
Let’s talk security, though. First of all, I could argue that a Linux box shouldn’t be participating in any trunking at all. There will be exceptions, but, in my experience, a Linux box (read: server) should only be on one network at a time and shouldn’t straddle networks. Do you really trust the Linux guys to keep their boxes from doing bad things on more than one network? I don’t. Heh.
If, however, you need to use VLANs on a Linux box, you’ll need to make sure you have only the proper VLANs running across this port (like we did with the CSM VLAN). If a box were to be compromised, the bad guy could simply start adding VLANs to the server and suddenly get around your routers and firewalls. Awesome, right? Make sure you put in the switchport trunked allowed vlan x directive so the server only has access to those VLANs.
As always, send me any four-leaf clovers questions you have.
P.S.: For the record, since I haven’t tried this in the field yet, I can’t tell you how well it works with IPTables, but, from what I’ve been reading, it should work fine. Good luck.
For the record, I’ve got this working at my house connected to a Cisco Catalyst 2950 trunk port. I’m happy to report that it works like a champ with IPTables and everything.
VTP and You
VLAN Trunk Protocol (VTP) is a little gem on Cisco switches that allows you configure VLANs in one place and have them appear on all of your switches. This is great for large enterprises with 8457839 switches all trunked together because who wants to configure the new VLAN for that one-off application on all 8457839 switches?
VTP works by having designated VTP servers (not real servers like your Linux box, but a switch) tell the rest of the switches in the network with what VLANs they should be configured. All the designated VTP clients say “OK” and configure themselves with those VLANs. When you take a VLAN out of the server, all the clients take it out; when you add a new VLAN, all the clients add it as well. The server and client designation is known as the VTP mode, and there’s one more to mention. When a switch is in VTP transparent mode, he will see VTP from the servers but will ignore them and pass them on to the next switch as if nothing ever happened.
To configure VTP, you have to define the VTP domain, which is a name for a group of switches that exchange VTP information. In general, in an enterprise, all your switches will probably be in the same VTP domain, but there are situations where you have multiple VTP domains on a LAN (like production and test networks). Next, you set the mode on the switch — server, client, transparent. I would suggest you configure a password as well to at least appear as though you’re practicing good security practices (I’m not sure it’s actually secure or not since every switch has the same password and the hash is sent in every packet…somebody enlighten me, please).
Get on your configuring gloves while I give you some more detail. VTP only works over 802.1q or ISL trunks. Also, VTP packets are Ethernet frames with the VTP stuff shoved into the data section. A switch can only be in one VTP domain at a time and can only be in one mode at a time. You can have more than one VTP server for a domain if you’d like (via a revisioning mechnism where the latest update always wins).
Here’s your scenario. You have SwitchA’s port G1/1 plugged into SwitchB’s G2/1. You want SwitchA to be the VTP server and SwitchB to be the client. Let’s get the config in place.
SwitchA(config)#interface G1/1
SwitchA(config-if)#switchport trunk encapsulation dot1q
SwitchA(config-if)#switchport mode trunk
SwitchA(config-if)#exit
SwitchA(config)#vtp mode server
SwitchA(config)#vtp domain VTP-DOMAIN
SwitchA(config)#vtp password VTP.PASSSwitchB(config)#interface G2/1
SwitchB(config-if)#switchport trunk encapsulation dot1q
SwitchB(config-if)#switchport mode trunk
SwitchB(config-if)#exit
SwitchB(config)#vtp mode client
SwitchB(config)#vtp domain VTP-DOMAIN
SwitchB(config)#vtp password VTP.PASS
Both switches are now in VTP domain “VTP-DOMAIN” and share the password “VTP.PASS”. When you add a VLAN to SwitchA, it automatically gets added to SwitchB. When you add a VLAN to SwitchB, it actually errors out and says something like “Dude, I’m a VTP client. You can’t update me.”
At work, we have a large number of ports but a small number of switches. We also make very few changes to VLANs, so the work needed to maintain the VLANs is very low. Because of that, we actually do use transparent mode on all the switches. I’m also a bit of a purist, and configuring VLANs is something I like to do by hand; in my opinion, some things, especially mechanisms that can take whole networks down, are better left to the human. That’s just me, though.
Setting Up VLANs on an ASA 5505
I’ve had my ASA 5505 in place at home on my Comcast cable for a few weeks now, and, let me tell you, this thing rocks. I did, however, have a few problems finding a clear answer on how I could set up my VLANs. It turns out that the base license on the ASA 5505 comes with a few restrictions with regards to VLANning — in particular the number of VLANs and the number of trunks.
When you have the base license and the ASA is in routed mode (you have IPs on interfaces), you can have three VLANs configured. One of them, however, has to be configured to not forward to one of the other VLANs. I had to go over that a few times before I got what the doc was saying. Basically, you have two VLANs that are fully functional and one that can only initiate traffic to one of the others. At home, I consolidated my network down to three VLAN — outside (I’net), inside (Users), and a DMZ (Guests). The inside interface can initiate connections to both outside and the DMZ, but the DMZ can only talk to the outside VLAN. This is probably not a very big deal to an average user, but I’m a network guy and will add networks just to say I’ve got one more subnet than you do. :)
I also had some confusion over the number of trunks available on this guy. My Aironet 1231 is set up to tag multiple bridged VLANs to the Ethernet so that I could have multiple SSIDs on it with each in their own VLAN. I did this by making the switch port on the 2950 into a tagged trunk. With the base license on the 5505, you don’t get any trunks. I didn’t find any docs that said you couldn’t, but it’s pretty obvious from the “show version” info.
VLAN Trunk Ports : 0
This really puts a damper on my AP setup. I had to reconfigure it to just use a single, untagged bridged interface with a single SSID. If I wanted to implement the Aironet 1231 the way I had it beforehand, I would have to upgrade my license on the 5505. I’m not paying any more money for this thing, so I’ll have to get one of my lab APs in place for guests. Thank God for eBay.
So, how do you configure this thing? First, let decide on our VLANs — say VLANs 11, 12, and 13. The configuration is just like any PIX 7.X
interface Vlan11
description OUTSIDE
nameif outside
security-level 0
ip address dhcp setroute
!
interface Vlan12
description GUESTS
no forward interface Vlan13
nameif guests
security-level 10
ip address 192.168.13.1 255.255.255.0
!
interface Vlan13
description USER
nameif inside
security-level 100
ip address 192.168.14.1 255.255.255.0
The interface VLANs are what handles the IP addresses. We’ll put specific ports in the VLANs in a minute, so hold tight. The configuration is pretty self-explanatory with VLAN11 being the outside (I’net), VLAN 12 being the DMZ (guests), and VLAN13 being the inside (users). There is, however, the one line that reads no forward interface Vlan13. This is the line that makes dictates which VLAN is a DMZ with respect to the base license and that this VLAN (VLAN12) can’t initiate connections to VLAN13. You also might have noticed the line ip address dhcp setroute. You can read one of the earlier articles on that guy.
The last thing you do is to put ports in VLANs. The ports on the 5505 are labeled Ethernet0/0 – 7, and you treat them just like an IOS switch with the switchport access vlan X command. At home, you would plug your cable modem (or whatever) into one of the interfaces (say, Ethernet0/0) and put that port into VLAN11.
interface Ethernet0/0
switchport access vlan 11
Plug all your stuff in, put the ports in the right VLAN, and enjoy.
Ideas That Seems Good At the Time
When I started in IT, I tried to get my gear as standardized as possible to impress everyone. I worked at it and worked at it until I realized that there were a handful of things that sound good but just won’t work. If you’re just getting started in the field, you may not agree, but come back in 5 years and see how right I am. Heh.
- Assigning switchports to VLANs in chunks just doesn’t work. This seems like a great idea. You can put client A on port 1 through 12 and client B on ports 13 through 24. Then client A winds up with 13 servers, and B only has 3, so your whole scheme is in pieces on the floor. It’s just easier to plug servers into the next available port and forget physically organizing the ports. The switches don’t care if the ports are in order by VLAN. Just keep it simple.
- Color-coding cables only works for a while. Let’s cable web servers with green cables and application boxes with blue cables and the database servers with pink and the mail servers with aubergine. I promise you, though, that you will run out of cables of one color or another and wind up having a database server in green. Then you’ll have something else wrong. It won’t be long before the color standard only applies on paper.
- Labeling switchports by name only works if you buy servers all the time. If you’re in an environment where servers change roles and names, I guarantee you that your ports are mislabeled. The only time that labeling really works is if you’re lucky enough to work for a company with enough money to buy new stuff for every project. I’ve actually resorted to labeling ports with serial numbers instead of names since those won’t change.
- Complicated naming schemes don’t work. They may sound cool, but simpler names are almost always better. Name your router “r1″ or something. Don’t try “rtr001prod1″ or something as ludicrous. I once made up this awesome naming scheme, and it worked until the business took on other projects that didn’t fall into the standard, so I was screwed. Save yourself some problems and keep it simple.