Posts tagged datacenter
Migrating CSM Serverfarms to Other Server VLANs
Jan 25th
A coworker brought an interesting problem to me the other day. He wanted to move a serverfarm from one server VLAN to another without taking an outage. Since I didn’t want to have to come into the office late at night to do work, I decided to see what we could do.
It turned out to be pretty easy. We tend to think of CSM VLANs as pairs — you have the client VLAN for the web servers where the vserver sits and the server VLAN where the serverfarm sits. The CSM doesn’t know about these relationships; all it cares about is whether the servers are in a server VLAN, and we can use that to our advantage here.
Here’s a snippet of what the original config looked like (not really since I’m not telling you how my company’s network is set up). The original serverfarm included a serverfarm called SFARM-ORIG that included 192.168.0.10[12]. That farm is used by the vserver VSERV-ORIG that listens to 1.1.1.1 on HTTP. The probe is in there, too.
probe HTTP tcp port 80 ! serverfarm SFARM-ORIG real 192.168.0.101 inservice real 192.168.0.102 inservice probe HTTP ! vserver VSERV-ORIG virtual 1.1.1.1 tcp http vlan 100 serverfarm SFARM-ORIG inservice
To make the move, we start by creating a new vserver and serverfarm that contains all the IPs invovled — both the original IPs that are already in service as well as the new IPs to which the servers will migrate. The new vserver listens for 2.2.2.2. In this case, we’re moving the servers to 10.10.1.10[12].
serverfarm SFARM-NEW real 192.168.0.101 inservice real 192.168.0.102 inservice real 10.10.1.101 inservice real 10.10.1.102 inservice probe HTTP ! vserver VSERV-NEW virtual 2.2.2.2 tcp http vlan 200 serverfarm SFARM-NEW inservice
When you first drop in the config, the original RIPs should come up as operational, and the new ones should fail since they don’t exist yet (duh!). When everyone’s ready, you then move the service over to the new VIP and run off of that for a while to make sure everything’s working as expected. When all the parties involved are happy, you can then start moving over the servers one at a time. The probe should fail out a server pretty quickly, then, when the server is reconfigured and put on the right VLAN, the CSM should eventually see the new RIP come up and put it back in the available server pool for the farm.
Configured like that, you can move the servers whenever you would like, and the CSM will automatically detect the changes and act accordingly. You just have to remember to remove the old IPs out of the serverfarm when a server moves.
Send any alternative study techniques questions my way.
An Interesting Problem with Multiple DCs on a Stick
Mar 24th
We talked about running multiple data centers on a stick back in August, which is where you have multiple logical pairs of client and server VLANs on a single CSM for different tiers or functions. The big point of the article was that you had to do some fancy forwarding to get a server-initiated connection from one server VLAN to appear out the appropriate client VLAN. Well, we ran into an interesting issue with the given solution.
Let’s set up a scenario. Check the diagram for an overview. We have many pairs of client and server VLANs each with a firewall interface as the gateway into the DCOAS. Let’s just focus on just one, though — client VLAN 101 and server VLAN 102. In VLAN 101 is ServerA (not pictured) with an IP of 1.1.101.45; in VLAN 102 is our web farm that needs to connect to ServerA to drop off some data. We add a static route on ServerA pointing traffic for 1.1.102.0/24 back through the CSM.
When you try to connect from the web farm, though, it just times out. Why is that?
Remember that weird forwarding vserver that we had to use to get traffic to come out of the right client VLAN? Well, that’s stabbing you in the eye right now. When the web server initiates a connection, it sends traffic to the server VLAN IP of the CSM. The forwarding vserver grabs the new connection and load balances it to its only RIP, which is the IP of the firewall. What happens when any good firewall accepts traffic destined on an interface destined for a host out of the same interface? It drops the packet, and, eventually, the server times out.
What’s the fix, then? There are a few that come to mind. The first may be to just move ServerA to another network segment. Another may be to change the process around a bit by having ServerA pull the data instead of it being pushed since client-initiated connections will work like a champ.
A really outrageous one would be to set up another forwarding vserver that has only ServerA as it’s serverfarm. You would then add a static route in the web servers pointing to ServerA through that VIP, which would foward it over.
On the CSM, you’d do something like this.
serverfarm SERVERA-SF no nat server no nat client real 1.1.101.45 <--- ServerA inservice vserver SERVERA-VS virtual 1.1.102.5 any vlan 102 serverfarm SERVERA-SF inservice
On the server, you would add a static route to ServerA through 1.1.102.5. If you’re using some brand of Linux, you’d do this.
route add 1.1.101.45 gw 1.1.102.5
Don’t forget the static route on ServerA.
Send any Peeps questions my way.
