Aaron’s Worthless Words

We all have limited budgets these days. Long gone are the days of unlimited resources and uncontrollable expansion of the network, so it’s important that any network dude or dudette pay attention to the open-source world. Below is a list of stuff I use at the office and at home to monitor, trend, and alert the network. All this stuff is free and runs on Linux to save even more cash.

• Cacti – This is a system for trending pretty much anything. If it has an SNMP value, Cacti can trend it. It’s also really flexible, allowing multiple displays of data and even a mechanism to get values from scripts you write. At the office, we use it to monitor utilization of the circuit and Ethernet ports, CPU and memory of the gear, and the number of connections on the load-balancer. At home, I use it to watch utilization and track the number of connections to the wireless networks.
• Nagios – This is a monitoring and alerting system for all sorts of stuff. It watches hosts and applications for availability and response time, then alerts based on threshold. This is one of the most complicated apps to configure, but, once it’s up, it rocks. I use it at home to monitor all the network gear and systems for response times. I also use it to monitor the web servers and restart them if they’re down.
• Apache – You know what Apache is. You use it already. About 71% of webservers on the Internet are Apache.
• Squid – A caching proxy server by the same guys who do Apache. It can be configured for both inbound and outbound application acceleration. It’s great to put in front of a CMS like Drupal or Joomla. It has a mess of built-in functions that can look for bad requests, do redirects, or completely rewrite requests. At work, it fronts our application and CMS servers so users don’t have access directly to them. At home, it runs on the firewall to serve pages to the Internet. The real webserver actually sits on a box behind the firewall for security.
• Subversion – This is a version control system. Subverions lets you create repositories, check out the contents, edit them, and check them back in. This is good for keeping track of configuration files or scripts you write. We use it at work to track configuration files for Apache, NTP, yum, etc. At home, I use it to keep track of my scripts and Perl module.
• Rancid – This is configuration management for Cisco (and other network) devices. It gets configs from devices and checks them for changes. It’s got built-in alerting and is easy to set up. I use it at home to keep track of the configs on the switches and access points.
• nfsen/nfdump – These are [tag]netflow[/tag] tools. Nfdump is a suite for collecting the data, while nfsen is for displaying the information. Check out netflow if you’ve never worked with it…it’s pretty cool.
• Dyanmips/dynagen – These apps let you run virtual Cisco routers on a machine. You can set up full network deployments for testing and configuration experimentation. It takes a good bit of resources, but it’s well worth it for the functionality. I use it all the time at work to test or tweak configs. I also use it to simulate certification labs.

• (0)

I assume you take every word I say to heart and that you’ve been using Dynamips/Dynagen for a few days now, right? Good. That’s a start, but let’s break down a simple lab to make sure everyone’s on the same page. I run my labs on Linux most of the time, so you’ll see my commands for that platform. You’re a smart one, so you can figure out what to do on Windows.

First of all, everyone download the lab file. This is a very simple lab that I created to do some experimentation with BGP. I modified it a bit to save resources for the general public, though; it’s a lot easier to run 2 2651XMs than 2 7206s, right? Let’s go through the lines.

autostart = False
[localhost]

Yes, that means don’t start up the routers when you fire up dynagen. The second line is the dynamips server you want to use. This will always be localhost unless you’re leveraging another box to use as the emulator. That’s an advanced topic that I’m not going to cover right now, though.

[[2651XM]]
image = /home/jac/labs/images/c2600-adventerprisek9-mz.124-17.img
ram = 96

These lines define the parameters for any 2651Xm that we’ll be using. The image file is the actual IOS image to use. I can’t provide an IOS file for you, so you’ll have to download one yourself and change this line to where you put the file. Guess what the “ram” line is. Wow…you’re a genius if you said it was the amount of RAM to give each 2651XM.

[[ROUTER R1]]
s0/0 = R2 s0/0
f0/0 = LAN 1
model = 2651XM

[[ROUTER R2]]
f0/0 = LAN 2
model = 2651XM

Here’s the meat of the lab. We’re creating 2 2651XMs, called R1 and R2, that each have s0/0 and f0/0 interfaces.

The 2651XM has 2 FastEthernets, so, when you fire up the lab, the routers will have those interfaces, but we don’t care about f0/1 in the scope of this lab. We’ll just ignore it for now. The “LAN” keyword in the f0/0 lines tell how you want the FastEthernets cabled up. We’re trying to do BGP here, so the FastEthernet interfaces are connected to different network (R1 to LAN 1 and R2 to LAN 2).

Notice that, in the R1 configuration, we say that R1’s s0/0 is connected to R2’s s0/0. This lets the routers talk across the serial interfaces in the virtual world. In our lab, this is the link that we’re going to run BGP over to share the paths to the f0/0 interfaces. Very simple setup.

So, let’s fire this guy up. First, make sure you’ve started your dynamips server. I run mine in a screen session to get it out of the way, but it’s your choice. Run “dynamips -H 7200″ to get going and listening on port 7200. When that’s up and running, you simply run dynagen against your lab file with a “dynagen bgp.net”.

If everything is right, you’ll be see the “=>” prompt. Remember that we set autostart to false, so we have to start up our routers. You can type “start R1″ and “start R2″ to get them going, but it can be easier to do a “start /all”. You may run into problems with CPU or memory if you do that, though, so be careful.

If you do a “list” at the dynagen prompt, you should see both of the routers in a “running” state. That’s good, but now what? Time to get on the console. If you’re running your lab on your workstation (the box that’s attached to the keyboard you’re typing on), you can do a “console /all” to bring up all the consoles at once. If you’re doing the lab remotely, you’ll have to telnet to the right port to get a console. That info’s in the last column of the “list” command.

Now comes the configuration, so get to it. Configure these guys to share their routes on the f0/0 interface via BGP. Experiment a little. Experiment a lot. You’re not going to break anything, but remember to save the configuration when you’re done. You can reuse the lab later.

—–

If you’re lazy, you can use the configs that I used for this lab. They’re very, very simple, but they work.

• R1
• R2

• Read comments... (1)

Most firewalls should block [tag]ICMP[/tag] requests to them, so how do you know that your router or server has layer-2 connectivity to one? It’s pretty elementary, actually, but I’ve found that not a lot of people know this trick. If you ping the firewall, it will receive the ICMP packet and drop it per the rulebase. In this process, though, the firewall has to answer [tag]ARP[/tag] requests, which will be stored in the router or server’s ARP table. If you see it in there, you have connectivity.

On [tag]IOS[/tag]:

show arp

On [tag]Linux[/tag]:

/sbin/arp -an

This won’t help you if you’re not on the same network as the firewall, but it’s very helpful — especially if the firewall group is separate from the group you’re in. These commands have saved me a lot of time by not having to get a bunch of people on the phone to sniff packets as I generate them only to find that the firewall isn’t talking to my router.

• Leave a comment... (0)

Ever heard of Samba? You should.

Samba is an open-source project “that provides seamless file and print services to SMB/CIFS clients.” That’s from the project’s website, but what the hell does it mean? In a nutshell, it’s an open-source application that lets non-Windows machines share files and printers with Windows machines. In most cases, people use Samba to share files on a Linux box in a really simple setup. I’ve read about several cases, though, where superhuman admins have used Samba machines to set up a Windows domain. I’m talking full-scale domain login, domain machine registration, and everything. I tried that once and all my Windows machines stopped working. It sucked.

I’m going to be lazy again and not tell you how to configure it. Or am I smart and efficient and saving it for another article? Either way, I wanted to talk about what Samba can get you. Samba lets you provide a way to store files on a network share. If you set up a share for everyone to use, everyone can edit an address spreadsheet or view a home inventory sheet. You can set up Samba to share your home directories to every machine on the network. Everything’s on the same machine. Think about that for a second. If everyone’s files are on the same box, it’s easy as pie to back everything up, and, since everything’s on the network, you can have access to your stuff from everywhere.

I use Samba for file sharing. Home directories for me and the wife are shared out to local drives on our laptops. If I log in to hers and I see my home drive. I also took some time and put in a Highpoint Tech 1740 with a set of 4 300G SATA drives attached to it (hot-swappable and full fault-tolerant, by the way). These guys are put into a RAID5 array to provide about 900G of space to the operating system. I used Logical Volume Manager to split this beast up into a bunch of different volumes — images, music, videos, and a share. Samba shares all these guys out, and our machines have each of these mapped to a drive. When the wife has new images to share, she just drops them on the images drive. When I finish a new video, I drop them in the videos drive.

Samba works well and is easy to set up. Give it a try. It doesn’t really scale very well, though, so using it in an enterprise may cause problems. There’s a whole new article brewing in there about file locking and sharing, but I won’t go there yet.

—-

Remember that I mentioned backing stuff up if everything’s on the same box? It’s not related to Samba technically, but I wrote a quick and dirty bash script that takes a list of directories and tars them all up to a 400G external drive I have attached to the file server. If I didn’t implement Samba, I’d really have no easy way to back it all up in one fell swoop.

• Leave a comment... (0)

A few months ago, a friend of mine told me about the concept of port knocking, where you send packets to a server on certain ports to authenticate access to the box. A daemon running on your server detects the sequence of packets that you send and runs a script (usually IPtables commands), waits a certain amount of time, then runs another script (usually to take the IPtables commands out). This seems like a good way to get access to your home firewall from anywhere without having to open up access to the whole Internet.

To set it up, you have to install knock, which is the daemon that listens to the port knocking. Just use yum or apt-get to install it and you’ll wind up with the configuration file in /etc/knockd.conf. This is where you set up one or more knock sequences to do what you want. I won’t go into the internals of how it works or how you should set it up but I will go into a few examples.

I use port knocking on my home network to protect administrative access to everything on the network. I wrote a custom IPtables script that, when activated, open access from my IP on the wireless network to SSH (TCP/22) on my firewall, file server, access point, and switch. After 30 seconds, another script runs, and those rules are removed. Here’s an example of a config file that opens up SSH when you hit ports 1234, 5678, 9876, and 5432. After 30 seconds, it kills the rule.

[options]
logfile = /var/log/knockd.log

[openssh]
sequence = 1234, 5678, 9876, 5432
seq_timeout = 5
tcpflags = syn
start_command = -A INPUT -s %IP% -d 192.168.1.1 –dport 22 -j ACCEPT
cmd_timeout = 30
stop_command = -D INPUT -s %IP% -d 192.168.1.1 –dport 22 -j ACCEPT

So, how do you generate these packets? On my CentOS boxes, you get the knock command which is the port knocking client. On Windows, I use KnockKnock. I have no clue about Macs, but there are lots and lots of clients out there, so just look around and I’m sure you’ll find one.

• Leave a comment... (0)

You can use an old PC as a firewall at home (and at work, I guess). It’s not that hard to do if you have a basic knowledge of Linux, DHCP, and IPtables, but that may be saying a lot.

Why would anyone want to do this, though? If you’re like me, you like to know what’s going on in the network. One of the Linksys routers you buy at Best Buy or Circuit City just doesn’t let you monitor very well. You can’t get very good logs off of it, so you don’t really know what it’s doing or complaining about. It also doesn’t let you query the interfaces, so you really don’t know how much bandwidth you’re using.  If you have a Linux box as your router/firewall/gateway, you can get really good logs, monitor the interfaces with SNMP, and have some really great, granular control over your network.

To set up a simple network, you only need a machine with two NICs in it and a Linux distro, then follow these steps. This is a simplified procedure, but it should get you started.

1. Install Linux on the box. Make sure you have IPtables and DHCPd on it when you’re done.
2. Cable one NIC to the cable modem (WAN) and the other to your switch (LAN).
3. Hard-code the IP address of the LAN interface to your favorite IP space.
4. Set up DHCPd to serve IPs to your LAN.  Remember to set your default gateway option to the IP of the LAN interface and your DNS servers to your ISP’s servers.
5. Set up IPtables as your firewall. This is not a simple endeavor, but there are several links for help below.
6. Enjoy your newfound infrastructure.

This is how I started out, but my network has grown tremendously in the last year or so. I’ve got four NICs in my gateway to create different network segments, and my IPtables file is about 5k now with 79 rules in the system. The price you pay for being a network dude.

Another thing you may want to do is recycle the wireless router.  If everything’s working in place after you implement the new box, you can turn off DHCP on your wireless router and stick it into the LAN segment.  You can still attach to it with your notebook or whatever, but you’ll get your address from the server just like the wired hosts.

Since you’re running Linux now, you can also look at using some other tools to help monitor everything.

• Nagios — for monitoring the status of the network
• Cacti – for trending and usage graphs
• Syslog – for collecting log files
• Apache – the de facto web server
• Bind – for serving DNS locally
• NTP – for syncing all your machines’ clocks
• tcpdump – for capturing packets coming in and out of the network

Let me know if you have any problems or need any help.
—–
Some IPtables links to help you on your way:

• http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch14_:_Linux_Firewalls_Using_iptables
• http://www.sns.ias.edu/~jns/wp/iptables/
• http://www.yolinux.com/TUTORIALS/LinuxTutorialIptablesNetworkGateway.html
• http://www.faqs.org/docs/iptables/traversingoftables.html

• Leave a comment... (0)

The Principle of Least Privilege says that users or applications should only have access to the what it needs to access and that access should be as limited as possible.  This idea can be applied to any number of things, but it is a very important topic when talking about security.

The idea is that processes, users, modules, or whatever can only access what they need to in order to function.   This keeps users in check since they don’t have any access to anything outside their home directories (or whatever).  It keeps developers in check since their code can only access a small set of files or processes.  It keeps hackers in check since the Apache server they’re hacking can’t access the password file.  It even keeps administrators in check since it forces them to use sudo, which is logged to syslog.

Like everything, this is a simplistic description, but it’s very important for every administrator to adopt this idea.

You may also want to check out SELinux if you’re a Linux admin, which is a system that is used to enforce least privileges for nearly everything on a Linux box.

• Leave a comment... (0)