It turns out that Juniper is in talks to buy D-link – one of the big names in home networking.  The idea is that D-link already has some large, medium, and small business offerings, so the catalog is very wide.  The big money, though, is in the consumer market.  My account exec went on to tell me that they’re already working on a new consumer-level product based on the SRX; it will be marketed as a D-link device with the Juniper name on it somewhere.  I can’t wait to see a home router with Junos on it.  Think about running the Pulse client to connect back home.  Awesome!

I also learned that Juniper is going to buy the EasyShare line from Kodak.  Kodak just announced that they have already lost $200 million in the first two months of the year, so they are definitely hurting and looking to get some help after their Chapter 11 bankruptcy filing.  The EasyShare line provides a way for Juniper to get themselves positioned in the video market with the rest of the big hitters.  There is no word on why they want to get into video, but the logical next step would be an enterprise video solution or maybe it’s a building block for another larger product.  We’ll have to see.

I’ll have to get my account exec a few more drinks in him the next time he’s in town.  Maybe he’ll tell me when Juniper is going to have a voice solution.  LOL

Send any similar stories questions to me.

I think most of you know that I started a new job late last year, and I’ve spent my waking hours getting caught up on how the new company works, how everything fits together, and all that jazz.  One of the big reasons that I (and a number of others) were brought in was to fix the biggest problem; the company doesn’t have a real central control over customer-facing technologies.  There’s a group that does central IT for the company (Exchange, SharePoint, Oracle apps, etc.), but there are dozens and dozens of applications out there.  That means there are dozens of “network teams” around the world doing their own thing.

One of those groups gave me a call the other day for some help.  Their stack of old 2950s was having some issues, and they needed my help to figure out what was going on.  Among the symptoms were flapping interfaces on the firewall and – the best of all – every command was greeted with an memory error.  Want to see the config?  Too bad.  Want to see the memory utilization?  Too bad.  How about configure the thing?  Too bad.  The only command that I could actually get to work was a show version, but that’s pretty pointless when trying to troubleshoot issues.

So, what did I find?  Nothing that could help with the problem, but plenty of stuff to fix.  Bascially, the switch has VLAN 1, it’s layer-3 interface, and a single username configured.  Nothing else. The configuration items that I consider to be basic just weren’t there thanks to the group’s network guy being a jack of all trades and master of none.  Does putting every host on VLAN1 work?  Sure it does.  Would you just turn on your switch and not configure anything?  I hope not.  Does someone who does networking part-time thing it’s a problem?  Obviously not.

So, what was missing?  For starters, there was no syslog server configured (or even existed on the network at all).  That’s a problem since the only way that I could see the logs was to reboot the switch and try again.  What did the logs say when I finally rebooted the guy?  Nothing since the buffer is empty, but the logs for the boot messages started with “1h3m” by the time I got back to it.  That means the service timestamps commands for logging were missing.  That lead me to ask what the time was on the box.  Did you guess it was March 1, 1993?  Yeah – no NTP server set, either.  Without these basic configuration items, the odds of doing any troubleshooting are just about zero.  Actually, they are zero in this case.  The basics were missing, and now we have no idea what the real cause of our problems was.

I found a whole mess of other issues, too.  The second switch was connected over an access port.  No password encryption service.  Both switches were unconfigured VTP servers.  Not a single interface description.  My OCD definitely kicked in that day.

I guess I’ll have to work for my paycheck this week.

My boss is over all the network guys in the company, and that includes guys that support different divisions and departments.  He told me he was tired of waking up at 2am every morning to fix a problem the other groups can’t handle, so he’s working to get the junior guys motivated to learn for themselves.  One technique he’s implemented is to force them to get their CCNAs and JNCIAs by June.  Since he made it part of the job description, that means that everyone above the Analysts has to meet those requirements, too.  I made the deadline with plenty of time to spare.

Do you remember the full day off of work I had to take to sit the CCNP exams?  The 2-hour drive to a prison town, lunch, a 2-hour exam, and 2 hours back?  That sucked.  I live in a major metropolitan area now, so my travel time to the nearest testing center is 45 seconds.  I mean, literally 45 seconds.  It’s right across the street from my apartment complex.  Easy walk if it wasn’t so cold.  That’s good, too, because I showed up this morning, and the center didn’t have any power!  Someone plugged in a coffee maker in the break room, and power went out in a whole wing of the building.  Since I always get there early, I was actually able to drive home, wait for them to fix the problem, and still be there at my scheduled time.  Convenient for sure.

I must say that the exam was pretty darn good.  It may, in fact, be the best IT exam I’ve ever taken.  The breadth of material was awesome; it had questions from the absolute basics to some of the stuff I saw on the CCIE R&S written. Since I’ve been doing networking for so many years and have my string of certs, the exam was pretty easy to me, but I’m sure an absolute network newb would find the material’s scope a little overwhelming.  The exam scores very high on the fairness meter, as well.  The questions were clearly written; the exhibits were legible and well-marked.  Best of all, there were no real trick questions.  They asked what they wanted you to answer and provided the answer to you.  There were no assumptions or judgments involved in trying to figure out what was being asked.

Overall, I was very impressed with the exam.  Two thumbs up.  I can only hope the rest of the exam in the track are this good.  I won’t know until after I pass my CCIE lab, though.  :)

Send any certification delays questions my way.

This isn’t hard stuff at all.  I’m sure there are a couple of cool tricks I don’t know yet, but let’s try anyway.  I”m working on an SRX240 here running 11.1 and some change.

Let’s put interfaces ge-0/0/0.0 and lo0.0 in OSPF area 0. If you know the Junos configuration hierarchy, this will be very easy to you. Even if you don’t, you can stare at the config for a little bit and see what we’re doing.

set protocols ospf area 0.0.0.0 interface ge-0/0/0.0
set protocols ospf area 0.0.0.0 interface lo0.0

This is the only OSPF configuration you need, but guess what?  It won’t work.  Since a Junos device is also a firewall, it will drop OSPF packets as they come into the interface; you have to declare that you do indeed want to accept OSPF packets.  You do this by creating a security zone, putting the right interfaces in the right zone, and then enabling OSPF on that zone.

We’ll create a zone called INSIDE for our purposes here.  Note that there are about billion more steps (I counted) to fully configure your security zones, but that’s way beyond our scope here.

set security zones security-zone INSIDE
     interfaces ge-0/0/0.0
set security zones security-zone INSIDE
     interfaces lo0.0
set security zones security-zone INSIDE
     host-inbound-traffic protocols ospf

You can also allow OSPF on specific interfaces like this. These commands will also put those interfaces in the right security zone.

set security zones security-zone INSIDE
     interfaces ge-0/0/0.0 host-inbound-traffic protocols ospf
set security zones security-zone INSIDE
     interfaces lo0.0 host-inbound-traffic protocols ospf

I’m not sure if you need to do this to lo0.0, but it won’t hurt.

Now you can see your OSPF neighbors come up and start exchanging routing information.  That is, of course, assuming you did everything else right.

Send any blog deadlines questions my way.

At the beginning of the year, I posted my goals for 2011.  How did I do?  Not too well.  I batted .500, so feel free to boo me.

• Hurry up and finish CCNA Voice : I finished that on 7 February.  Was it worth it?  Not really.  I haven’t used the knowledge, and voice isn’t my thing.  I got it to spice up the resume, but it didn’t really come into play at all.  Oh, well.  It’ll expire in about 2 years.
• Pass CCIE R&S written exam : I got this one finally.  I flunked out at Cisco Live this year, but I redeemed myself on 23 August with a pass.  Jody still owes me a drink since I hold the record for lowest passing score.
• Select a CCIE training vendor : Yeah…I never got to that one.  When I finally got through the written, my job had completely drained my motivation.  I fixed that problem by getting a new job, but that didn’t help free up any time to figure out which vendor I wanted to use.  #fail
• Schedule CCIE R&S lab : That obviously didn’t work out, either, since it’s dependent on selecting a training vendor.  #fail

What does this hold for this year?  Getting some training and scheduling an exam is obviously priority.  Since my new job is going all Juniper, going through those certifications would be next.  Another super-busy year, I’m sure.

Good luck to everyone in 2012.

We all know that the configuration on a Junos box is very hierarchical. Sometimes it doesn’t make a lot of sense, but it’s all a pretty cascade of code. One of the big messes that I’ve found is the VPN configuration hierarchy; there are way more items to configure than on an IOS device.  To reinforce the stpes in my head, I thought I’d get some of the pieces into a post. These aren’t all the options, but it’s all you need to get a static IPSec tunnel up and running.

That’ll do, pig.  I’ll fire off a real configuration post later.  Feel free to add your pair of pennies since I’m a total Junos n00b.

Send any stocking stuffers questions my way.

One of the big things that I’m dealing with lately is VRFs.  I’ve implemented some VRF-lite stuff, but I’ve never had any practical experience with the full force of them.  I’m definitely learning here.  Since the blog here is really about my sharing what I’ve learned, let’s go through something that came up recently – terminating VPNs on one VRF while passing traffic to another.

What I’m talking about is the old-school, static IPSec VPNs that we’ve all configured a million (or so) times.  You know the ones with crypto maps applied to interfaces?  Well, we’re going to configured one of those for the VRF called “CUSTOMER1″ terminated on an interface in the “INTERNET” VRF.  

There’s some terminology for these VRFs, actually.  The INTERNET VRF, which has the tunnel endpoint is called the front VRF (FVRF); CUSTOMER1 is called the internal VRF (IVRF).  I’ll try to remember to use those terms, but I make no promises.

First, we need to create the VRFs themselves.  Since the endpoints are in two different VRFs, we’ll need to have some routes leaked from the IVRF to the FVRF.  I could write 847829843828 words on route leaking and not cover everything in my limited experience, so you’ll have to look that up on your own if you don’t know what I’m talking about.  Route-target 65000:1 is exported from INTERNET and imported into CUSTOMER1

ip vrf INTERNET
rd 65000:1
route-target export 65000:1
!
ip vrf CUSTOMER1
rd 65000:101
route-target import 65000:1

At this point, we just put the interfaces in the right VRF along with their addresses.  We’ll also configure an ISAKMP policy just like we’ve done a million times.

crypto isakmp policy 100
 encr aes
 authentication pre-share
 group 2
!
interface Ethernet0/0
 ip vrf forwarding INTERNET
 ip address 192.0.2.1 255.255.255.0
!
interface Ethernet0/1.1
 encapsulation dot1Q 1
 ip vrf forwarding CUSTOMER1
 ip address 192.168.201.1 255.255.255.0

Next we’ll create a keyring that’s referenced by the IVRF.  This will make the key for the remote end available for use by that VRF.

crypto keyring KEY1 vrf INTERNET
  pre-shared-key address 192.0.2.101 key TEST.KEY

Now we create and ISAKMP profile, which is really the blood and guts that make all this work.  An ISAKMP profile references some of the important pieces of the tunnel – the IVRF in which to place the traffic, the keyring to use, and tunnel endpoint, and the FVRF where the tunnel terminates.

crypto isakmp profile CUSTOMER1-PROFILE
   vrf CUSTOMER1
   keyring KEY1
   match identity address 192.0.2.101 255.255.255.255 INTERNET

We’ll then create the ACL for interesting traffic. I’ll save some trees and not go through that since this should be pretty easy by now.

Now we can create the crypto map. This will be just like any other crypto map you’ve ever made with one exception; this is where you include that nifty ISAKMP profile we just made.

crypto map CM 100 ipsec-isakmp
 set peer 192.0.2.101
 set transform-set TS
 set isakmp-profile CUSTOMER1-PROFILE
 match address CUSTOMER1-TRAFFIC

Just like in other cases, we need to add a static route to make sure the router sends the packets destined for the remote end of the tunnel out the right interface. Since the FVPN is INTERNET, we’ll add static routes for that VRF. We’ll do the same for the tunnel endpoint just in case the default routes doesn’t go the right way.

ip route vrf INTERNET 192.0.2.101 255.255.255.0 192.0.2.2
ip route vrf INTERNET 10.0.0.0 255.255.255.0 192.0.2.2

Now the tunnel should be up, right? Probably not. If you take a close look, you’ll see that the FVRF has the route to the remote network, but the IVRF – the one that will use the tunnel – doesn’t. We’ll need to use MPBGP to leak those routes from one VRF to another. Did I mention that route leaking can get long-winded and that I’m not going to get into it? Yeah…it can get that bad. Just trust me that this works.

What we’re going to do is to start up BGP for both VRFs. At the same time, we’ll redistribute the static routes that we added above from the FVRF into the IVRF. Since we set up our imported and exported route-targets in the VRF definition, the static routes will magically appear in both VRFs.

router bgp 65000
bgp router-id 192.0.2.1
!
address-family ipv4 vrf INTERNET
 redistribute static
 exit-address-family
!
address-family ipv4 vrf CUSTOMER1
 exit-address-family

If we do a show ip route vrf CUSTOMER1, we’ll see the static routes from the INTERNET VRF. They’re real easy to spot. :)

...
B        10.0.0.0 [20/0] via 192.0.2.102 (INTERNET), 00:00:05
...
B        192.0.2.1 [20/0] via 192.0.2.102 (INTERNET), 00:00:05
...

That should do it. Now you should be able to talk from your local network in the CUSTOMER1 VRF and talk through a tunnel that’s established on the INTERNET VRF.

Send any Juniper configs questions my way.

I am really excited about the new position.  Since I haven’t started yet, I’m not going to reveal who the company is, but you’ve all seen the name.  They’re forming a new group to handle specialty services for customers, and I’ll be working for the manager of that team as the Senior Network Engineer.  My future boss is a CCIE, so that’s a great start; we didn’t even have a CCNA at my current company until about 2007.  The job is going to be great, and the wife and I are both up for new adventures.

So, why am I leaving?  I’m going to take the high road here and not whine about all the little issues, but there are several that pushed me to look for a new job.  For one, my current company is primarily in print media – newspapers, magazines, books, etc.  When was the last time that you read a newspaper?  Hell, I haven’t read a newspaper in 13 years and I work for a newspaper company.  Sales of print media have been dropping quickly since this whole Interwebs came about, and revenue from the online versions of media are less than 10% of those from print.  It’s just a matter of time before the whole industry goes away, and I wanted to move on gracefully.  Scrambling for a job in this economic climate wouldn’t be a happy place.

The most pressing reason for leaving the company, though, is the fact that the IT services are in the process of moving to a new joint venture company headed by NIIT Technologies.  While this opens up a much broader world, there is always the question of how long my job will be in place with the new company.  I’ve spent the last 4 weeks meeting with the NIIT guys, and they are absolutely wonderful.  They know their stuff, ask the right questions, and do their homework when learning about the network.  Without a doubt, I’d enjoy working with them, but the my group’s future (and even the future of the whole new company) is no longer a certainty.  Again, it’s time to move on gracefully.

Now we have to move to the big city of Atlanta.  I have a small apartment there to live in while we work to sell the house.  The wife is staying for nowto get the house ready to sale, but I’ll make the 5-hour drive on weekends to help out.  It’ll be a struggle for the next few months until the house is sold, but it’ll definitely be worth it in the end.

I start the new job on Monday, 7 November, so I’ll be making my way up with my TV and blow-up mattress this weekend to start the adventure.  My last day with my current company is actually tomorrow, so I’ll be on the road to headquarters to hand in all my stuff.  Hands will be shaked (?), lunch will be eaten, “do they have any openings” jokes will be told.

Most importantly, though, something bigger and better will begin.

I got a call from one of our business units looking for some routing help.  We don’t usually care about their production networks, but they were seeing some funky traceroutes, so I agreed to try and help them out.

They sent over two fresh traceroutes from a host on a 7600.  In one of them, the trace went to the 7600 and then on down the line as expected.  In the other, the trace showed the 7600, another router’s far interface IP (that is, an interface not facing the 7600), then the 7600′s interface facing that router.  Every few minutes, the path was switch between the two.  The dude told me that they were an OSPF shop, so I asked him to send me the standard show ip route and show ip ospf database commands so I could see what’s going on.  The word “unexpected” comes to mind when trying to describe what I found.  So do other words that aren’t very appropriate.

The 7600, the main router at the main campus, was in OSPF area 50.  The router that showed up in the trace was also in area 50.  The same was true for every other router at that location, so I figured that area 0 was at another location.  Nope.  All routers at all locations (probably around 20 total) were all in area 50, and area 0 was nowhere to be found.  I always thought you could run a single non-backbone OSPF area, but I never understood why you would actually choose to do so.  If you want one area, that’s fine, but why not make it area 0?

That single area was working so I didn’t ask too many questions and looked again at the outputs they sent over.  I chuckled a bit when I noticed that the routes to the target network were showing up as an OSPF type-2 external.  I got a copy of the config at the far network and, lo and behold, I found that there is a single network statement for the transit network back to the main campus along with redistribute connected subnets.  For some reason, instead of actually advertising networks natively in OSPF, all the networks with hosts on them were being redistributed.  I wasn’t there to redesign their network, so I just sighed out loud and kept looking.

I got a copy of the OSPF config for the main campus’s 7600 to see if would show why the traceroute was weirding out on them.  Here’s the part where I actually laughed out loud on the phone.  Right in the middle of the config, I see “area 50 nssa”.  Yes, this single non-backbone area with no real costs being advertised was configured as a not-so-stubby area.  Not only did they go out of their way to make it a non-backbone area but they also wanted it as a stub area.  Since they had all the other networks redistributing into the area, they had to make it NSSA.  It’s a week later, and I still roll my eyes.

How did this happen?  When this business unit was being turned up, they actually outsourced the initial build to a company who will not be named here.  They’re the ones who put in this creative OSPF configuration that I’m putting in my hall of shame (if I had one).  They’re also the ones who caused the reported problem.  After a few more hours of looking around, our guys discovered that the other company put in a new VPN endpoint configured with the IP of the SVI of the 7600.  IP conflicts aren’t good, eh?  Once that was changed, everything returned to normal.

A fun few hours indeed.  At least it was entertaining.

I’ve already used the phrases “skin of my teeth” and “a pass is a pass” on Twitter today for good reason.  Passing is a score of 790, and I blew that away with a 790.  One more lapse in concentration and I would have been making up more excuses instead of smiling.  I think I’ve mentioned this before, but I have this weird reaction to taking exams where I don’t get nervous at all until after I’m finished.  Walking into the testing center, I was fine.  Walking out, I was shaking like Northern Virginia.  It was so bad that I could barely hold on to the door knob when trying to leave, so I guess that I’m really prouder than I thought I was.

The exam was a total piece of crap.  Nearly every diagram was so compressed and blurry that I couldn’t see them at all.  Most of the time I can infer what the diagram is showing, but, when your bridge priorities are listed there, it’s pretty hard to find root ports.  Absolutely horrible.  There were the inevitable spelling errors in there, too.  Most of those are fine, but STP and SPT are two different topics that are both covered on this exam.  I had no problems figuring out which one they were talking about, but it’s pretty unacceptable to have spelling errors in this exam.  Of course, there were also the questions that I feel are unanswerable.  Switches in VTP transparent mode behave differently from version 1 to version 2, eh?

After being recommended at Cisco Live this year, I added the Boson ExSIM-Max to the pile of prep materials.  It not only helped teach a few new things, but it cleared up a bunch of foggy details.  I’m sure that using any other study materials will do the same to some extent, but the Boson stuff provided something else – it helped to teach me to take the exams.  I was able to go through the questions and practice figuring out what was being asked, which choices were completely wrong, and how to not get utterly frustrated with the questions.  Practice makes perfect, right?

The wife came with me to take her ICND1 exam.  She did better than she thought she would, but, alas, no dice this time.  She says that she’s glad she’s been through it now and knows exactly what to study.  I’m trying to convince her to start her own blog since she’s starting up her cert journey from such a unique place.  We’ll see how that works out.

What’s next?  I have to find a company to help me prep for the lab now.  I’m sure that’s not cheap at all.  Maybe I should just blindly sit the lab and see what happens.  Maybe not.  :)