Me and the Terrible, Horrible, No Good, Very Bad Weekend

After a wonderful week at this year’s OpenNMS Dev-Jam, I came home on Saturday to find some problems.

On Friday there had been a pretty bad storm, and the farm lost power for over 12 hours. Earlier on Saturday Andrea had called me when I was packing:

Andrea: How many times do you have to pull on the generator cord to get it to start?
Me: Usually about ten times or so.

Andrea (calling me back): Thirty-five!

So, okay, perhaps I should run the generator more often. Of course, as soon as she got it going Duke Energy called to tell her the power was back on.

Anyway, due to the power outage I had to turn on a few machines when I returned. Everything is on APC UPS systems, but they only last 20 minutes or so. We have a room we use as an office that we call “The Study”, and I run a CAT6 cable from there to the living room. The living room is in the center of the house, so that’s where the DSL modem and the wireless router live, but for some reason I couldn’t get from the Study to the Internet. Also, I found out that my new Synology NAS wouldn’t even power on. No real worries, as I live by the rule that “RAID is not Backup“, but still annoying.

From what I can deduce, I think lightning hit near the house and generated enough of a spike that it was picked up by the CAT6 cable and the port on the router was fried. In fact, the port next to it was fried as well. When I moved the cable to the third port, the link light came back on.

However, that didn’t really fix the problem. Now that I could connect to the router, I could tell that it wasn’t behaving all that well. There was a lot of packet loss, etc. So, add a fried ASUS RT-N66U to the list along with the Synology.

I ordered a replacement from Amazon, but since I didn’t want to live without Internet, on Sunday morning I went to the office and borrowed an Airport Extreme we used to use for trade shows. It wasn’t a drop in replacement. First, this Airport only had three LAN ports (the Asus has four) and Apple’s software limits what network address ranges you can use. Of course, my network is different from their defaults, so I had to go around and renumber everything. I added in a small switch to make up for the missing port.

And it still didn’t work.

I wasn’t getting an IP address from Centurylink, so I called and sure enough, there was an outage (caused by the same storm).

Now, this year after Dev-Jam, several people followed us home: one Italian (Antonio) and four Germans. Antonio came in on Saturday and the Germans were landing Sunday evening. By this time on Sunday I needed to go pick up Antonio for the afternoon, and then we were going to go to the airport to pick up everyone else. So I had to leave the problem of the Internet connection for later.

The reason I got chauffeur duty was that I am the main driver of the new UlfMobile. Our company, OpenNMS, has grown to the point that we have a lot of people visiting from out of town and we need to rent a car for them. The frequency of these visits was costing us enough money that it made better sense to lease a company car. We got a Toyota Highlander, which can seat seven in a pinch as well as haul around gear for conferences, etc. Not my first choice in vehicle (I like small cars, preferably without tops) but now we had something that visitors could drive and instead of the hassle of renting a car (that more times than not shows up nasty) they can drive a nice, clean vehicle big enough to haul a bunch of people around.

Anyway, Andrea and I picked up Antonio, we had lunch, and then decided to visit the North Carolina Botanical Gardens. These are beautiful gardens in Chapel Hill and since it was a nice day it was a wonderful time to visit. As we were leaving, we noticed that the UNC police were in the parking lot. Apparently, someone smashed a window of a car to steal a phone. Andrea looked at me and said “Uh, I left my purse in the car”.

As I looked toward the UlfMobile, I could tell that the passenger side window was smashed, and was certain her purse was no longer in the car.

(sigh)

In all, the thieves hit seven cars in that fairly busy parking lot.

This set off a number of tasks. First, Andrea had to go and talk with the police officer. I had to call David to make other arrangements to pick up the Germans. And then there were all of the bank/credit card accounts to close.

The thief or thieves got her purse, which contained her wallet, ID, credit cards, ATM card, checkbook, house keys, car key, truck key, work key and ID. It also had a Nexus 5 that we were using as a media player (it didn’t have an active SIM card).

I called Citi to cancel the credit card and Bank of America to cancel the ATM card as well as close our checking account. Her work ID didn’t identify the exact place where she worked, so it was doubtful they could use the ID to get in or know where her car would be, but still I called to get it canceled. I then called my Nationwide insurance guy (if you are ever looking for an insurance guy in the RTP area, call Jody Shover) on his mobile phone and he promised to get that process started.

All in all it took about two hours before we were told we could leave. Then came the process of cleaning up the broken glass to the point where we could drive the car back to Pittsboro and return Antonio to his Bed and Breakfast.

I told Antonio, “Welcome to America”.

Of course, that didn’t end the evening. My Internet connection, which I could have really used to deal with all of this, was still down. I called Centurylink back, and when there was no “there is an outage in your area” message, I waited until I was connected to a support representative.

By this time I had my laptop connected directly to the DSL modem. It looked like the DSL circuit itself was fine, but it wasn’t passing data. I told this to Lei, the lady at Centurylink, and her response was “No, your Internet is okay, the light is green”.

(sigh)

Now I try very hard to be nice to people like Lei. But as the conversation progressed, she obviously wasn’t equipped to handle the issue, and to be frank I was pretty much at the end of what I could deal with at that moment. I started getting more and more angry, because, damn it, I just really needed to get one thing fixed before going to bed.

I finally just asked to be transferred to second level support. To my astonishment, she didn’t resist, and I found myself talking to Joseph.

Many years ago, when I worked at Northern Telecom, they had us go through a two day race relations course created by C.T. Vivian. It was one of the best things Northern ever did for me. It was intense. At one point we discussed the idea of “talking black”. It’s a stereotype that all black people talk a certain way and it can be harmful.

Anyway, Joseph immediately struck me as someone who “talked geek”. When dealing with technical support people, I often omit the fact that I run Linux and other open source software on pretty much every device I own. It’s just easier to lie. With Joseph, I just laid it all out. Look, I said, I have a Zyxel 660R that I’ve set up in bridge mode to talk to my router. I’m in the web interface and it looks like the DSL circuit is fine but I’m not getting an IP address.

He immediately went to work. The first issue was that it was provisioned wrong, and he adjusted the numbers which was immediately reflected in the modem’s webUI. Still no incoming packets, however. We tried a number of things but nothing seemed to work. I can remember at one point when I was resetting the router, he asked me to check the VCI/VPI values when I could get back into the interface. I told him “8 and 35” without waiting (I had seen it earlier) which turned out to be correct.

We narrowed it down to an issue with the cross connects in the DSLAM, but he couldn’t fix the issue remotely. About that time he gets excited and tells me that an outage has been registered on the device (apparently three people have to complain). They have people 24×7 who can work on things and he promised that my service should be better soon.

Centurylink needs to find Joseph and give him a raise. It makes me wish this XKCD comic were true.

So, I put everything back, brushed my teeth and by the time I got into bed the service had been restored. Yay! One thing off the list.

And it was a very long list.

I didn’t sleep well Sunday night. Knowing someone has your address and house keys is nerve-wracking. Even though I have a security system and two Dobermans, I knew I wouldn’t get a good night’s sleep until the locks were changed.

On Monday I didn’t go to work since I had to go with Andrea to the bank to open up a new account. That afternoon I worked from home as the NC DMV for being able to request a duplicate license on-line. It hasn’t shown up yet but it certainly beats a trip to the nearest office.

I did keep checking on Google to see if the Nexus 5 would ever show up with a valid location. It hasn’t, which I didn’t expect since there wasn’t a SIM card in it. What does frustrate me is that I do have the IMEI (358239-05-036647-7 if anyone is interested) but that doesn’t seem to matter. In the US that phone will only work on AT&T or T-mobile. The moment it joins the network they will know, and they should be able to associate it with a person. It doesn’t look like they do that, though. I don’t blame them entirely (they sell data services and are not police) but still – the technology exists to locate that phone.

A slew of other, small problems also arose but this post is long enough so I won’t go into detail. Weather delayed our replacement credit cards as well as my router. I run Tomato by Shibby and I was frustrated to find out that the configuration backup and restore doesn’t work across different modems (even those of the same make and model). Apparently the NVRAM settings are too specific. I tried to cherry pick a few of them but obviously didn’t get all of the right ones, so I ended up manually configuring the router. The hardest part was re-figuring out my Hurricane Electric IPv6 tunnel, but overall the process wasn’t too bad.

So, most of my home network is back in business. The Synology box is on its way to Washington State for repair, and we still have a few loose ends (mainly dealing with automatic drafts) to clean up, but things are getting back to normal.

Well, somewhat. I’ve noticed a change in my attitude toward certain things, such as surveillance. The Botanical Gardens are a public place, and had they put up a few surveillance cameras there is a chance that seven cars would have been unmolested. This isn’t a certainty (the UlfMobile was parked next to a metal fence with a clear view from a very busy street) but at a minimum it would have provided a few more clues. I’m still not decided about this, but had there been cameras I know I would have been grateful.

I’m certain that the people who did this will be caught. The theft was too brazen to be the work of cautious thieves, although I don’t expect to get anything back. I know that we’ll be more cautious in the future.

On the upside, I guess it can’t hurt to periodically reset your life. I know exactly where all the keys to my house are located. We have new credit card and bank account numbers which have yet to be used. I just wish I’d had more say in the matter.