You ever have one of those weeks where everything that can go wrong, does? And even things that can’t go wrong, still do? Last week was that week for me. But more on that later.
I’m finally relaxing with a beer in my home office, Friday afternoon, after said hell-week, when suddenly I notice that my desk phone has mysteriously powered off. Beyond the visual cue of no screen display, and a nagging suspicion that something was still not right in the world, I also heard it when it shut down (The Cisco 7940 models in particular seem to make some noise when turning on and off.) Since this phone is using PoE from a Cisco ASA 5505, I glanced over at the 5505 to see what might be causing all of the unhappiness. Immediately I noticed that something wasn’t right, as the status light was orange for a second, then the whole unit rebooted. At that point the display lights go all green, then amber, then another reboot… ad naseum.
What the @#$!?
Trying to log in via either SSH or ASDM yields no love at all, so I hook up a console cable. There it is: the ASA apparently doesn’t have any OS to boot.
Again with the @#$!?
So, I take the cover off and pull out the *cough* expensive-as-hell *cough* flash card to double-check things on desktop computer. Sure enough, the computer reports that the flash card is not formatted. So I formatted it, reinstalled the OS and license information from–wait for it–the backup I had made recently. At this point I could have used this as another learning experience and re-configured the unit from scratch, just to test myself, but at the time I was trying to get some movie tickets purchased and had just gotten done with a very, very tiring week. Not surprisingly then, I took the easy way out and restored the configuration as well.
After a quick reinstall, the unit came back up and everything is fine.
My main lesson learned here–and I’m wondering if this is something that has happened to other people, or happens frequently with the ASA units–is that the Cisco ASAs seem to occasionally wipe out the installed flash card (cards plural if you have 5510s or bigger.) Either that, or the ridiculously expensive (like $300 or something) Cisco-branded 512mb flash cards are flakey as hell. I don’t tend to go with that last option, however, simply because when flash cards go bad they’re not usually amenable to being re-formatted and working properly again: it’s usually game-over, buy another one.
So, another random problem at least solved in the short-term. I’ll keep everyone posted on whether or not this happens again. I haven’t seen anything indicating that this is a known issue, but admittedly I haven’t really been looking. Auto-magically wiping out the entire boot OS, configuration, licenses, etc. would seem to be a fairly non-useful type of bug to have–even by, say, Microsoft standards… let alone Cisco.
So, has anyone seen this before?