I spent Thursday and Friday last week on an internal introductory course covering the ins and outs of the Norwegian Broadcasting Corporation. Like all courses some of the lectures were interesting while some were… less interesting. I particularly enjoyed the tours of the TV and radio buildings and the main control room for all the TV channels. With its 3500 employees, NRK is a fairly large company on a Norwegian scale, but if you look at similar international companies it’s a drop in the media ocean: As an organization, NRK likes to compare itself to the BBC, which has so many people employed that the number is not even available anywhere on the internet.

One of the most interesting things I learned, however, was not about NRK per se, but about a project that’s currently in development. I’m not sure if the project details are publicly available information or not and I vaguely remember something about a non disclosure agreement in my contract, so to avoid getting whipped I’m not going to go into any details here. What you need to know that it’s a very complicated computer system handling huge amounts of data - and the data is not being backed up externally.

Say what!?

It’s interesting to see that even if a company looks very professional from the outside, you keep stumbling across things like this when you’re allowed to venture inside the corporate box. Of course, this is not unique to my employer, I’ve heard similar stories from other people and I’ve also experienced the same with other companies. I’m sure you have experienced the same yourself. But seriously, not making external backups of the data stored by the computer system in question - or any computer system for that matter - is complete and utter madness. Even if you make physical backups of all the data it doesn’t matter when both the original and the backup is located at the same room.

When the server room melts down - and it will - you’re fucked. As simple as that. Let me give you a very relevant and recent real life example:

The T-Mobile Sidekick is a mobile phone and communications device that uses a computer cloud to store the user’s personal data. A computer cloud in this setting is a collection of servers hosted by a Microsoft-owned subsidiary called Danger. Now, according to TechCrunch, the Sidekick users have been suffering from a major outage all week. In a statement released by T-Mobile yesterday, they report the following:

[…]

Regrettably, based on Microsoft/Danger’s latest recovery assessment of their systems, we must now inform you that personal information stored on your device – such as contacts, calendar entries, to-do lists or photos – that is no longer on your Sidekick almost certainly has been lost as a result of a server failure at Microsoft/Danger. That said, our teams continue to work around-the-clock in hopes of discovering some way to recover this information. However, the likelihood of a successful outcome is extremely low.

[…]

Can I get a whoops-a-daisy?