05:46 AM

Dirty Little IT Secrets: Our Privacy Is Protected By Massive, Inefficient Corporate IT Systems

There has been lots of chatter about data privacy but fortunately, the ineptitude of IT systems is a protective factor. Take a look at this amusing tale from my ZDNet colleague Dennis Howlett.

Dennis is an Englishman living in Spain. He tells of trying to make changes to his account at Vodafone Spain. Vodafone Spain saps my will to live

Each time you call, it seems necessary to go through a grinding series of questions that include the number you are calling from (they don't have caller ID?), providing your passport or residency number, your password and bank name. Then you have to slowly and carefully explain the problem. At least twice.

... Vodafone claimed they have no record of my landline number or that I have requested that service. Duh? I have SMS messages telling me they activated the service on my mobile. I"m staring at them. 'We have no record.'

...Knowing what it is like dealing with anything that smacks of government or utilities, I pack every piece of paper I can think that will successfully identify me as the person I say I am. At the Vodafone shop I get the same answer. No record.

Thinking laterally, I ask if there is anything registered to the new DSL service and oh by the way, can you search against THAT phone number? Bingo! Up pops the details Vodafone said it didn't have.

It turns out that when I acquired my Vodafone mobile, I gave my passport number. When I applied for DSL etc, I gave my NIE (resident's) number because that is what it asked for. Vodafone doesn't match these and cannot alter what it now sees as the master identifying record locator. So now it seems I have two accounts.

It's a frustrating story but in a way, it is uplifting in the context of data privacy. Vodafone is a large IT user with massive data centers, yet its databases can't handle a simple match between just two pieces of data.

The dirty little secret of enterprise IT systems is that while they do store a lot of data, they are terrible at using that data. Relational database systems are extremely difficult to query, it can take weeks to design and run set of queries against the data. All major corporations have these relational database systems.

We thus have a large measure of protection from egregious uses of our personal data because of the inefficiency of corporate IT systems.

And this situation won't change anytime soon, even though far more efficient database technologies are available (such as Mark Logic). No company wants to mess with its core database systems -- a lucrative secret that Oracle discovered a long time ago.

Companies will rip out and replace servers but they will do everything they can to keep their databases intact -- it's core to their business.

Facebook, Google, and many others, are using specialist databases, some that they've developed themselves to overcome the limitations of relational database systems. But, because those databases are customized for their unique uses, it is very difficult to map their data against data in other types of databases. Using people's personal data to sell products and services is very difficult.

So, while we do need to be concerned about the commercialization of our personal data, and campaign for tougher controls, the likelihood that corporations will be able to use all that personal data in a meaningful way is very low. Data goes in, but getting it out in a useful way is shockingly difficult.

There is even more good news: inefficient database systems will continue to dominate the Fortune 1000 for many years. Database architectures have to remain the same because they have to maintain backward compatibility.

It could be said that the biggest defender of our data privacy is Oracle and its massive installed base of relational database systems.