Single Source of Truth, Nirvana and a Good Run Spoilt

This post came out of a presentation I recently gave to a group of colleagues. I’d promised to put something down on paper to draw out the key points for that group so thought I’d take the opportunity to share with the wider community at the same time. 

When presented with the question “is a single source truth a good thing?” how would you answer? For most people, particularly those of us with business intelligence backgrounds, the intuitive answer is likely to be “yes, of course”.   We’d answer in quick-fire fashion because it’s just one of those things that we know to be true. In a similar fashion we’d quickly voice our agreement with the statement “there has been almost no good music released since 1989”.  Wouldn’t we?

Unexpectedly, I recently had my morning run violently collide with a single source of truth issue. Before I left for the run I loaded up my iPod shuffle with a playlist drawn from the Classic Rock genre within my iTunes library. Half an hour into my run all was going well, the sun was shining and I was happily singing (puffing) along to the sounds of The Police, Queen, Peter Gabriel, Iggy Pop and Nirvana. Hang on, Nirvana? In what universe is Nirvana classified as Classic Rock? So thrown was I by this heinous data quality issue that I had to physically stop running to jump to the next track. Talk about poor data quality having a huge impact on productivity!!

But what could have gone wrong? Surely iTunes is the best single source of truth for information about the music from its own library.  But the problem is just that – it’s a single source of truth but its expected to serve a huge diverse audience. How can we expect this one source and one version of the truth to suit us all? While I’d put Nirvana into the Grunge genre I’m sure many others would agree with the iTunes view and call it Classic Rock, while some of the current crop of teenagers would probably relegate it to the Golden Oldies genre.

For me, one potential answer to this problem is to draw different information from different sources, taking what’s the best fit for a given use and perspective from whatever the most appropriate source is and bringing it together into a composite view – a 360 degree view of an item. A key point to note here is that some of these sources of truth may not be the system of record, there may actually be more appropriate data hidden on a PC under a desk somewhere.  The critical data should, and perhaps, even must come from the system of record, but don’t rule out other data. In my iTunes example I’d expect items like the track names, the album name, the artist name and the year of release to all come from whatever central source feeds iTunes as these are all hard and fast and should be indisputable. But as for the rest, let them be drawn from elsewhere. Let people be able to take genres and ratings from sources which match their perspective and have it surfaced together on their screen with the core information. If one of those sources is unofficial then there’s an opportunity to bring it into the tent so that there are some controls and governance around it, and this should be explored.

Now, for all I know this iTunes data mash-up may already be possible with some combination of settings in iTunes and on my iPod, but I’m just a dumb user and I couldn’t find it in the 30 seconds that my patience allowed me to look for an answer. In a business scenario our users aren’t going to spend time looking for ways to work with the tools we give them either. We need to make sure what we give them is flexible, easy to use and intuitive or they’ll find other ways to work with the data, potentially opening the door to a raft of data quality and integrity issues in so doing.

And as for those of you asking why I had Nirvana in my iTunes library in the first place. There was a passing moment in Philadelphia in the 2003 when I was listening to them, but the attraction has long since passed.  We’ll leave the discussion of my failure to implement a decent Information Lifecycle Management Policy over my music collection for another day!

Oh, and by the way, if anyone does happen to know an alternate source of truth which presents genres as perceived by the average forty something middle class man please drop me a line J