Single Source of Truth, Nirvana and a Good Run Spoilt

This post came out of a presentation I recently gave to a group of colleagues. I’d promised to put something down on paper to draw out the key points for that group so thought I’d take the opportunity to share with the wider community at the same time. 

When presented with the question “is a single source truth a good thing?” how would you answer? For most people, particularly those of us with business intelligence backgrounds, the intuitive answer is likely to be “yes, of course”.   We’d answer in quick-fire fashion because it’s just one of those things that we know to be true. In a similar fashion we’d quickly voice our agreement with the statement “there has been almost no good music released since 1989”.  Wouldn’t we?

Unexpectedly, I recently had my morning run violently collide with a single source of truth issue. Before I left for the run I loaded up my iPod shuffle with a playlist drawn from the Classic Rock genre within my iTunes library. Half an hour into my run all was going well, the sun was shining and I was happily singing (puffing) along to the sounds of The Police, Queen, Peter Gabriel, Iggy Pop and Nirvana. Hang on, Nirvana? In what universe is Nirvana classified as Classic Rock? So thrown was I by this heinous data quality issue that I had to physically stop running to jump to the next track. Talk about poor data quality having a huge impact on productivity!!

But what could have gone wrong? Surely iTunes is the best single source of truth for information about the music from its own library.  But the problem is just that – it’s a single source of truth but its expected to serve a huge diverse audience. How can we expect this one source and one version of the truth to suit us all? While I’d put Nirvana into the Grunge genre I’m sure many others would agree with the iTunes view and call it Classic Rock, while some of the current crop of teenagers would probably relegate it to the Golden Oldies genre.

For me, one potential answer to this problem is to draw different information from different sources, taking what’s the best fit for a given use and perspective from whatever the most appropriate source is and bringing it together into a composite view – a 360 degree view of an item. A key point to note here is that some of these sources of truth may not be the system of record, there may actually be more appropriate data hidden on a PC under a desk somewhere.  The critical data should, and perhaps, even must come from the system of record, but don’t rule out other data. In my iTunes example I’d expect items like the track names, the album name, the artist name and the year of release to all come from whatever central source feeds iTunes as these are all hard and fast and should be indisputable. But as for the rest, let them be drawn from elsewhere. Let people be able to take genres and ratings from sources which match their perspective and have it surfaced together on their screen with the core information. If one of those sources is unofficial then there’s an opportunity to bring it into the tent so that there are some controls and governance around it, and this should be explored.

Now, for all I know this iTunes data mash-up may already be possible with some combination of settings in iTunes and on my iPod, but I’m just a dumb user and I couldn’t find it in the 30 seconds that my patience allowed me to look for an answer. In a business scenario our users aren’t going to spend time looking for ways to work with the tools we give them either. We need to make sure what we give them is flexible, easy to use and intuitive or they’ll find other ways to work with the data, potentially opening the door to a raft of data quality and integrity issues in so doing.

And as for those of you asking why I had Nirvana in my iTunes library in the first place. There was a passing moment in Philadelphia in the 2003 when I was listening to them, but the attraction has long since passed.  We’ll leave the discussion of my failure to implement a decent Information Lifecycle Management Policy over my music collection for another day!

Oh, and by the way, if anyone does happen to know an alternate source of truth which presents genres as perceived by the average forty something middle class man please drop me a line J

So How Does This Blogging Thing Work Again?

OK, I’ll admit it. Things have been a little quite on my blog recently. Eighteen months is quite a time between posts. Surprisingly, my lack of writing is not due to a lack of time – although my second child did enter the world not long after my last blog entry. It’s more a reflection of a change in circumstances for me, a move from hands on designing and building business intelligence solutions to a role which involves strategizing, positioning, promoting concepts and attempting to envision future ways of working with information.

It’s amazing how not having specific, tangible and concrete problems to solve, issues to deal with or systems to build seems at first to rob you of anything worthy of sharing with the community. But now I’m hoping that some of the concepts I’m working with and the bigger issues that I’m chipping away at over time will carry interest for at least some of you. I doubt I’m the only person facing these challenges and a little sharing and exchange of opinions might be helpful.

Among other things, in the coming months I expect to be trying to establish and build traction around data governance, shape several data migration activities and find ways of better using metadata in our business. If all goes well I’ll also get the opportunity to kick around some data mining based R&D. I’ll endeavor to share what I can where I think it might be of interest. Hopefully there’ll be a bit of two way communication so please do drop me a line if you’ve tackled the problems I’m writing about or have a view on the concepts I’m discussing. I’m a big believer in leveraging multiple points of view where I can!

The first cab off the rank will be a blog entry that combines the concept of a single source of truth, the band Nirvana and my morning run. Keep an eye out for that post in the next day or so.