Thursday, October 27, 2011

Selling the Data Migration Strategy.

There are two types of data specialists in this world: those who have been through a failed, or severely challenged, data migration project and those who will go through one in their future. Stick around in our field long enough and you'll join the former group, and it may not even be your fault. Even with the best intentions in the world, the foresight to see problems well ahead of time and a plan mapping out a path which has business users, data stewards and processes owners heavily involved and engaged you may still fail!

Why is that?

In reality there are several reasons but I'll drill in on just one here: the lack of senior management buy in and top cover for your data migration strategy. And I'm not talking about a data migration strategy just at the project level, but rather a data migration strategy set at the enterprise level. You do have one of those, don't you?

If you don't, do yourself a favour and start putting one together. Without such a strategy you'll likely find yourself fighting the same battles over and over,  having to justify your choices in the face of tight project budgets and deadlines which were set without regard for your tacit data migration principles, years of hard won experience or contemporary data migration best practices. You really can't [solely] blame the project manager who put the plan together either, after all what guiding documentation did he or she have to consult when they built the plan? Good PMs will look to be guided by existing strategy and principles documents if they exist and excellent PMs will probably seek out those people within a company with migration experience and specialist skills to see what they can leverage. I'm lucky enough to be working with one such PM now and have worked with a select few others in my time - you guys know who you are! But unfortunately the vast majority won't take such steps, preferring instead to depend upon their own past experiences or tasking business analysts to come up with a number - a hard task without specialist skills, experience and guidance. I've worked with far more than a few of these in my time - you guys probably don't realise who you are!

So, you need an overarching data migration strategy document to influence and guide behaviour when it matters - both early in the project planning and design stages and also later on as the pressure mounts close to the go-live date.  But, that's not enough. You also need to sell it to senior management, you'll need their support when the pressure hits. That pressure will come, there's no doubt about it. It may strike at different stages  depending upon the type of organisation you're working with. If you're faced with the task of introducing the concepts that data migrations are business, not technology issues, that business leaders cannot be passive in the migration and that ownership cannot be divorced from accountability then chances are you'll strike push back from business managers who don't want to lose key people to the project, don't have budget for business as usual back-fill or feel that the approach is just IT trying to abdicate responsibility. Alternately you may be faced with a aggressive timeframes or budgets which need to be challenged to ensure that you give the migration at least some chance of success. And, at some time a go-live deadline will mean that pressure will  come for data migration processes to be sidelined in a frantic rush to just get the data in any way you can.

When these things happen you'll need someone with real authority in your corner. So, how do you sell your overarching data migration strategy to the higher levels of the org chart in such a way that you don't just get moral support, but true buy-in? Chances are it will vary from organisation to organisation and even from person to person, but here are three ideas:

  • Link the failure of data migration to delayed or foregone project benefits realisation. There's plenty of research out there that can show this linkage and whilst you won't know any specific numbers until the next major project business case appears you can at least plant the seed. Bonus points if you can look back in your company's history and quantify the amount of unrealised benefits for a prior project due to data migration issues. 
  • If your company has culture of, or current focus on, safety then draw out how failed migrations could feasibly lead to a major safety incident or death.
  • Show how data migration issues lead to damage reputation and brand. There are plenty of cases where this has occurred. I can count two in my city in the past three months alone.
Ideally back up your case with real world examples where a lack of data migration strategy, or a strategy which differs from yours in key elements, contributed to major data migration failures and manifestation of one or more of the above points. You'll find these in the press, case studies and academic papers, but perhaps the best source of such information will come from Auditor General's reports into failed or problematic Government projects as these will be both detailed and publicly available. Also gather some evidence of successful projects which were guided by a strategy which is well aligned with the one you've developed. These can be harder to find. There are academic papers and industry case studies, or you can leverage your network to gather these stories (and hard evidence to support them). Of course, if your strategy is well researched then chances are you may already have amassed this material as you built the strategy. 

Your data migration strategy should be a living document. Take the opportunity to learn from each data migration project, consider these learnings and adjust the strategy as and when there is benefit in doing so. Also keep your eye on the wider area as well. You may only do two or three data migrations every five years inside your company, but there will be many more going on in the broader community. Take the opportunity to learn from those where you can. To that end, I'm always keen to hear what others are  finding has worked (or not worked) for them. Drop me a line with your pearls of wisdom or to share your data migration battle scars - I'm keen to evolve my strategy too!


  1. Fantastic post, every company undertaking a migration(and according to Bloor most big companies do 4-5 a year) should stamp this on their boardroom wall.

    I created Data Migration Pro (and Data Quality Pro) to gather these kind of stories so would love to feature you on the site(s).

    I think the tide is turning but all too slowly. I came across a migration "plan" the other day to migrate an entire national database in 8 weeks with a weekend of testing at the end. They haven't even received a cut of sample data yet so they clearly have no data quality vision or landscape analysis in place.

    The problem wit ownership is that no-one really wants to own a migration. Because the company typically has no clue as to the processes, skills, contracts and technology involved they get it wrong.

    So many organisations just download frameworks and plans off the web and pass off as "strategy" but these are just guides, they need to create a roadmap, the vision, the end goal and actionable steps that reflect their type of business. Your living document tip is another tip that so many ignore.

    All this needs leadership and ownership, the crux of your post I guess.

    Awesome writing, keep it coming!

  2. Thanks, Dylan, for your kind words. I'm glad to hear of your sites and would be more than happy to be featured on either or both. Anything that helps raise the profile and importance of these topics is a winner in my book!

  3. Articles and content in this section of the website are really amazing. Great ideas indeed! I will surely keep these in my mind!

    Lizenzgeb├╝hr(en) & Sportlizenzen