Data Migration: Pushing Risk to the Vendor is not the Answer

Data migration exercises are risky. I've previously blogged about steps that can be taken to reduce risk in my posts on Selling the Data Migration Strategy and Data Migration: Track Record or Process Maturity. One area that I've not touched on in either of these posts, but one that has come up as an idea in pretty much every data migration project I've ever been involved with, is the use of the commercial negotiations process to handle data migration risks. Here I'm talking about payment milestones being closely tied to successful migration of the data. This must be a good thing, right? If our vendor doesn't get our data across to the new system then we don't pay them, or we withhold a sizable chunk of cash. I'm not so sure - for me this approach is at best a blunt instrument, something to beat a vendor over the head with, to cajole them into getting the job done.  

Step back and analyse the position you'd be in if you exercised the right to withhold some part of payment and you may start to share my point of view. OK, so your data migration project has hit problems, it's either running over time or the vendor has flat out just failed to deliver. That's not good, but at least you've got a good chunk of the project budget left, you've not wasted your company's money, or have you? Take a look at the projected benefits for the data migration project and / or the larger project the migration is part of and consider which of those benefits has either been forgone or deferred to some future date due to the migration problems. How do the financial outcomes look now? Sure, you may still have $200k in the project budget courtesy of the non delivery clause you negotiated, but you're going to have to spend that (and then some) to attempt the migration again, or fix what you have, with another vendor. Also some or all of the (financial savings) benefits the company was counting on in this year's budget now won't come through until next year (or later). Not the best situation to be in!

Of course, the counter argument can be made that by having financial penalties built in to the contract, or payment milestones tied to delivery milestones it's more likely that the vendor will be motivated to ensure that the job gets done, that your data gets migrated no matter what it takes. It's those five words which bother me - "no matter what it takes". What this may well mean is that vendor is sacrificing the true desired outcomes of your project in order to protect their financial interests. There will always be a degree of this apparent in any vendor- client relationship, but this is much easier to manage when there aren't extra pressures looming in the shape of potential financial forfeits. Shortcuts, a focus on short term fixes to specific problems rather than techniques in line with the best overall outcomes, and circumventing process and testing in order to hit deadlines are all behaviours you may need to deal with if you insist on these sorts of clauses. 

But what about the scenario where the vendor is the party suggesting this type of arrangement? In the past I've seen vendors offer arrangements where the lion's share of their payment (as high as 90% of the project total) is not due until the migration project is complete and accepted. You could argue that in such cases there is no harm in using these terms, after the vendor is offering the option freely, rather than being forced to this arrangement. In reality, the vendor may be being forced to these terms, not by you (their client) but rather by market forces. They may have to offer such arrangements in order to win work. Often this will be the case when they are inexperienced / new to the market space or have failed to build a portfolio of past successful projects which they can use as reference sites. There will be exceptions, but often the vendors who need to offer these types of terms are perhaps the vendors you least want to have on your data migration project. 



So, what's the alternative? In my opinion it's to be proactive. Don't push the risk to the vendor, but step up and own that risk yourself. Recognise and publicise the data migration risks early and take steps to mitigate them. It's your data so take accountability for the success of the data migration process. You can't afford to be simply a passive by-stander through the migration. If you are, when things fail (and they probably will at least to to some degree) yes, you'll be able to point the finger of blame at the vendor and say they failed to deliver.  But you'll still have a failed project on your resume, your company will have missed some or all of its projected benefits, and you may well have an extended miserable time as you attempt to mop up behind the project and live with work-arounds for years to come.  A data migration strategy, upfront planning, carefully selecting a vendor who is aligned to your strategy and likely to be able to deliver because they work effectively to an established and proven process, and staying actively involved throughout the migration are far more likely to keep  your risks from becoming realised issues. 


So, be brave, step up and own that risk. Good luck!

Selling the Data Migration Strategy.

There are two types of data specialists in this world: those who have been through a failed, or severely challenged, data migration project and those who will go through one in their future. Stick around in our field long enough and you'll join the former group, and it may not even be your fault. Even with the best intentions in the world, the foresight to see problems well ahead of time and a plan mapping out a path which has business users, data stewards and processes owners heavily involved and engaged you may still fail!

Why is that?

In reality there are several reasons but I'll drill in on just one here: the lack of senior management buy in and top cover for your data migration strategy. And I'm not talking about a data migration strategy just at the project level, but rather a data migration strategy set at the enterprise level. You do have one of those, don't you?

If you don't, do yourself a favour and start putting one together. Without such a strategy you'll likely find yourself fighting the same battles over and over,  having to justify your choices in the face of tight project budgets and deadlines which were set without regard for your tacit data migration principles, years of hard won experience or contemporary data migration best practices. You really can't [solely] blame the project manager who put the plan together either, after all what guiding documentation did he or she have to consult when they built the plan? Good PMs will look to be guided by existing strategy and principles documents if they exist and excellent PMs will probably seek out those people within a company with migration experience and specialist skills to see what they can leverage. I'm lucky enough to be working with one such PM now and have worked with a select few others in my time - you guys know who you are! But unfortunately the vast majority won't take such steps, preferring instead to depend upon their own past experiences or tasking business analysts to come up with a number - a hard task without specialist skills, experience and guidance. I've worked with far more than a few of these in my time - you guys probably don't realise who you are!

So, you need an overarching data migration strategy document to influence and guide behaviour when it matters - both early in the project planning and design stages and also later on as the pressure mounts close to the go-live date.  But, that's not enough. You also need to sell it to senior management, you'll need their support when the pressure hits. That pressure will come, there's no doubt about it. It may strike at different stages  depending upon the type of organisation you're working with. If you're faced with the task of introducing the concepts that data migrations are business, not technology issues, that business leaders cannot be passive in the migration and that ownership cannot be divorced from accountability then chances are you'll strike push back from business managers who don't want to lose key people to the project, don't have budget for business as usual back-fill or feel that the approach is just IT trying to abdicate responsibility. Alternately you may be faced with a aggressive timeframes or budgets which need to be challenged to ensure that you give the migration at least some chance of success. And, at some time a go-live deadline will mean that pressure will  come for data migration processes to be sidelined in a frantic rush to just get the data in any way you can.

When these things happen you'll need someone with real authority in your corner. So, how do you sell your overarching data migration strategy to the higher levels of the org chart in such a way that you don't just get moral support, but true buy-in? Chances are it will vary from organisation to organisation and even from person to person, but here are three ideas:

• Link the failure of data migration to delayed or foregone project benefits realisation. There's plenty of research out there that can show this linkage and whilst you won't know any specific numbers until the next major project business case appears you can at least plant the seed. Bonus points if you can look back in your company's history and quantify the amount of unrealised benefits for a prior project due to data migration issues. 
• If your company has culture of, or current focus on, safety then draw out how failed migrations could feasibly lead to a major safety incident or death.
• Show how data migration issues lead to damage reputation and brand. There are plenty of cases where this has occurred. I can count two in my city in the past three months alone.

Ideally back up your case with real world examples where a lack of data migration strategy, or a strategy which differs from yours in key elements, contributed to major data migration failures and manifestation of one or more of the above points. You'll find these in the press, case studies and academic papers, but perhaps the best source of such information will come from Auditor General's reports into failed or problematic Government projects as these will be both detailed and publicly available. Also gather some evidence of successful projects which were guided by a strategy which is well aligned with the one you've developed. These can be harder to find. There are academic papers and industry case studies, or you can leverage your network to gather these stories (and hard evidence to support them). Of course, if your strategy is well researched then chances are you may already have amassed this material as you built the strategy. 

Your data migration strategy should be a living document. Take the opportunity to learn from each data migration project, consider these learnings and adjust the strategy as and when there is benefit in doing so. Also keep your eye on the wider area as well. You may only do two or three data migrations every five years inside your company, but there will be many more going on in the broader community. Take the opportunity to learn from those where you can. To that end, I'm always keen to hear what others are  finding has worked (or not worked) for them. Drop me a line with your pearls of wisdom or to share your data migration battle scars - I'm keen to evolve my strategy too!