GenAI First Steps — Use Case Selection is Key. How to Pick a Winner.

 GenAI has been on my mind lately, as it has for many data leaders. This piece struck a chord with me in the way it considered the early steps toward GenAI adoption.

I particularly like the approach of describing those early moves as experimentation and controlled experimentation at that. That’s a powerful way of putting it; it implicitly conveys the need to proceed with due caution to reap the benefits. It is indeed a balance between risk and reward, between caution and speed.

These early steps toward GenAI are a delicate balancing act—not just for organisations but also for the reputations of those who lead their data functions.

Generated with AI. September 19 2024 11:26PM

For me, a well-considered choice of use cases is critical. Pick the right few to start the journey:

• Ones that don’t reach too far and won’t see the light of day without a squadron of consultants
• Ones that recognise and can take advantage of the (actual) state of readiness of your data estate (and associated tech)
• Ones that don’t carry too much downside risk if things go wrong
• But, equally, don’t choose those that only deliver benefits so small (or easily achievable by other means) that the outcome is seen as inconsequential

You need to signal you’re moving to keep key stakeholders happy and stop a raft of parallel fragmented activity. But more than that, there’s also an excellent opportunity to act to control a risk that is emerging (already manifesting) across a much larger stakeholder cohort. That’s staff rushing to embrace the productivity boost they see GenAI can give them but not thinking about / knowing about / caring about the risk that may bring. Find an early use case that helps staff do the right thing rather than feeling their only choice is between reaching outside or doing nothing.

Choosing the first GenAI use cases well can also offer a means to deal with this risk. Before corporate (or customers’) data goes outside organisational walls or a hallucination is blindly acted upon unchecked.

And then there’s the question of AI governance.

- When should we introduce guide rails vs. guiding principles vs. hard and fast standards and robust, comprehensive frameworks?

- How much is too much? And how will we know when that point has been eclipsed?

- Can this governance development work as a parallel activity rather than something that needs to be landed first?

Pragmatically, I see the need to implement “enough” governance while actively watching for unchecked behaviour or people seeming mired and unable to move. We must also be prepared to learn, iterate, and adjust quickly.

This brings us back to one more criteria for use case choice—getting those right (or at least right enough) is critical!

The Rise of the Chief AI Officer: A Strategic Imperative for Data-Driven Leadership

 

As we stand on the precipice of a new digital dawn, the role of the Chief AI Officer (CAIO) emerges as a beacon of innovation. This role is not just another title in the executive suite; it is a clarion call for a transformative approach to leadership in the age of artificial intelligence. The CAIO is envisioned not as an isolated figure but as a unifying force, seamlessly integrating into the very core of an organisation’s strategy.

 

Hold on there! I didn’t write that, AI did. Sounds compelling though, doesn’t it? Either that or it sounds like the first paragraph of a novel in which our hero, CAIO, saves the day.

Image generated with AI, August 25 2024 at 6.19pm

I came across this article (The Changing C-Suite: Chief AI Officer In, Chief Diversity Officer Out (itprotoday.com)) which put forward different points of view about the potential for the emergence of a Chief AI Officer and where the role might fit within the broader C-Suite. It got me thinking and I dropped a few quick thoughts on LinkedIn.  Now that I’ve mulled some more and my friendly Co-Pilot has grabbed your attention with his (its?) prose, here’s a little more reflection…. (This bit actually written by a human!)

 

The corporate world is abuzz right now with the potential of artificial intelligence and the lure of its promise to redefine the boundaries of innovation and efficiency. But the success of AI, to my way of thinking at least, in many cases can’t be separated from the quality of the data that fuels it. Well-managed, accurate, trusted and understood data is the cornerstone upon which AI systems should be built. Without this critical foundation, AI projects are at risk of underperforming, leaving a chasm between the expected value and what’s actually delivered.

 

Beyond the Silo: The Collaborative Paradigm for Sustainable AI Integration

I’d suggest that if Chief AI Officers do appear as a mainstream role, the successful amongst them will be those who embraced (and were allowed to embrace) the multifaceted nature of the role. A leader who is solely focused on AI and what it can do for the organisation isn’t setup for success. He or she is potentially isolated from key enablers. At best, isolated in their own process, they lack the influence to attract the necessary consideration and support from others. At worst, they’re blind to the things that could erode the value AI brings or cause serious organisational harm springing from an unseen implication or missed consideration.  Without an holistic view of the organisation’s data and all its nuances, they may well find themselves at a disadvantage. This narrow focus can lead to a short-sighted approach chasing quick perceived value, but potentially overlooking the broader implications on risk and governance.

A siloed Chief AI Officer may find themselves navigating a labyrinthine path, fraught with challenges, short-lived victories, not to mention their own short tenure!

The sustainable integration of AI into business practices needs a leader who is not only doggedly chasing the value from AI but also has a deep understanding of the business landscape. The Chief AI Officer must not just be a collaborative master personally, but also build a culture of collaboration, working hand-in-hand with both line of business leaders and data leaders.  Anything else may well fail to ensure that AI initiatives are more than a momentary shining star.

 

Chief Data Officer: Sidekick or hero in this story?

The Chief Data Officer, already attuned to the intricacies of managing both defensive and offensive with regard to data assets, provides a complementary perspective to the Chief AI Officer. By joining forces, either as a single role or a collaborative partnership, the CAIO and CDO can create a synergistic relationship that uses AI for its transformative power and new ways to deliver value while also building upon and strengthening pre-existing data imperatives.

My view is that early integration is a good thing. Sure, find the box canyons and the gotchas the hard way. Innovate and potentially fail before trying again. But, wouldn’t things be easier with an ally to help point out the pitfalls ahead of time?

Balancing offensive gains with a defensive awareness and building from a solid base, gives our hero half a chance to move from just starring in a quickly forgotten short story to become the feature act across a box set of novels.

---

 

A Christmas Rant - Who Needs Data Quality in Marketing?

A few days before Christmas my family received a Christmas card in the mail from the real estate agent who handled the sale of the house we bought about 18 months ago. It read:

"Dear Scott, Louise and family, hope you are enjoying the big house. Thanks again and warm regards...."

No doubt it was part of a campaign to have us keep the agent top of mind next time we choose to sell a house, but a nice touch nonetheless. Or at least it would have been had my wife's name been Louise!

What's more the "big house" reference was odd as we had down-sized when we bought this house in order to get a property closer to the city. Something the agent should have known only too well as it had come up in conversation on a number of occasions.

Whilst I found the use of the wrong name a little amusing and the reference to the size of the house didn't bother me, my wife was less than happy. It's a safe bet that when we do choose to sell this house this agent will indeed be top of mind - as someone we will definitely not be using.

For less emotive reasons, I'd actually arrived at the same conclusion as my wife. When I sell a house I want the agent acting for me to employ numerous tools to ensure the fastest sale for the best possible price. One of the tools I expect him to use is a database of potential purchasers which he or she has compiled and curated over time. Now, if the agent in question is unable to get basic details right (such as the name of a purchaser and the size of houses they have lived in) then what faith can I have that any of the other details he has will be accurate. Chances are that all he has is a list of names and a collections of facts which may or may not be right. the marketing campaign he'll conduct on my property may well be targeted incorrectly, either missing an obvious potential buyer or having me waste funds marketing to those who have no interest in my property.

So, he's lost a potential future client. So what? Does that really matter, he hasn't really lost any money, has he? Well, I think he has. If you consider the standard fee for an agent (where I live at least) is 4.25% of the sale price and my house might sell for around half a million dollars then he'll actually miss out on the chance of around $21,000. What's more, if my wife tells 10 of her friends that number balloons out to close to a quarter of a million dollars. Sure, not all of those would have been in the market to sell, nor might he have been one of their considerations for a listing agent, but the point is that there is indeed a cost to what at first glance seems like an almost irrelevant data quality issue.

You don't need to work in real estate to be subject to cost stemming from data quality problems. I'm sure that something of the same ilk could happen in many industries. So next time you're faced with what seems like a data quality issue too trivial to worry about just remember John* and his lost income.

* Name changed to protect the not so innocent.

The Second Most Important Question in the World

There’s a lot to be said for business intelligence practitioners being responsive and being able to deliver to and meet the needs of their clients quickly and much has been written about the benefits of BI teams partnering with their business colleagues in order to provide the best outcomes and value.

But doing so shouldn’t just be interpreted as giving business users everything they ask for as fast as humanly possible. Across my career I’ve found one simple two word question very powerful. That question is: “so what?”

The value of asking "so what" comes partly from asking the “how will you use this data?”, “what decisions will it inform?”  type questions, but it is even more powerful when turned around. Ask the question: “If I didn't have that data available to me - so what?" If the answer is something along the lines of: “I wouldn't know that we had a preventable problem and in 5 minutes there'd be raw sewage running down the main street” then it's worth doing something about. Write that report, enable those analytics – and do it quickly. However, if the answer sounds more like: “I might need that extra piece of information one day”, then perhaps let's wait for one day to come before we spend the effort, time and money.

In today's climate funding and resourcing are not what they once were in BI. There are few open cheque books left and those of us managing the function have to work with other leaders in our organisations to prioritise. Not only does working on low value things demoralise the members of the BI team, it's also an opportunity cost to the organisation. Working on a low value or low impact requests means that something else, something that may have enabled a better decision, led to time saved, a lower cost to serve or a customer prevented from churning was pushed aside and missed.

The “so what?” question is a powerful one. Just be careful how you ask it, and never presume to know the answer or to be able to second-guess your business colleagues!

And, as to the matter of the [first] most important question in the world. Having just spent two weeks travelling long distances on planes and in cars whilst on holiday with my kids I can assure you that the most important question in the world is “ARE WE THERE YET?”

What To Do When Your Client Wants Sizzle Not Sausage

I love a good dashboard- really I do. So, you'd think I'd be pleased that a trend seems to be emerging that has more business folks becoming aware of dashboards and, by extension wanting to make use of them. But I'm not. Sure, I'm all for the opportunity to provide insight and a better means of managing business operations with an effective dashboard, but the trouble is far too many of the dashboards I've seen built don't do that. Sure, they're flashy, with widgets and gauges galore and often even visually appealing at first glance, but they're simply not designed to give the right sort of insight, or provide that insight in a way that allows them to be used effectively and efficiently. Now that's often not the fault of the person charged with designing the dashboard it may stem all the way back to the client - the person requesting the dashboard.

Unless you have a long standing relationship with your client, excellent rapport and tremendous trust it's unlikely that you'll have carte blanche to determine how they dashboard should look. In the majority of cases they will have come to you with a design already in mind. Chances are that that design will have been influenced by what they've seen elsewhere - be it in other dashboards, or in advertising from dashboard vendors or implementation consulting firms, many of whom seem to promote [just] the eye-catching nature of dashboards, showing off the latest and greatest controls and visual capabilities. And here's the hard pill to swallow, no matter how many of Few's books you've read, no matter how many industry awards you may have won for your prior dashboards, no matter if you have a PhD with a thesis written on visual cues and how people respond to them, your client is still likely to look at your with a mixture of confusion and disappointment if you suggest alternative design and layout options which are too far apart from their own ideas. If you just build and present him or her with what you believe is the better option then 8 or 9 times out of 10 he'll say it's not what he asked for or wants. Push the issue, spouting all of the theory you like, and you'll do little more than back him into a corner. He'll still want the same design, you'll still need to build it, and the company will have an ineffective dashboard - no-one wins in this situation.

So, what's the answer? Why not build prototypes for both - build him his, but also build what you think will work. Give your client what he wants and you may in turn get what you want, either through the chance to explain why you believe you dashboard layout is more effective, or he may just see it for himself and have a eureka moment. At the very least you will have opened the door for a discussion and now be better placed to have a conversation about the things you really need to know - what's the core aim of the dashboard, who will its audience be and what behaviour is it trying to produce, etc.  And, who knows, the same may happen in reverse, having seen your client's option made real you might have a similar moment and realise that it does in fact work more effectively than your own concept. That's not a bad thing, the company will have a usable and useful dashboard and you'll have discovered another design pattern that you can store away to be leveraged later.

The Hidden Properties of SAP Business Objects Dashboards

Aka: What to do when the Properties window of SAP Business Objects Dashboards isn't visible.

Having had this happen to me three times now on three separate laptops, and also having spent, and lost, countless hours looking (largely unsuccessfully - until now) for a way to remedy the situation I thought this one deserved a blog post. So here goes. Hopefully the next person who strikes this problem can solve it with a five minute Google rather than a five hour session of window movement and software uninstalls and re-installs.

My problem originally occurred with SAP Business Objects Dashboards 4.0. I'd been happily working with the product for about a year with no problems until one morning I plugged in my laptop and opened up an existing dashboard to find that the Properties window was nowhere to be seen. No matter what I did I couldn't make it visible. I tried everything I could think of:

• A Google search;
• An SCN search;
• Right-clicking an object and selecting Properties;
• Highlighting an object and using the Alt-Enter keyboard shortcut;
• Highlighting an object and Selecting View->Properties for the menus;
• Using keyboard shortcuts to give the Properties window focus and then moving it with the keyboard arrows thinking it may have been hidden off screen or somehow thinking that it was on an extended display on another monitor;
• Plugging in a second monitor and then toggling which display was primary and which was extended.

All of these were tried (several times) but to no avail. Next I bit the bullet and tried re-installing the software. Several hours later I found that the Properties window still had an extreme case of shyness and wouldn't show up on my screen.

Luckily, not too much later, I was in line for a laptop update so when my laptop arrived with a fresh install of the Dashboards software all was fine once more - the Properties window was no longer hidden..........at least for about three weeks when the problem re-occurred. Faced with this situation I did what any good manager would do and gave the dashboard work to someone else to take care of.

A week or so ago I moved to another new more portable laptop and installed SAP Dashboards 4.1 on it. Hardly daring to look, I opened up my first dashboard, and .............. joy of joys - I had a Properties window once more. Or at least I did until this evening.  It was there when I left work at 6pm, but not there when I logged back on to my laptop at 9pm. Facing a looming deadline to publish some new corporate dashboards and with no more budget to change laptops again I decided to dive into the Registry.   I found the key that sound about right, exported it for safety,  and started flicking through each of the sub-keys looking for anything that sounded like it might be about window positions, toolbar settings or last saved details. After numerous failed attempts to reset values in keys that looked to fit the bill, and urged on by the dulcet tones of Peter Gabriel, I pulled out the sledgehammer and deleted all of the sub keys under the settings key. I figured / hoped that they would be rebuilt next time I opened the Dashboards tool and I'd trigger a reset to factory settings type operation by brute force. With fingers crossed I re-opened Dashboards, and there it was in all it's glory - the Properties window: 

So, should you find yourself unable to make the Properties window show itself fire up Regedit and navigate to HKEY_CURRENT_USER>Software>SAP Business Objects>Dashboards>Settings>en

From there delete every subkey under en. Don't delete the en subkey as well, as that seems to render Dashboards unable to start. 

I've tried to identify the specific subkey that needs to be deleted, but with no luck. But that doesn't seem to matter as deleting them all fixes the problem and doesn't seem to have any nasty side-effects. 

This works for a Dashboards 4.1 system and I suspect it would also work for Dashboards 4.0 and probably even for earlier Celsius versions too.

If anyone else has experienced this problem and has figured out what triggers it, I'd love to hear from you!

LCMs - Oldies Just Don't Get It!

Here in Australia we have a breakfast bar that goes by the name LCMs. For those of you that don't know them, they're much like the Rice Krispie Squares found in other parts of the world. The marketing folks behind LCMs appealed to the kids market with the tagline: "LCMs - Oldies Just Don't Get It".

Recently I've been dealing with another LCM - the Life Cycle Management component within SAP's Business Objects suite. Strictly speaking, it's now known as Promotion Management, but I'm old school enough to still think of it as LCM.

Now, I'm all for the style of deployment that Promotion Management enables, but recently I've struck an odd error with its use that perplexes me. There's one think about LCM that this oldie doesn't get!

Recently I've had to turn my hand to something that it's been a while since I've done. I've had to play 3rd level support / SysAdmin to a production Business Objects reporting environment and its supporting development and testing tiers. Some new developers had joined a project using these environments to bring in some new WebI reports along with a few tweaks to the Universe. All went well until one of the new folks tried to promote a report between the development and testing tiers. The nature of the data in play means that the two tiers aren't allowed to have a communications path between them so the promotion needed to make use of a BIAR file. When attempting to create the promotion job after importing the BIAR file to the test server the developer was getting an odd, and not particularly graceful error. It read:

"AjaxRequestThrowsError":"True"

Attempting the same task with a user with admin rights was successful and, interestingly, so was undertaking the task when signed in as one of the original developers.

Effectively this seemed to rule out something more basic wrong with Promotion Management on that box. So, to me this looked like a permissions issue at heart. My first instinct was to look to see if the new developers had been granted to access to use Promotion Management. Aha! None of the new folks had access to the Promotion Jobs folder on the test server. Surely that's the problem.

Easily fixed. A quick application of user security and back to the developers to test. No luck - the same error we'd seen earlier persisted. Now I'm no masochist and I'm far to busy to re-invent the wheel, so off I went to ask Google and SCN. Sure enough both came back with a number of hits that described the same error, but none that matched our case or symptoms closely enough for my liking. So, like it or not, there was a bit more digging to be done. Through a process of comparison and elimination I noticed that there was one group membership that the new developers were lacking. Digging some more showed that this membership gave access (Full Control)  to the folder on the test server that the WebI report to be promoted would live in. And sure, enough after adding one of the new developers to this group the Promotion Management error disappeared and he was able to successfully promote via BIAR file.

Problem solved!

However, one puzzling aspect the remains for me is why this particular error was thrown on the creation of the promotion job and not on the execution of the promotion itself. Even more confusing was the fact that when we temporarily allowed a communications path between the the development and test servers the developer was able to successfully directly promote his changes from server to server (i.e. taking out the need for the BIAR file) even before we'd add the group membership.

With luck, if I ever have to solve the same problem again, this blog post will prompt me to remember the solution without the need for the investigations that went along this time. Even better, it may help someone else being confronted and confused by the same error!

Even better again, if you can shed some light on this odd set of behaviour, I'd love to hear from you. If I'm missing the obvious here, please point it out to me. This oldie doesn't mind it if he's able to get this LCM! And I'm sure the marketing folks as Kellogs won't care either!

The Desire Paths in Your Data

On the days that I drive to work I have around a one mile walk from my car park to my office. There's one particular spot where I can take a shortcut by going off the paved path and cutting across some open ground. It saves me a minute or so on the way to work. Despite the fact that there is a perfectly good sidewalk built by the local council it's obvious that a lot of people must do the same as me, and opt for the faster route as well, because there's now a clearly worn path across the open ground, established by people choosing this path every day. Urban planners call these desire paths - the paths that people choose to take between two points regardless of other options that may be available to them.

If you look hard enough you'll find desire paths through your data as well. Your business intelligence solution offers many conventional paths (like the sidewalks built by my local council). These are the pre-built reports and other instruments that you present to your users. But I bet many of your users have built desire paths - the tips and tricks they use to get and analyse the information they need to do their everyday jobs. Or perhaps the extra things they have to do, those downloads and manipulations in Excel or the sneaky Access database sitting on a spare PC under someone's desk,  because your BI solution doesn't let them do their jobs or perform the analysis they need in its entirety. Now you could argue that desire paths in a BI solution actually represent a form of self-service BI. However, unless you've engineered things this way then you're probably only kidding yourself. Yes, desire paths are examples of your users finding ways to help themselves, but chances are that it's due to shortcomings, not features, within your BI solution.

Desire paths have a dark side in the natural world - they often result in trampled vegetation, can contribute to erosion and (not being as safe as constructed sidewalks) can also lead to injury for those using them. Similarly, in the IT world they are actually likely to cause harm to the reputation of your BI solution or of your BI team. Just like people in the natural world wandering a direct route desire path, shaking their head(s) and wondering why their local planning authority spent all that money building a meandering sidewalk twice as long as it needed to be, your users may well wonder where all of that IT spend on BI went, especially when they "can do the same thing (but better) in Excel". Perhaps worse still desire paths can cause actual harm to your organisation, be it financial or other type of loss, due to decisions based upon flawed analysis or data as desire paths can lack the checks and balances that come from underlying planning.

Our challenge as practitioners, managers and custodians of our organisations' BI assets is to find these desire paths (and in so doing also to find which of our pre-built paths are no longer as relevant as they could be). Can we build established footpaths where the users' desire paths have been worn in over months and years of usage? It works in the natural world and it can work in the technology world as well. There are stories of park planners in some countries visiting areas after fresh snowfall to get a sense of where those using the area are choosing to walk. I've heard of at least one university which built no structured sidewalks on a new campus until such time as the students established desire paths between buildings, simply choosing to pave these paths one year on.

In today's reality of shrinking budgets we all get busy just keeping our BI environments running and meeting the needs for new reports and analytics, let alone spending time worrying about how we'll deal with the next big (data) thing looming on the horizon. But I'd advocate there may be good value to be had in taking the opportunity to look for and act on desire paths when we can. The trick will be finding, or engineering, those occasions, like those times of fresh snowfall, when the circumstances are right to see the desire paths and incorporate key ones amongst them into your BI solution.

Happy (re)building!

Your BI Project Shouldn’t Be a Lag Activity

There's a school of thought which advocates that Business Intelligence work should lag behind the other pieces of a larger project and that one should never attempt to undertake business intelligence (BI) activities in parallel with the other "core" activities of the project. The line goes something like this: you can't undertake a major project, say the implementation of a new ERP system, and design and implement BI over that system at the same time; the BI components have to happen 6 months or so later, they can't be designed and built as part of the main project because they are dependent on the main project items being in place first. Those espousing this popular line would have us believe that one can't build BI until the system is up and running, there's a good volume of data available and the business users have worked out what reports they really want and need. Anyway, there's no rush - reports and other BI items are only an output after all!

I disagree. Business intelligence items are as much an input as an output. Yes, they draw upon data in the system, but, compliance reports aside, the information they present should be used as part of a decision making process which then causes some course of action to be taken in a larger process. If this isn't the case then I'd argue that the report in question adds no value. It may just as well not be there!

Look at the idea of doing the BI components at a lag behind the main project through another lens. What this is really saying is one of a few things: 

• We don't need the information we'd get from the reports to base decisions upon;

•  We’re happy coming up with and using a range of time consuming workarounds;

• We’ve got a good gut feel for the business. We can get by making guesses for a while; or

•  We can get by not doing some of the things that we used to do before we put this new system in place. This could be anything from something as simple as not providing team managers with trend reports of absenteeism through to things with much more potential for a financial sting in the tail such as not re-forecasting project budgets after the initial baseline as been set.

Also take some time to think about the resultant behaviors that may arise from not having reporting and analytic capability available early in the life of your new system. People in your business need to make choices and they need information to do so. If they can’t get it from BI, from the right channels, then they’ll find and develop a myriad of workarounds in attempts to continue to do their jobs. Wait too long to bring BI to the party and chances are a good number of those workarounds will persist in various pockets throughout your business. 

So I’d suggest a pause for reflection before rushing to adopt the common [BI] scheduling wisdom. Put it in the above terms, mull over it, have the leaders in your business consider it too and then see if there’s still the same appetite to defer that BI work for six months. Yes, bringing the BI work in to the main project isn’t easy, challenges abound, at times it feels like you’re building on shifting sands but the pain of not doing it may be much worse.

Data Migration: Low Hanging Fruit Can Be Bad For Your Health!

 If you’ve been through a few data migration exercises then you’ll probably recognise the scene.  You’re in the midst of a busy data migration, the pressure is on, there’s too much to do and too little time to do it in. You’ve been back and forward with the business users trying to resolve data quality issues that are preventing data from successfully loading in the new system. One, two, three, or more trial data loads have come and gone and still there are pieces of data which stubbornly refuse to play ball and load in to the new system.

With deadlines looming you start looking for ways to solve the problem, to migrate the data, to just get it in there. Or perhaps, even worse, those above you begin to apply the pressure for you to do the same. The questions / demands start to come. “Look for the low hanging fruit, what can you do to just make it work? Just get the data loaded!”

One common idea is that a whole parcel of load errors associated with free text fields can be easily and quickly solved by simply truncating the data to match the maximum allowable length of the fields in the target system.  Load the data in; produce an exception report for the text fields that were too long and were truncated and, hey presto, problem solved! You’ve got a bundle of errors out of the way and proved that the data can load. There’s now confidence that, come the next cutover trial or the real cutover and go live, that the migration in this area will be fine. Now it’s been done successfully once it can be repeated successfully and can only get better. After all the business users will get your exception report and immediately and diligently get to work on cleansing those fields which needed attention.

They will, won’t they? It’s not like they got anything else competing for their time. Their line manager doesn’t need them to do anything else. They’re ready and waiting to cleanse data for you as their top priority.

OK, wake up and head back over to the real world! Once (or if) they finish doing the day to day tasks of their own job, not to mention the tasks of those of their colleagues who have been seconded to that big project that is underway they may have time to add the data cleansing tasks to their to do lists, but I bet those tasks don’t get added to the top of the list.  If that data is going to get cleansed then you need to present a reason to prioritise its cleansing.  A hard failure is that reason, and a much more compelling and immediate reason than an exception report for data which (after all) has already loaded.  If the data doesn’t load then it needs to be cleansed if it is to be available I the new system.

Give the business all the help you can. Point out exactly and ambiguously what the problem is, show them exactly which field(s) in which record(s) need to be cleansed and what good looks like (i.e. what rules and conditions must be met before the data can migrate) and use data profiling tools to give them some indications ahead of the next trial load that the cleansing they are doing is effective and having the desired effect. But, don’t own their problem nor try to solve it for them in isolation. Most times, the business folds will know the data much better than the data migration folks. As soon as the data folks start making assumptions about what is safe to do with the data in the name of getting it to load, be it truncating text fields or some other technique, then the organisation takes on a long tail risk and liability in that the meaning of the data may have been lost or changed which may manifest in negative impacts to process or operations at some point post go-live. Bad for the organisation and bad for you as often times it is the data migration folks that will be perceived as having been the cause of the problem.

But, in the end, pragmatism must rule the day. Your data migration must run. There will be those (hopefully few) areas where you’ll just have to make some calls for the sake of allowing the data to get into the system at final cutover. If you don’t you’ll find your name quickly in the same sentence as the words “…we didn’t go live because of…” So go ahead, pick that low hanging fruit, but wait until you absolutely must, it still may be poisonous, but less immediately deadly than the alternative!