The Second Most Important Question in the World

There’s a lot to be said for business intelligence practitioners being responsive and being able to deliver to and meet the needs of their clients quickly and much has been written about the benefits of BI teams partnering with their business colleagues in order to provide the best outcomes and value.

But doing so shouldn’t just be interpreted as giving business users everything they ask for as fast as humanly possible. Across my career I’ve found one simple two word question very powerful. That question is: “so what?”

The value of asking "so what" comes partly from asking the “how will you use this data?”, “what decisions will it inform?”  type questions, but it is even more powerful when turned around. Ask the question: “If I didn't have that data available to me - so what?" If the answer is something along the lines of: “I wouldn't know that we had a preventable problem and in 5 minutes there'd be raw sewage running down the main street” then it's worth doing something about. Write that report, enable those analytics – and do it quickly. However, if the answer sounds more like: “I might need that extra piece of information one day”, then perhaps let's wait for one day to come before we spend the effort, time and money.

In today's climate funding and resourcing are not what they once were in BI. There are few open cheque books left and those of us managing the function have to work with other leaders in our organisations to prioritise. Not only does working on low value things demoralise the members of the BI team, it's also an opportunity cost to the organisation. Working on a low value or low impact requests means that something else, something that may have enabled a better decision, led to time saved, a lower cost to serve or a customer prevented from churning was pushed aside and missed.

The “so what?” question is a powerful one. Just be careful how you ask it, and never presume to know the answer or to be able to second-guess your business colleagues!

And, as to the matter of the [first] most important question in the world. Having just spent two weeks travelling long distances on planes and in cars whilst on holiday with my kids I can assure you that the most important question in the world is “ARE WE THERE YET?”

The Desire Paths in Your Data

On the days that I drive to work I have around a one mile walk from my car park to my office. There's one particular spot where I can take a shortcut by going off the paved path and cutting across some open ground. It saves me a minute or so on the way to work. Despite the fact that there is a perfectly good sidewalk built by the local council it's obvious that a lot of people must do the same as me, and opt for the faster route as well, because there's now a clearly worn path across the open ground, established by people choosing this path every day. Urban planners call these desire paths - the paths that people choose to take between two points regardless of other options that may be available to them.

If you look hard enough you'll find desire paths through your data as well. Your business intelligence solution offers many conventional paths (like the sidewalks built by my local council). These are the pre-built reports and other instruments that you present to your users. But I bet many of your users have built desire paths - the tips and tricks they use to get and analyse the information they need to do their everyday jobs. Or perhaps the extra things they have to do, those downloads and manipulations in Excel or the sneaky Access database sitting on a spare PC under someone's desk,  because your BI solution doesn't let them do their jobs or perform the analysis they need in its entirety. Now you could argue that desire paths in a BI solution actually represent a form of self-service BI. However, unless you've engineered things this way then you're probably only kidding yourself. Yes, desire paths are examples of your users finding ways to help themselves, but chances are that it's due to shortcomings, not features, within your BI solution.

Desire paths have a dark side in the natural world - they often result in trampled vegetation, can contribute to erosion and (not being as safe as constructed sidewalks) can also lead to injury for those using them. Similarly, in the IT world they are actually likely to cause harm to the reputation of your BI solution or of your BI team. Just like people in the natural world wandering a direct route desire path, shaking their head(s) and wondering why their local planning authority spent all that money building a meandering sidewalk twice as long as it needed to be, your users may well wonder where all of that IT spend on BI went, especially when they "can do the same thing (but better) in Excel". Perhaps worse still desire paths can cause actual harm to your organisation, be it financial or other type of loss, due to decisions based upon flawed analysis or data as desire paths can lack the checks and balances that come from underlying planning.

Our challenge as practitioners, managers and custodians of our organisations' BI assets is to find these desire paths (and in so doing also to find which of our pre-built paths are no longer as relevant as they could be). Can we build established footpaths where the users' desire paths have been worn in over months and years of usage? It works in the natural world and it can work in the technology world as well. There are stories of park planners in some countries visiting areas after fresh snowfall to get a sense of where those using the area are choosing to walk. I've heard of at least one university which built no structured sidewalks on a new campus until such time as the students established desire paths between buildings, simply choosing to pave these paths one year on.

In today's reality of shrinking budgets we all get busy just keeping our BI environments running and meeting the needs for new reports and analytics, let alone spending time worrying about how we'll deal with the next big (data) thing looming on the horizon. But I'd advocate there may be good value to be had in taking the opportunity to look for and act on desire paths when we can. The trick will be finding, or engineering, those occasions, like those times of fresh snowfall, when the circumstances are right to see the desire paths and incorporate key ones amongst them into your BI solution.

Happy (re)building!

Big Data - Is it Really New?

Despite the fact that Big Data is fastest becoming one of the buzz words in the data industry I still wonder if we yet know exactly what it is. Is it anything more than just a concept, an umbrella topic if you like, to attempt to signal that something new is beginning to emerge in our industry? But, is big data really something new? Is the buzz justified, do we really need to reinvent our approaches, our practices and the skills we engender in those coming up through data careers?

I’d argue that some, if not many, data professionals already know a thing or two about big data. Well over a decade ago I was the architect leading a project to build a data warehouse to handle call traffic and billing data for one of Australia’s biggest telecommunications companies. Back then the volume of data we were working with was hardly small and I’d suggest that those that the people who work with that warehouse, and the BI solutions in enables, today are facing even larger data volumes thanks to the proliferation of mobile phones and other devices. We found ways of handling the challenges that the volume of data posed and more than a few of these approaches would still work today, even without the additional leg up we now get from increased computing power.

But what of the other characteristics of big data? It’s not just volume that’s the challenge with big data, but the velocity too. I’ve heard some commentators discussing that it is the rate at which data arrives which is in fact the biggest issue that comes with big data. But we’ve been dealing with high velocity data for a while now as well. Complex event processing has been available in major database products for at least a few years and people working with any form of operational technology will be used to data flowing in from sensors and other protection or control devices at millisecond intervals.

So I believe that we can leverage much of what we already know and practice as data professionals to start to address the volume and velocity aspects of [structured] big data. It’s the complexity and variety aspects of big data that I think will give us the real problems we need to deal with. The problem of making, or taking, the correct meaning from unstructured data, especially that coming from outside our organisations, such as sentiment analysis from social media, and then somehow find a way to effectively integrate it with our structured data sets is where I believe we’ll find our headaches. It’s here that we’ll need innovative new tools and techniques, but even then I think they’ll rely on key areas and practices which at least some of us have worked with for a while. Metadata will play a big role here to establish the right context and data mining may well make an appearance as well.

So, in my opinion at least, we already know a thing or two about big data. Let’s not re-invent the wheel, but rather try to build on what we’ve learned in the past. It’s a novel idea I know, and perhaps not one that IT professionals have much of a proven track record with, but hopefully this type of approach might shorten the time to value for those making early investments in and around big data.

BI - It's Not Just Reporting

Many times across my career I've been frustrated by what is, in my opinion, a gross under utilisation of business intelligence tools, platforms and even functions and teams such as Business Intelligence Competency Centres. All too often significant investment has been made in purchasing software, building data warehouses and other solutions, training staff and hiring external consultants, only to have this considerable horsepower and potential being set to work on (just) "the reporting".

Building BI solutions often comes at considerable cost in both time and money and if the chief item provided is purely a set of static reports that tell us what has happened in our organisations then someone, somewhere (and probably somewhere higher up in the organisational chart) will sooner or later start to question the wisdom of the investment. And that's not just a problem for the here and now of the current project, but for the future too. If he has been tainted as to the general value of business intelligence then he may well push back against future initiatives in the area, potentially robbing the organisation of future competitive advantage or effectiveness gains. The problem is made worse still when the default approach to setting project scope is to examine the set of existing reports available from in and around the incumbent solution. In this case the company will have a shiny new toy and (maybe) a prettier set of reports, but may not have gained much in the way of additional business value or actionable insight.

We need to remember that BI offers the potential for much more than just reporting. This was true well over a decade ago and is even more true today with the technology and tool advances we've seen between then and now. Just look at the name: BI, that's Business intelligence folks - intelligence for the business. Simply knowing what happened (via a report), for the sake of compliance, audit or because the procedures say we need to have a piece of paper punched and filed on a shelf, really does little for the running of the business, let alone its advancement. BI should be about helping make decisions that better the business, that help it carry out its operations in the best way possible.

The term "decision support" may seem a bit old school, but perhaps it does capture what BI can offer to the business other than just "the reports". Recently I heard the phrase "the three tenses of BI" and it resonated with me. The first phase is historical - this is the looking back, telling us what happened, function of BI - these are the reports that so often get developed as the majority, or even the entirety, of a BI project. The second tense is the present tense. BI instruments in this area tell us what is happening right now. Reports can serve this function too, but often they miss a key component that stops them playing an effective role. Whatever is used here, be it reports, OLAP components or some type of complex event processing, the information conveyed must be provided in a form and in a time frame that allows it to be used by the recipients in a manner and a timeliness which causes a change which benefits the business that would not otherwise have occurred. The third tense of BI looks to the future - using not only internal information but also looking at wider information sets to try and predict what may happen in the future in and around the organization, thereby providing food for thought for those setting the strategic direction for the organization. What's important to note there is that the second and third tenses of BI require information and intelligence to be presented in order not just to enable decisions to be made, but perhaps also go so far as to highlight that decisions should, or even must, be made.

Simply having a great set of tools available isn't enough. If we want to move on from being perceived as just report writers then it's up to us, as BI professionals, to educate others as to the potential of BI, challenging and assisting them to find ways in which it can be better used across our organisations. So go on, take the first step - share this blog post :)

Where’s the BI Value Proposition?

I can remember a time when the pitch for BI could (almost) be made along the lines of reducing headcount – “put in this reporting solution and you’ll no longer need the team of people who currently spend all of their time compiling numbers in spreadsheets for others in the business to use”. Even when BI cost a mint this scenario still offered an organization the chance to come out in front financially. These days things aren’t so simple. Despite decreasing costs for BI the sales pitch is probably harder today than it was a decade ago. And I’m not just talking about vendors and consulting companies selling BI products and services into their prospects. I’m also talking about selling the value of BI inside companies – making the case for a BI initiative to internal funding boards is getting tougher and tougher with shrinking budgets and competition from other projects and programs.

To have the best chance of success the value proposition needs to be clear.  Often now BI projects will be seen as providing productivity support. They won’t save headcount but they may free up some percentage of time for resources throughout the company or help those within the organization make faster and / or better decisions. Proposing cost savings based upon these time savings is a dangerous endeavor. Such savings may not be viewed as real given fixed wage and salary costs or, perhaps worse still, the department may simply have next year’s budget cut by that amount to ensure the proposed benefit is locked in for realization.

Coming up with the appropriate value proposition may need you to stand back and honestly examine what’s within the scope of the proposed BI program. Is it just static reporting or are you largely replacing existing reports like for like due to a burning platform? If so then perhaps it’s time to swallow your pride and pitch the program like an infrastructure project. Chances are your client or your internal funding board looks at the criteria for assessing these types of projects differently than others, recognizing that they are unlikely to deliver strategic value, cost savings or drive top line growth, but rather just have to be done to sustain the business as usual state – to keep the lights on, if you will.

If your proposed initiative is broader and can be aligned to more tangible business objectives then you have a number of things you should be mindful of when building the BI value proposition:

·      Value is a subjective thing. Look at the perceived value of the initiative at different levels of the organization. Do they all see the value, or does your push to realize enterprise value actually make things harder for one or more departments? If you can find win-win scenarios or at least avoid win-lose scenarios then it will be easier to get agreement on the value.

·      Can you show how the value can be measured over time (and is it actually going to be possible to measure this value)? Financials are just one aspect of this. Speed of decision making, greater visibility and transparency and reduced risk might be things to consider. What ever you decide to include make sure that you can measure and articulate the current state. Not only will it give you a benchmark to measure against but you’ll also have concrete examples of the issues you are trying to solve, rather than just pitching the value of the initiative based around academic or theoretical concerns or problems that have manifested in other industries.

·      Is all of the value expected to be realized in a short to medium timeframe? Chances are this isn’t the case, especially if the project involves analytics or data mining. There likely to be a perceived drop in value whilst the design and implement stages are underway and the business resources are having to participate in workshops, field questions and test while holding down their day jobs. The perceived value is likely to spike once the initiative goes live as the low hanging fruit are harvested but chances are the perceived value will again dip as the business gets more complex and costly due to changed and changing processes, new ways of working and information overload. The real value pay-off may well not come until this stage passes and this state of anxiety and uncertainty can be resolved by working with the business to work with these new learnings and new information to further enhance the analytics and any new or changed processes to again simplify the way people work. Expect this, plan for it and build it into your value proposition. If this pushes the benefits realization out too far to the right then look for ways of chunking the program down into smaller tactical projects all building toward the strategic end game. Show the early value and paint the alignment with the strategic aims and the later value that will be realized.

And of course if you’re coming from the IT side of the fence don’t forget to work with the business through all of this. Not only are you likely to have covered off more stakeholders prior to the pitch, but chances are you’ll reap a higher value from the finished product as well.

Oh, and if you’re from the business side, bring the IT guys in on things too!! Involving them early will lead to higher longer term value.