Data Culture: Experts and Expertise

Power BI lets business users solve more and more problems without requiring deep BI and data expertise. This is what self-service business intelligence is all about, as we saw when we looked at a brief history of business intelligence.

At other points in this series we also looked at how each app needs to be treated like the unique snowflake that it is, that successful data cultures have well-defined roles and responsibilities, and that sometimes you need to pick your battles and realize that some apps and data don’t need the management and care that others do.

But some apps do.

Some BI solutions are too important to let grow organically through self-service development. Sometimes you need true BI experts who can design, implement, and support applications that will scale to significant data volumes and number of concurrent users.

In this video we look at a specific approach taken by the BI team at Microsoft that developed the analytic platform used by Microsoft finance[1].

This is one specific approach, but it demonstrates a few fundamental facts that can be overlooked too easily:

  • Building an enterprise BI solution is building enterprise software, and it requires the rigor and discipline that building enterprise software demands
  • Each delivery team has dedicated teams of experts responsible for their part of the whole
  • Each business group with data and BI functionality included in the solution pays for what they get, with both money and personnel

Organizations that choose to ignore the need for experts tend to build sub-optimal solutions that fail to deliver on stakeholder expectations. These solutions are often replaced much sooner than planned, and the people responsible for their implementation are often replaced at the same time[2].

This isn’t the right place to go into the details of what sort of expertise you’ll need, because there’s too much to cover, and because the details will depend on your goals and your starting point. In my opinion the best place to go for more information is the Power BI whitepaper on Planning a Power BI Enterprise Deployment. This free resources delivers 250+ pages of wisdom from authors Melissa Coates and Chris Webb. You probably don’t need to read all of it, but odds are you will probably want to once you get started…


After this video and post were completed but before they were published, this story hit the global news wire: Botched Excel import may have caused loss of 15,841 UK COVID-19 cases | Ars Technica (arstechnica.com)

Wow. Although I am generally a big fan of Ars Technica’s journalism, I need to object to the sub-headline: “Lost data was reportedly the result of Excel’s limit of 1,048,576 rows.”

Yeah, no. The lost data was not the result of a  capability that has been well-known and documented for over a decade. The lost data was a result of using non-experts to do a job that experts should have done.

Choosing the wrong tool for a given job is often a symptom of not including experts and their hard-earned knowledge at the phases of a project where that expertise could have set everything up for success. This is just one example of many. Don’t let this happen to you.


[1] If you’re interested in a closer look at the Microsoft Finance COE approach, please check out this article in the Power BI guidance documentation.

[2] If you’ve been a consultant for very long, you’ve probably seen this pattern more than once. A client calls you in to replace or repair a system that never really worked, and all of the people who built it are no longer around.

Data Culture: Community champions

What would an epic battle be without champions?

Lost. The epic battle would be lost without champions.

Don’t let this happen to you battle to build a data culture. Instead, find your champions, recognize and thank them, and give them the tools they need to rally their forces and lead them to victory.

Let’s do this!!

Despite what the nice short video[1] may lead you to believe, it’s not absolutely necessary to provide your data culture champions with literal swords[2]. But it is vital that you arm[3] them with the resources and connections they need to be successful.

In any community there will be people who step up to go the extra mile, to learn more than they need to know, and to do more than they are asked. These people are your champions, but they can’t do it all on their own. In the long term champions will succeed or fail based on the support they get from the center of excellence.

With support from the BI COE, champions can help a small central team scale their reach and impact. Champions typically become the primary point of contact for their teams and business groups, sharing information and answering questions. They demonstrate the art of the possible, and put technical concepts into the context and language that their business peers understand.

This is just what they do – this is what makes them champions.

An organization that’s actively working to build a data culture will recognize and support these activities. And if an organization does not…


[1] This video is about 1/3 as long as the last video in the series. You’re very welcome.

[2] But why take chances, am I right?

[3] See what I did there? I shouldn’t be allowed to write blog posts this close to bedtime.

Data Culture: All BI apps are not created equal

Every app is a unique snowflake.

From a distance many of them look the same, and even up close they tend to look similar, but each one is unique – and you cannot treat them all the same.

This post and video take a closer[1] look at the topic introduced when we looked at picking your battles for a successful data culture. Where that post and video looked at more general concepts, this video looks at specific techniques and examples used by successful enterprise Power BI customers around the world.

I won’t attempt to reproduce here everything that’s in the video, but I do want to share two diagrams[2] that represent how one organization has structured their community of practice uses Power BI, and how their Power BI COE supports and enables it. I chose this example because it hews closely to the standard successful approach I see with dozens of large organizations building their data culture around Power BI, but also puts the generic approach into a specific and real context.

This first diagram shows the “rings” of BI within the organization, with personal BI at the outside and enterprise BI on the inside. Each ring represents a specific point on the more control / less control spectrum introduced in earlier videos, and demonstrates how one large organization thinks about the consistent and well-defined criteria and responsibilities represented by points on that spectrum.

This second diagram “explodes” the inner ring to show how a given application may mature. This organization has well-defined entry points for self-service BI authors to engage with the central BI team to promote and operationalize reports and data that originate with business authors, and a well-defined path for each app to follow… but they also understand that not every app will follow the path to the end. Some apps don’t need to be fully IT-supported solutions, because their usage, impact, and value doesn’t justify the up-front and ongoing work this would require. Some do, because they’re more important.

It depends.

And the key factor that this organization – and other successful organizations like them – realizes, is that they can put in place processes like the ones illustrated above that examine the factors on which it depends for them, and take appropriate action.

On a case by case, app by app basis.

Because one size will never fit all.


[1] And longer – at nearly 23 minutes, this is by far the longest video in the series.

[2] If you’re ever feeling impostor syndrome please remember that I created these diagrams using SmartArt in PowerPoint, looked at them, and exclaimed “that looks great!” before publishing them publicly where thousands of people would likely see them.

Session resources: Patterns for adopting dataflows in Power BI

This morning I presented a new webinar for the Istanbul Power BI user group, covering one of my favorite subjects: common patterns for successfully using and adopting dataflows in Power BI.

This session represents an intersection of my data culture series in that it presents lessons learned from successful enterprise customers, and my dataflows series in that… in that it’s about dataflows. I probably didn’t need to point out that part.

The session slides can be downloaded here: 2020-09-23 – Power BI Istanbul – Patterns for adopting dataflows in Power BI

The session recording is available for on-demand viewing. The presentation is around 50 minutes, with about 30 minutes of dataflows-centric Q&A at the end. Please check it out, and share it with your friends!

 

Data Culture: Getting stakeholder buy-in

Have you ever heard the term “Excel hell”?

Odds are, if you’re reading this blog you’ve heard it, and possibly lived it once or twice. If not, you may want to take a minute to search online and discover what the internet has to say about it.

Managed self-service BI with Power BI is one way to escape or avoid Excel hell, but any self-service data tool brings with it some of the same risks that Excel brings. Power BI introduces guardrails to make it easier to manage the work that self-service authors produce, but wouldn’t it be nice to address the problem at its root, and prevent unwanted content from being shared in the first place?

The video introduces a set of techniques to gain explicit stakeholder buy-in. For the enterprise Power BI customers I work with, these steps are usually prerequisites to getting a Pro license and permission to publish to the Power BI service, but they may be required for other activities as well.

  1. Ask for it – rather than automatically issuing licenses in bulk, issue licenses only to users who explicitly request them
  2. Sign a data covenant – require users to sign “terms of use” for working with data in ways that align with the goals of the organization.
  3. Take a test – require users to take and pass a simple[1] test
  4. Complete training – require users to attend Dashboard in a Day or similar introductory training

None of these barriers is designed to keep anyone from getting access to the tools and data they need. They’re designed to make people work for it.

As mentioned in earlier posts and videos, humans are complicated and tricky, but most people value what they earn more than they value what they’re given. And if someone works to earn the ability to publish and share data and reports, they’re more likely to think twice before they publish something and forget it.

This is a small but effective step that can reduce the ongoing effort required to manage and govern the content published to the Power BI service. And if you put the entry point[2] to requesting access in your central portal, you’ll be helping reinforce the behaviors that will make your data culture grow, right from the beginning.


[1] Emphasis here on “simple” – every company I talked to who used this approach designed the test so that anyone could pass it.

[2] Power App, Customer Voice form, ServiceNow ticket, whatever fits your processes and requirements.

Data Culture: Now you’re thinking with portal

In an ideal world, everyone knows where to find the resources and tools they need to be successful.

We don’t live in that world.

I’m not even sure we can see that world from here. But if we could see it, we’d be seeing it through a portal[1].

One of the most common themes from my conversations with enterprise Power BI customers is that organizations that are successfully building and growing their data cultures have implemented portals where they share the resources, tools, and information that their users need. These mature companies also treat their portal as a true priority – the portal is a key part of their strategy, not an afterthought.

This is why:

In every organization of non-trivial size there are obstacles that keep people from finding and using the resources, information, and data they need.

Much of the time people don’t know what they need, nor do they know what’s available. They don’t know what questions to ask[2], much less know where to go to get the answers. This isn’t their fault – it’s a natural consequence of working in a complex environment that changes over time on many different dimensions.

As I try to do in these accompanying-the-video blog posts I will let the video speak for itself, but there are a few key points I want to emphasize here as well.

  1. You need a place where people can go for all of the resources created and curated by your center of excellence
  2. You need to engage with your community of practice to ensure that you’re providing the resources they need, and not just the resources you think they need
  3. You need to keep directing users to the portal, again and again and again, until it becomes habit and they start to refer their peers

The last point is worth emphasizing and explaining. If community members don’t use the portal, it won’t do what you need it to do, and you won’t get the return you need on your investments.

Users will continue to use traditional “known good” channels to get information – such as sending you an email or IM – if you let them. You need to not let them.


[1] See what I did there?

[2] Even though they will often argue vehemently against this fact.

Standardizing on Power BI

Last week’s post on migrating to Power BI was intended to be a stand-alone post, but this excellent comment from Matthew Choo via LinkedIn made me realize I had more to say.

I could not agree more, Matthew![1]

In my conversations with leaders from enterprise Power BI customers, if they mention to me that they’re standardizing on Power BI as their analytics tool and platform, I try to ask two questions:

  1. Why?
  2. Why now?

The answer to the first question is almost always a combination of reasons from the previous post on migration and the fact that the customer sees Power BI as the best choice for the organization’s needs moving forward. There’s very little variation in the answers over dozens of conversations.

The answers to the second question are more diverse, but there are some common themes that I hear again and again.

One theme is that the selection of multiple tools happened organically. Sometimes a new BI tool is adopted through a merger or acquisition. Sometimes a new CIO or other senior leader mandates the adoption of their favorite tool, and as leaders change the old tools are left behind while the new tools are added to the stable. Sometimes a key data source only works well with the data source vendor’s reporting tool because the vendor refuses to open their APIs to 3rd parties.[3] Often the plan has been to eliminate excess tools at some point, but the point hasn’t been now… until now.

Very often the factor that makes standardization a priority is pain, which brings me back to a point I made in the introductory post in the “building a data culture” series:

I strongly believe that pain is a fundamental precursor to significant change. If there is no pain, there is no motivation to change. Only when the pain of not changing exceeds the perceived pain of going through the change will most people and organizations consider giving up the status quo.

The most common general reasons for this “pain balance” shifting[4] involve money. Organizations need to eliminate inefficiencies so they can invest in areas of strategic importance. A leader may proactively seek out inefficiencies to eliminate, but it’s more typical for me to hear about external market pressures[5] necessitating a more reactive consolidation.

The other common theme in responses to the question “why now?” is that the organization has a new chief data officer, that the CDO is focused on building a data culture, and for the reasons listed in last week’s post has has made consolidation a priority.

What’s interesting about this theme is that the hiring of a CDO is part of a larger strategic shift in how the organization thinks about data. The C-level executives know that they need to start treating data as a strategic asset, and they realize that it’s too important to be rolled up into the responsibilities of the CIO or another existing leader. Very often, they already have a good idea of the types of changes that need to happen, but want to hire a senior leader who will make the final decisions and own the actions and their outcomes. In other words, the hiring of a CDO is often an early-but-lagging indicator that there’s executive support for a data culture. That’s a good thing.

Before making a big change always important to understand what you hope to achieve, but it’s also important to take a little time to examine how you got to the point where change is necessary, so you can better avoid the mistakes of the past…


[1] Usually when I say this I’m agreeing with myself[2] so it’s nice to be referring to a different Matthew here.

[2] Not that i agree with myself very often, mind you.

[3] Yeah, you know who I’m talking about.

[4] I wasn’t expecting to coin this phrase, but as soon as I typed it, I loved it. I think I may quit my day job and start a new business as a consultant delivering expensive “Shifting the Pain Balance(tm)” workshops for chief data officers and other senior executives.

[5] Since it’s still 2020, I should point out that the COVID-19 has been at the root of many external market pressures. I’ve heard dozens of companies say that they’re increasing their investment in building a data culture because of pandemic-induced challenges.

Data Culture: Showcasing the Art of the Possible

The last post and video in this series looked at the broad topic of training. This post looks at as specific aspect of this topic: letting people know what is possible, and sparking their imagination to do amazing things.

A lot of content and training materials will focus on capabilities: here is a feature, this is what it does, and this is how you use it. Although this type of content is important, it isn’t enough on its own to accelerate the growth of a data culture.

The most successful organizations I’ve worked with have included in their community of practice content specifically targeting the art of the possible. This might be a monthly presentation by community champions across the business. It might be someone from the center of excellence highlighting new features, or the integration between features and tools. The most important thing is planting the seed of an idea in the minds of people who will say “I had no idea you could do that!”

My colleagues Miguel and Chris are some of my greatest personal sources of inspiration for building reports[1] because each of them does amazing things with Power BI that make it powerful, usable, and beautiful – but they’re just two of the many people out there showing me new techniques and possibilities.

Who will you inspire today?


[1] And by now you probably realize that I need all the inspiration I can get for anything related to data visualization.

Migrating to Power BI

One aspect of building a data culture is selecting the right tools for the job. If you want more people working with more data, giving the tools they need to do that work is an obvious[1] requirement. But how many tools do you need, and which tools are the right tools?

Migrating to the cloud

It should be equally obvious that the answer is “it depends.” This is the answer to practically every interesting question. The right tools for an organization depend on the data sources it uses, the people who work with that data, the history that has gotten the organization to the current decision point, and the goals the organization needs to achieve or enable with the tools it selects.

With that said, it’s increasingly common[2] to see large organizations actively working to reduce the number of BI tools they support[3]. The reasons for this move to standardization are often the same:

  • Reduce licensing costs
  • Reduce support costs
  • Reduce training costs
  • Reduce friction involved in driving the behaviors needed to build and grow a data culture

Other than reducing the licensing costs[4], most of these motivations revolve around simplification. Having fewer tools means learning and using fewer tools. It means everyone learning and using fewer tools, which often results in less time and money spent to get more value from the use of those tools.

One of the challenges in eliminating a BI tool is ensuring that the purpose that tool fulfilled is now effectively fulfilled by the tool that replaces it. This is where migration comes in.

The Power BI team at Microsoft has published a focused set of guidance articles focused specifically on migrating from other BI tools to Power BI.

This documentation was written by the inestimable Melissa Coates of Coates Data Strategies, with input and technical review by the Power BI customer advisory team. If you’re preparing to retire another BI tool and move its workload to Power BI – or if you’re wondering where to start – I can’t recommend it highly enough.


[1] If this isn’t obvious to a given organization or individual, I’m reasonably confident that they’re not actively trying to build a data culture, and not reading this blog.

[2] I’m not a market analyst but I do get to talk to BI, data, and analytics leaders at large companies around the world, and I suspect that my sample size is large and diverse enough to be meaningful.

[3] I’m using the word “support” here – and not “use” – deliberately. It’s also quite common to see companies remove internal IT support from deprecated BI tools, but also let individual business units continue to use them – but also to pay for the tools and support out of their own budgets. This is typically a way to allow reluctant “laggard” internal customer groups to align with the strategic direction, but to do it on their own schedules.

[4] I’m pretty consistent in saying I don’t know anything about licensing, but even I understand that paying for two things costs more than paying for one of those things.

Data Culture: Training for the Community of Practice

The last few posts and videos in this series have introduced the importance of a community where your data culture can grow, and ways to help motivate members of the community, so your data culture can thrive.

But what about training? How do we give people the skills, knowledge, and guidance that they need before they are able do work with data and participate in the data culture you need them to help build?

Training is a key aspect of any successful data culture, but it isn’t always recognized as a priority. In fact the opposite is often true.

I’ve worked in tech long enough, and have spent enough of that time close to training to know that training budgets are often among the first things cut during an economic downturn. These short-term savings often produce long-term costs that could be avoided, and more mature organizations are beginning to realize this.

In my conversations with enterprise Power BI customers this year, I’ve noticed a trend emerging. When I ask how the COVID-19 pandemic is affecting how they work with data, I hear “we’re accelerating our efforts around self-service BI and building a data culture because we know this is now more important than ever” a lot more than I hear “we’re cutting back on training to save money.” There’s also a clear correlation between the maturity of the organizations I’m talking with and the response I get. Successful data cultures understand the value of training.

I’ll let the video speak for itself, but I do want to call out a few key points:

  1. Training on tools is necessary, but it isn’t enough. Your users need to know how to use Power BI[1], but they also need to know how to follow organizational processes and work with organizational data sources.
  2. Training material should be located as close as possible to where learners are already working – the people who need it the most will not go out of their way to look for it or to change their daily habits.
  3. There is a wealth of free Power BI training available from Microsoft (link | link | link) as well as a broad ecosystem of free and paid training from partners.

The most successful customers I work with use all of the resources that are available. Typically they will develop internal online training courses that include links to Microsoft-developed training material, Microsoft product documentation, and community-developed content, in a format and structure[2] that they develop and maintain themselves, based on their understanding of the specific needs of their data culture.

Start as small as necessary, listen and grow, and iterate as necessary. There’s no time like the present.


[1] Or whatever your self-service BI tool of choice may be – if you’re reading this blog, odds are it’s Power BI.

[2] I’m tempted to use the term “curriculum” here, but this carries extra baggage that I don’t want to include. Your training solution can be simple or complex and still be successful – a lot of this will depend on your company culture, and the needs of the learners you’re targeting.