Problem Domain, Solution Domain

I’ve been thinking about problem domains and solution domains a lot lately. I’ve posted on this topic before, but the more I think about it, the more I think I should explore it more. Starting now.

Image of interlocking puzzle piece hearts from https://pixabay.com/illustrations/puzzle-heart-love-two-hearts-1721619/
When the right problems meet the right solutions, it can be magical

Let’s begin by defining our terms.[1]

A problem domain is a subject area where people work.

If you’re a business intelligence or data professional, the problem domains of interest are often a business function like finance, supply chain or HR. The problem domain experts[2] are typically people who are work in one of these fields, and who might come to you looking for solutions, or for help building solutions.

A solution domain is also a subject area where people work.

If you’re a business intelligence or data professional, the solution domains of interest are often some combination of data visualization, data modeling, data transformation, and so on. They may also be DAX, Power Query, Power BI, or another specific set of tools and technologies.  The solution domain experts[3] are typically people who build data and BI applications and systems to solve problems in other problem domains.

On the other hand, if you’re a member of the Power BI product team[4], the solution domain you’re working in is commercial software development – and the problem domain of interest is building, deploying, monitoring, and/or administering Power BI solutions. Everything is relative, and whether a given subject area is a problem domain or a solution domain is a function of the context in which it is being evaluated.

Let’s pause to let that sink in for a minute. None of the information above is particularly new, and it may not seem profound at first glance, but these two terms are some of the most foundational concepts of building a data culture.

A successful and mature data culture is the product of the right people doing the right things with the right data as part of the right processes.[5] This means that a successful and mature data culture involves solution domain experts and problem domain experts having healthy partnerships and mutual respect… which is also a foundational concept that sounds simple until you look at it more closely.

If you think about the traditional relationship between business and IT, “partnership” probably isn’t the first word that leaps to mind. All too often this relationship is characterized by conflict and a lack of mutual respect that is in part a function of misaligned priorities. Like many entrenched conflicts it is also partly a function of history and  the mistrust produced by historical wrongs – actual or perceived.

Most interesting things – interesting conversations, interesting projects, interesting jobs and careers – exist at the intersection of the problem domain and the solution domain. Interesting things happen at the edges where one thing ends and another thing begins. This is where complexity is found, because multiple domains are involved and making sense requires experts in each domain.

Unfortunately this is also where things tend to go wrong. Too often things fall into the cracks between the problem domain and the solution domain. Experts in one domain don’t value the other part of the picture, or they don’t see it as their job, or they assume that someone else will figure that part out… or maybe it’s none of these things, and they just lack the vocabulary and the context to close that gap.

Please take a moment to think honestly and critically about the problem domains and solution domains in which you operate, and your relationships with the folks in other domains with whom you interact. This is all I have for today, and although I don’t know exactly where I’m going down this path, I know I’m not done – and I know a little introspection can go a long way.


[1] These are my personal definitions that I’m making up as I write this post. You’ll find other definitions if you’re willing to go looking, and although those definitions will align broadly with these, they will have different emphasis and context and nuance because they’re not mine.

[2] The subject matter experts in a given problem domain.

[3] The subject matter experts in a given solution domain.

[4] Or another team building commercial tools for data and analytics.

[5] And the state in which these things are the norm and the exception.

Tabular Editor and “Desktop Hardening”

Back in June[1] the guys in cubes at Guy in a Cube were joined by Daniel Otykier and Marco Russo for a live stream on YouTube. Daniel is the creator of Tabular Editor, a powerful tool for developing Analysis Services Tabular data models and Power BI datasets. Marco is one of the Italian Data Models[2] behind SQLBI.com, and is a recognized expert on tabular development and DAX.

This special live stream was a Q&A session with Daniel and Marco about the then-new announcement of Tabular Editor 3, an improved and expanded paid version of what had previously been a free, community-supported too. I watched the stream to learn what the excitement was all about, and to see if there were any off-topic questions[3] I could answer in the chat.

Quite a few questions came up where the term “desktop hardening” was part of answer. Questions like this one:

I strongly encourage you to watch the live stream replay for the questions and discussion. Daniel does an excellent job of describing how Tabular Editor uses Microsoft’s Tabular Object Model library to modify tabular data models, and some of the complexities of what’s allowed or not allowed depending on where a given tabular model is located – and I’m not going to attempt to replicate his wisdom here.

What I am going to do here is talk a little about “desktop hardening,” which is a term I’ve heard hundreds of times in meetings and work conversations[5], but which I discovered is practically undefined and unmentioned on the public internet. The only place where I could find this term used in context was in this reply from Power BI PM Christian Wade… and if you don’t already know what “desktop hardening” is, seeing it used in context here isn’t going to help too much.

Here is my definition of desktop hardening:

Desktop hardening is the internal shorthand used by members of the Power BI product team to describe a broad set of ongoing investments to expose the full contents of the Power BI Desktop PBIX file format as a supported public API.

There are a few things worth picking apart in this definition. Let’s start with “public API,” and let’s start by clarifying I don’t mean “REST API” or “.NET SDK” or anything of the sort. In this post I’m using the broadest possible definition of the term API, which to me is “a set of functionality that can be accessed programmatically.” More on this below.

As this excellent post from API Academy highlights, a public or “open” API “is an interface that has been designed to be easily accessible by the wider population of… developers” and that “opening an interface to external developers can significantly add to the management and security challenges associated with publishing APIs. For example, with many third-party client apps active in the field, it can be very challenging to ensure that interface updates do not break application functionality.”

Conversely, “Private APIs can significantly reduce the development time and resources needed to… build new systems that maximize productivity and create customer-facing apps that extend market reach and add value to existing offerings.”

In my experience, a private API can be updated with minimal or moderate effort. By definition it’s only being called by developers in your organization, so if you need to make a change you let them know and they update their client code[6]. On the other hand, updating a public API is very complicated and very costly, especially when there are legal and/or regulatory requirements that need to be met. When updating a public API you need to worry about a thousand different things, from documentation to backward compatibility. These considerations make every aspect of maintaining the API slower, more complicated, and more costly. Forever.

Now that we’ve looked at the “public API” part of the definition, let’s also look at the “Power BI Desktop PBIX file format” part.

The Power BI Desktop authoring tool uses the PBIX file format, which is implemented similar to the Open XML format introduced in Office 2007. Each PBIX file is basically a Zip archive that contains a set of folders and XML and JSON files. Each PBIX file contains the tabular data model, reports, Power Query mashups, and related metadata for the a Power BI solution. MVP Gilbert Quevauvilliers of FourMoo explored the contents of the PBIX format if you’re interested in taking a closer look.

Now let’s look at the “ongoing investments to expose the full contents” part of the definition.

As of today, only a limited number of operations on the tabular data model in a PBIX file are supported for write operations. These operations were exposed as part of the Power BI team’s addition of external tools support to Power BI Desktop, and are documented as part of the docs for external tools.

Today if a given operation isn’t on that list, it’s not supported for use by any client other than Power BI Desktop. If you edit another part of a PBIX, directly it might be OK, but it might not – your edit might make the PBIX unable to be opened by Power BI desktop. And even if a given operation might work today, it might not work tomorrow. That’s how private APIs work – the API owner can make changes at any time, and any client application or process that relies on the unsupported private API may be broken.

I should also point out that the scope of desktop hardening isn’t limited to external tools like tabular Editor. It also includes:

  • The ability for Power BI Desktop to accept dataset definitions that include features that currently cannot be built in Desktop, such as Partitions and DetailRows.
  • The ability to download a dataset defined or modified using the XML endpoint in the Power BI service as a PBIX file or PBIT template.
  • The ability for Power BI Desktop to accept dataset definitions that are produced as part of a git workflow or similar process where a developer might programmatically “merge” changes from multiple branches.[7]

The Power BI team has a goal to provide a public API for everything in the PBIX file (that’s what “desktop hardening” means, remember?) but it’s a long-term goal. You can keep an eye on the public roadmap at https://aka.ms/PBIReleasePlan, but I expect the full vision of desktop hardening to be delivered incrementally over time, rather than in one big release. Desktop hardening isn’t a feature – it’s a bunch of related features and non-feature investments. Because of the ongoing implications of having a public API, it’s important to do things right, and the complexity of this effort cannot be overstated.

That brings us to the final part of the definition I want to explore: the term “desktop hardening” is an “internal shorthand” used by the product team. This isn’t a term that is ever likely to show up in official Microsoft documentation, but it’s become part of the team vocabulary, and has unfortunately leaked into public community conversations as well… sort of like an unsupported API.

I hope that the next time you hear someone talking about desktop hardening you’ll have a better idea of what this term means, and I hope that the next time you hear someone asking about it you can direct them to this post.


[1] Back in June when I started writing this post, which might give you an idea of how my 2021 has been going.

[2] Think about it. “Marco” and “Alberto” have become mononyms not unlike “Fabio”. Also, they’re all Italian, they’re all absolutely beautiful people, and their names always come up when you’re talking about modeling. Seriously, I’m right and you know that I’m right.

[3] Guy in a Cube live streams are usually more “ask me anything” Q&A sessions rather than focused on a specific topic, and there are always more questions than there’s time for answers. Adam and Patrick were kind enough[4] to make me a moderator of their channel, so I do what I can to help out.

[4] Foolish enough.

[5] This is where I add the standard disclaimer that although I am a Microsoft employee and a member of the Power BI CAT team, this is my personal blog and all information here is my opinion and does not necessarily reflect the views of my employer or any of my cats, etc., etc.

[6] Somewhere an engineering manager is screaming and/or crying as they read this. Updating private APIs can indeed be complicated and costly – just not when compared with updating a public API of similar scope and complexity. When you’re building, shipping, and maintaining software that is used by millions of people around the world, nothing is actually simple.

[7] Thanks a million to the engineering manager who took time out of his busy day to provide constructive criticism on an earlier draft version of this post, and to let me know that he felt my definition was too constrained, and that my examples were too sparse. This list of additional examples is based on his feedback – thank you, Akshai!

Please help make Microsoft Teams more accessible

If you’re in a rush, please skip reading the post and instead vote on these two Microsoft Teams feedback items:

  1. “Please stop blinking” – requesting an option to disable the near-constant blinking of the taskbar button for the Windows app for Teams
  2. “Request User-Level Option to Disable GIF Autoplay” – this one is pretty self-explanatory

Thank you for your votes and comments!

Now on with the post…

I’ve blogged previously about the importance of diversity, including the importance of having a diverse team when building software. I’ve even posted about  specific accessibility challenges with the collaboration tool many of use every day – Microsoft Teams.

And in this context, I am requesting your help.

Last week I shared this quick Twitter poll, and over the course of a day I learned that my challenges with Teams’ default behavior – turning the taskbar button orange and blinking every time there’s activity in the app – are not just mine. Nearly 100 people responded to the poll, and over two thirds of them identified this behavior as a challenge for them as well[1].

For me, this constant flashing is like having someone scream at me all day along. If I’m having a challenging anxiety day, it’s even worse. Judging from the replies on Twitter, I’m not alone in this regard either. And because more and more functionality is being delivered via Teams, it’s not like we can simply close the app to make the blinking stop.

I’ve discovered an undocumented and unsupported hack that appears to solve this problem for some users, but it doesn’t appear to work for everyone. Also, it’s an awful hack that’s about as user-friendly as watching YouTube on your Peloton bike…

If this behavior is a problem for you, please take a few moments to vote up on this feedback item, and to add a personal comment as well. The Teams product team needs to hear from their users to understand what’s important, and this is apparently were they’re listening.

Thank you!


[1] This is a pretty vague interpretation of the poll results, so I’ve included the Twitter poll so you can keep me honest.

 

Rebranding Data Governance

To successfully implement managed self-service business intelligence at any non-trivial scale, you need data governance. To build and nurture a successful data culture, data governance is an essential part of the success.

Despite this fact, and despite the obvious value that data governance can provide, data governance has a bad reputation. Many people – likely including the leaders you need to be your ally if you’re working to build a data culture in your organization – have had negative experiences with data governance in the past, and now react negatively when the topic of data governance is raised.

They now treat data governance as a four-letter word.

Stakeholders’ past experiences can make your job much more difficult as you attempt to work with them to enable managed self-service within an organization. Governance is what they need. Governance is what you want to help them achieve. When you say “governance” you’re thinking about erecting guardrails rather than roadblocks, about making it easier for people to do the right things when working with the right data… but that’s not what they hear.

The label shouldn’t really matter – but it does.

Data governance, and building a data culture in general, is as much about people as it is about processes and technology, and that means effective communication is key. Effective communication requires a shared vocabulary, and a shared understanding of the meaning of key words.

It may be time to think about rebranding. Not unlike how a corporation with a reputation for all the wrong things might change its name in an effort to leave all those negative connotations behind without really changing its ways, maybe we need to rebrand data governance… at least some of the time.

My idea when starting to write this post was to propose “Data Culture Enablement” as the more friendly label, but as I was searching around for related content[1], I found that Dan Sutherland of EY proposed something simpler: Data Enablement.

Dan’s post is coming at things from a different angle, but it’s clear he has a similar goal in mind: “emphasiz[ing] empowerment, innovation and instant business value consumption.”

While I was searching, I found a few more interesting articles out there – they’re all well worth your time to read if you’ve made it this far:

Finally, in an interview with Data Quality Pro, data governance coach Nicola Askham emphasized the positive aspects of data governance:

People think governance is about somebody with a big stick but it’s not. It’s about getting people to communicate and talk about their data and being in a position to ask for what they need with their data. The people on the other end need to understand they have a responsibility to meet that requirement if possible.

But when asked explicitly about coming up with a new term, she replied “For a while, I toyed with the idea of starting a campaign to re-name it but I didn’t think it was worth adding to the confusion surrounding the term by coming up with another title.”

She’s probably right.

But because of the negative connotations that the term “data governance” carries today, we should all exercise care when we use it. We should be careful to ensure that the meaning we’re trying to convey is clearly received – regardless of the terms we’re using.

That feels like an ending, but I’m not done. I want to close with story.

Five years ago I was working on a data governance product, and as part of that work I talked with lots of customers and with Microsoft customer-facing employees. In those conversations I frequently heard people say things to the effect of “don’t use the ‘G word’ when you’re meeting with this leader – if you say ‘governance’ he’s going to stop listening and the meeting will end.” This didn’t happen in every conversation, but it happened in many.

Last month I hosted a multi-day NDA event with senior technical and business stakeholders responsible for adopting and implementing Power BI and Azure Synapse at some of the biggest companies in the world. The event was focused on Power BI and Synapse, but the customer representatives kept bringing up the importance of data governance in session after session, and conversation after conversation[3]. It was like night and day compared to the conversations I had when I was trying to get people to care about governance.

Has the world changed that much, or am I just talking to different people now?

I think it’s probably a bit of both. The world has definitely changed – more and more organizations are recognizing the value and importance of data and are increasingly treating data as an asset that needs to be managed and curated. But these days I also engage mostly with organizations that are atypically mature, and are further along on their data culture journey than most. With this selection bias, it’s probably not surprising that I’m having a different experience.

I’ll close with a question and a thought: Who are you talking to about data governance?

Before you begin any data governance conversation, do your best to have a clear understanding of your audience and your goals in communicating with them. No matter what terms you use, it will make all the difference.


[1] I’m always terrified that I will post something that someone else has already posted. On more than one occasion I’ve completed and published a new post, only to find out that I had written the same thing months or years earlier on this blog. Sigh.

[2] I’m pretty sure I’ve referenced this post before. It’s good enough to reference again. You should read it again.

[3] Thankfully we had also included some key folks from the Azure Purview team as well.

Data Culture and Digital Transformation

One of the key success factors for organizations to thrive today is adopting modern self-service business intelligence tools and transforming their businesses to become more agile, more automated, and more data driven. For years technology vendors and industry analysts have thrown around the term “digital transformation” to broadly describe this phenomenon, and technology has matured to catch up with the hype.

Cheesy graphic of zeroes and ones - Image by Gerd Altmann from Pixabay

I use the term “hype” here deliberately. In my experience the term “digital transformation” has been thrown around in the same way as the terms “cloud” and “big data” were thrown around, just a few years later. The cynical part of my brain initially categorized it as “marketing bullshit,” but the intervening years have shown me that this wasn’t actually the case. Digital transformation is real, and it’s a key driver for a successful data culture with real executive support.

Over the past few years I’ve had hundreds of conversations with executives, decision-makers, and business and technical leaders from hundreds of enterprise Power BI customer organizations. These are real people working to solve real problems, and they come from a broad range of industries, geographies, and levels of maturity. I learned a lot from these conversations, and have done my best to help Power BI improve based on what I learned[1], but when I step back and look at the bigger picture there’s a significant trend that emerges.

Stakeholders from organizations that adopt Power BI[2] as part of a digital transformation describe more mature data cultures, and a greater return on their investments in data and analytics.

As you can probably imagine, once I saw this correlation[3], I kept seeing it again and again. And I started looking more closely at digital transformation as part of my ongoing work around data culture. Two of the most interesting resources I’ve found are articles from the Harvard Business Review, which may not be the first place you think to look when you’re thinking about Power BI and data culture topics… but these two articles provide important food for thought.

The first article is almost six years old – it’s from 2015, and focuses on The Company Cultures That Help (or Hinder) Digital Transformation. In the article, author Jane McConnell describes five of the most difficult obstacles that prevent organizations from adopting the changes required by a digital transformation:

(Please feel strongly encouraged to click through and read the whole article – it’s well worth your time, and goes into topics I won’t attempt to summarize here.)

I suspect these challenges sound as depressingly familiar to you as they do to me. These obstacles weren’t new in 2015, and they’re not new now – but they’re also not going away.

Jane McConnell goes on to identify what characteristics are shared by organizations that have overcome these obstacles and are succeeding with their digital transformations. The alignment between her conclusions and this blog’s guidance for Building a data culture with Power BI is striking[4]:

  • A strong, shared sense of purpose alleviates many obstacles, especially those of internal politics. When an organization has a clearly defined strategy, it is easier for everyone to align their work towards those strategic goals, and to justify that work in the face of opposition.
  • Freedom to experiment helps people prioritize, make decisions, and rethink how they work. Having a culture where data governance takes the form of guardrails and effective guidance, not roadblocks, is key to driving meaningful positive change.
  • Distributed decision-making gives people at the edges of organizations a voice in digital transformation. Although there is a need for centralized decision-making and control for some aspects of a data culture (data sources, applications, policies, etc.) the real power of managed self-service BI comes from letting IT do what IT does best, and letting business experts make informed business decisions without undue governance.
  • Organizations that are responsive to the influence of the external world are more likely to understand the value digital can bring. My customer engagements don’t provide any insight to share on this specific point, but I suspect this is significant too. Organizations that are not responsive to external factors are unlikely to make it onto my calendar for those strategic conversations.

In her conclusion, Jane McConnell suggests that readers who see these obstacles in their way should “find ways to transform your work culture using digital as a lever.” In the context of the Harvard Business Review’s target audience, this advice makes a lot of sense.[5] If you are a senior business leader, shaping the work culture is something you are empowered and expected to do. If you’re not, this is where having an engaged and committed executive sponsor will come in handy. If you don’t already have that sponsor, framing your conversations using the vocabulary of digital transformation may help in ways that talking about data culture might not.

The second HBR article I found valuable and fascinating in the context of digital transformation and data culture is written by Randy Bean and Thomas H. Davenport and titled Companies Are Failing in Their Efforts to Become Data-Driven. This one is more recent, and focused more closely on the data side of things.

(As with the article discussed above, please feel encouraged to click through and read this one too. Both articles are written by professionals with significant experience, and an informed strategic perspective.)

This article starts off with a excellent statement of fact:

Whether their larger goal is to achieve digital transformation, “compete on analytics,” or become “AI-first,” embracing and successfully managing data in all its forms is an essential prerequisite.

It then goes on to inventory some of the ways that organizations are failing to deliver on this essential prerequisite, including “72% of survey participants report that they have yet to forge a data culture.”[6]

I’ll let you read the source article for more numbers and details, but there is one more quote I want to share:

93% of respondents identify people and process issues as the obstacle.

If you’ve attended any of my “Building a data culture with Power BI” presentations, you’ll know that I break it down into two main sections: the easy part, and the hard part. Spoiler alert: The easy part is technology. The hard part is people.

The article by Bean and Davenport includes a lot of insights and ideas, but not a lot of hope. They’ve talked to senior data leaders who are trying various approaches to build data cultures within their enterprise organizations, but they all see a long march ahead, with hard work and few quick wins. Technology is a vital part of the transformation, but people and culture is necessary as well.

Building a successful data culture requires top-down and bottom-up change. If you’re in a position of authority where you can directly influence your organization’s culture, it’s time to roll up your sleeves and get to work. If you’re not, it’s time to start thinking about the changes your can make yourself – but it’s also time to start thinking about how using the vocabulary of digital transformation might help you reach the senior leaders whose support you need.


[1] This is your periodic reminder that although I am a member of the Power BI Customer Advisory Team at Microsoft, and although I regularly blog about topics related to Power BI, this is my personal blog and everything I write is my personal perspective and does not necessarily represent the views of my employer or anyone other than me.

[2] I assume that this holds true for other modern BI tools, but since I’m only talking to Power BI customers I can’t really say. And since Power BI continues to increase its lead over the competition…

[3] Not causation.

[4] The bolded text in this list is taken from the HBR article; the rest of the text is from me.

[5] Even if the use of “digital” is sooooo 2015.

[6] Now I wish that I had found this article before I started my data culture series, because I definitely would have used “forge” instead of “build” as the verb everywhere.

The Blind Men and the Data Elephant

Are you familiar with the parable of the blind men and the elephant?

elephant-skin-245071_640
Would you have known this was an elephant, just from this picture?

Sometimes it seems to me that the world of business intelligence – and self-service business intelligence in particular – is made up of blind men feeling their way around a elephant made up of technology and people, each of them feeling something different but none of them able to effectively communicate to the others what they are experiencing.

This blog post was inspired by this email, sent by a technical job recruiter:

Data leader recruiter email
What does this email tell you about the role and the company?

More specifically this blog post was inspired by a tweet[1] from Power BI consultant Jeff Weir, in which he shared this recruiter’s email:

2020-01-18-14-16-43-964--msedge

In this conversation Jeff and I each took the role of a blind man, with this data analytics manager position playing the elephant. Jeff and I each reached very different conclusions.

Jeff may have[2] concluded that Power BI as a tool wasn’t ready for the job, or that that this company wasn’t fully committed to doing what it takes to build a successful data culture, including the right training for their Power BI creators and consumers.

I reached a different conclusion, and I the more I read and re-read the recruiter’s email the more I believed that my initial conclusion is correct: this is an organization that is stepping back from solutions to ensure that they’re focused on the right problems. This is an organization that already has tactics, but is willing to admit that those tactics aren’t getting them where they need to go without the right strategy. This is an organization that seems ready to make significant, substantive change.

Why did I reach this conclusion?

The first sentence: “Ultimately Power BI while being a tool they are currently using it is not the only tool they will consider.”[3]

Wow. As an organization they are avoiding the sunk cost fallacy and explicitly stating they’re open to change, even though that change is likely to be disruptive and expensive. Switching BI tools is hard, and it is not a change that is taken lightly. The leaders at this organization are willing to take that hit and make that change, and they’re taking the steps to find the right leader to help them make it… if that leader decides the change is necessary.

The second sentence, with my emphasis added: “Here they are really looking for someone with some strong technical ability, but more so strong leadership and strategy skills.[3]

Wow!  The recruiter is explicitly calling out the organization’s priorities: they need a strategic leader. They need someone to help them define the strategy, and to lead them in its execution. Yes, they need someone with the technical chops to help carry out

The third sentence: “there will be an element of helping there and mentoring the staff members currently in the team.”

Maybe not exactly wow on this one, but it’s clear that the organization knows that they need to build the skills of their current BI team, and they want someone able to help. Something missing here that I would prefer to see[4] is additional information about the size and skill levels of the team today, and what other learning resources the organization makes available.

The third sentence, again: “it is more about getting a grip on the current landscape of the data and what can be done with it, setting a vision and a strategy aligned with product development and then going from there…”

WOW! This sounds like an organization that knows their current path isn’t leading them to success. It sounds like an organization that acknowledges its challenges and is proactively looking for a strategic solution to root causes, not just a band-aid.

But… it also sounds like an organization that needs a Chief Data Officer, not just an Insights and Data Analytics Manager, and based on Jeff’s Twitter comment it sounds like they’re not willing to pay a competitive rate for the less senior position, much less the more senior one. They seem to want parts of a data culture, but don’t  appear willing to invest in what it will actually take to make it a reality

I was originally planning to structure this post around the theme of 1 Corinthians 13:11, but the more I thought about it the more I realized that describing a technical solutions focus as being “childish ways” that you then grow out of really didn’t match up with what I was trying to say.[5] Although to be fair, this post has traveled far afield from what I intended when I started writing, and I’m honestly not certain I ended up making a particularly strong point despite all the words.

Maybe if I’d had more time I could have just said “different people see different things in shared situations because of differences in context, experience, and priorities, and it’s important to take multiple perspectives into consideration when making important decisions.” Maybe not. Brevity has never been my forte.


[1] Please feel encouraged to read the whole thread, as it went in a few different directions that this post doesn’t touch on directly. I also did explicitly ask Jeff’s permission to reference the conversation and image in a blog post… although I doubt either one of us expect it to arrive quite as late as it did.

[2] I qualify my summary of Jeff’s conclusion because I’m only working with the information available to me in the Twitter conversation, and I do not know that this is a complete or accurate summary. I suspect it is at least accurate, but only Jeff can say for sure.

[3] Please pause for a moment to reflect on how difficult it was to type in that sentence without editing it for clarity and grammar.

[4] Yes, I understand that this is an excerpt from one email that shows only the most narrow slice into what is likely a complex environment, and that it is not realistic to expect every aspect of that complexity to be included in three sentences – no matter how long that final sentence might be.

[5] Thanks to everyone who shared their thoughts on Twitter about the wisdom of using a Bible verse in a technical blog post.

Power BI, PowerPoint, and the importance of shared goals

Do you ever get the feeling that you you’re participating in a completely different conversation than the person you’re talking to?

Maybe you are.

Image by Arek Socha from Pixabay
An actual photograph of Matthew’s brain most of the time.

Back in December[1], Power BI MVP Lars Schreiber shared this tweet:

I’ll bet that was a difficult conversation.

Although I wasn’t in the room with Lars and his client, I’ve been in similar rooms having similar conversations on many occasions where it gradually became obvious that my understanding of the meeting’s goals were not shared by the other participants.

Without a shared understanding of goals and priorities, collaboration quickly breaks down.

Without a shared understanding of goals and priorities, activity are falsely equated with progress – for how can you make progress without a chosen destination?[3]

Without a shared understanding of goals and priorities at the beginning, much of what is done will need to be undone and/or redone once the disconnect is identified and rectified. Often it’s not just a matter of going back to the start – it’s worse. The parties that argued against the project[4] the first time will be empowered and emboldened by any delays and false starts, and the greater the impact of the misunderstanding, the more difficult it will be to get things back on track.

In my experience the best thing to do in situations like this is to immediately step back, stop focusing on implementing a solution to a problem, and focus instead on clearly defining the problem that needs to be solved. Once you know not everyone is working on the same thing, ask the questions no one thought to ask before, or which were asked without the right people involved.

The best case scenario is that everyone can quickly get aligned on common goals and priorities – the disconnect was a bump, not a crash.

Even in the worst case scenario, it’s better to ask the hard questions and to avoid pressure to “just keep working” before your strategic goals are reestablished and validated. The sunk cost fallacy will work against you, but it’s important to remember that it is a fallacy.

If you’re six months into a year-long project, it’s hard to admit that you might need to throw away two or three months worth of work, and end up coming in over schedule and over budget. That’s hard, but it’s better than delivering something no one needs at the end of the year having consumed the entire schedule and the entire budget with nothing useful to show for it.

Going back to that tweet, I have every reason to assume that Lars and his client were successful. Even though Power BI and PowerPoint are very different tools, there are many use cases where either one could be applied to great effect… and at the risk of Abraham Maslow rolling over in his grave, if the only tool you have is PowerPoint, every problem starts to look like a presentation.

I suspect that Lars used this shocking statement as an opportunity to start a new conversation, and to explore the problems his client was using PowerPoint to solve. The next time you find yourself in a similar situation… maybe you will too.


[1] Which is also when I started this post. Life has been busy forever.[2]

[2] I added the previous footnote over a year before I added this one. Life has indeed been busy forever.

[3] This makes me think of one my favorite Sun Tzu quotes.

[4] Or other type of effort, but in my experience it’s usually a project.

Thoughts on community

This post is mainly a story about how community has affected my career, and how it might affect yours as well. The post has been sitting in my drafts for a few months waiting for inspiration to strike. That inspiration came in this recent[0] data culture video where I talked about communities of practice inside enterprise organizations, and the ways that positive behaviors can be promoted and encouraged for these internal communities of practice – because the same patterns often apply to public communities as well.

So let’s talk about community.

No, not that Community. Technical community. Community like this:

hand-1917895_640

I started off my tech career as a Microsoft Certified Trainer (MCT) in 1996. I spent much of the next decade as a trainer, mentor, and consultant, but even though I was learning and growing year after year, the pace of that growth felt slow. I quickly became a medium-size fish in a tiny pond[1] and that was pretty much that. My career growth was limited in part by the company I chose to keep.

To be fair, there was an online[3] MCT community where I would often lurk and occasionally post, but I never took real advantage of it. I didn’t feel confident enough to post often, and for some reason traveling to a conference or other in-person MCT event just didn’t feel like something that was available to me – this was something that other people did, not something I could do.[4]

In late 2004 I started thinking about getting back to more training after a few years focusing on software development and consulting. I decided that the upcoming SQL Server 2005 release would be a great opportunity for making this change. Microsoft had made available to MCTs the preview versions of two “What’s new in SQL Server 2005” courses – one for developers and one for admins – and I was going to get certified to teach these courses as soon as they were available to offer. I’d been working with SQL Server since version 6.5, and much of my consulting work included developing solutions on SQL Server 2000, so this was a logical direction that matched my experience, skills, and interest.

To make a long story slightly less long – and because I’ve shared some of the details while this post was sitting in my drafts – that focus on SQL Server 2005 was my jumping off point to become an active member of the MCT and SQL Server communities. I was still doing many of the same things I’d been doing all along, but now I was doing them where more people could benefit, and where more people could see me doing what I did. And people noticed.

People noticed that I knew what I was talking about on technical topics, but mainly people noticed that I engaged in a constructive and helpful manner. That I gave more than I took.[5]

And this brings us to the concept of visibility from that earlier blog post, and the opportunity that comes from being seen kicking ass – being seen doing the great work you do every day. For me, being actively engaged in the MCT community led to my first job at Microsoft. People who knew me because of my community involvement let me know that there was a job I should apply for, and they put a referral into the internal HR systems to make it more likely I would get called for an interview[6], and the rest is history.

For you, being actively engaged in your community – whether it’s the Power BI community or something else – will likely mean something very similar. The details will be different because the context is different, but adding your voice to the communities you frequent is how you let the community know who you are and what you do.

When you answer questions – and when you ask them too – other members of the community will begin to understand what you do, and how you work. When you write a blog post or record a video, more people will begin to understand how you think, and how you communicate. They’ll learn from you, and they’ll learn about you.

This is good.

As more people learn about who you are and what you do, and as more people begin to see and value your contributions, they’ll start thinking about how awesome it would be if you could contribute more directly to their projects and products. For me this meant being asked more frequently to teach SQL Server and SSIS classes, and an increased volume of inquiries about consulting availability.[7] I had more opportunities for more interesting and better paying work because of my engagement with the community, and it changed my life for the better in many ways.

For you it will look different because the context is different, but you can follow the same pattern. Your contributions to the community organically become your resume[8], your marketing web site, your demo reel. Potential employers and potential clients[9] will have a more complete picture of how you work than a job interview can provide, and that can make all the difference.

Implied in this pattern is the positive nature of your community contributions. If you’re a bully or a gatekeeper or engage in other types of abusive behavior… people notice that too. This can become its own opportunity for improvement, because if you’re more mindful about presenting the side of you that you want people to see, the more likely you are to become that better version of yourself.[10]

Engaging with the community changed my life. It can change yours too.


Interesting side note on timing: I joined Twitter ten years ago this month.

At that time I had just accepted a position on the SQL Server Integration Services  (SSIS) product team, and had asked one of my new colleagues where the SSIS community was centered, and he pointed me there. The rest sort of took care of itself…


[0] Referring to this data culture video as “recent” gives you some insight into how long this post languished in draft form after the introduction was written.

[1] This is kind of like saying “big fish in a small pond” but self-aware enough to realize I’ve never been a large fish on any scale.[2]

[2] I am so sorry, and yet not sorry at all.

[3] Two words: Usenet newsgroups.

[4] I’ve thought about this a lot in the intervening years, and believe that the root cause was probably a combination of my rural, poor, opportunity-sparse upbringing, and my mental health challenges, but I doubt I’ll ever know for sure.

[5] I’m generalizing here, and making some big assumptions based on the available evidence. This is what I think people noticed based on the way that they reacted, but only a handful said so. Of course, they typically said so when offering me work…

[6] This is something I have done a few times over the years myself, and there’s no better feeling than knowing I played some small part in helping Microsoft attract new talent, and in helping a member of the community achieve their career goals.

[7] For me it also meant that I roughly doubled my billing rate over the course of 18 months. I had so many inquiries that I was turning down almost all of them because I didn’t have the bandwidth to say yes, and decided to take advantage of the laws of supply and demand.

[8] This 2016-era post from MVP Brent Ozar presents a much more prescriptive approach to this theme.

[9] And potential employees as well, in the fullness of time.

[10] And let’s be honest – if you’re an insufferable asshole who refuses to change, participating in a community is a great way to let people know, so they can know that you’re someone they should avoid hiring, interviewing, or contracting. This is also good.

Free Power BI Intro Training From Microsoft

As anyone who is familiar with my series on building a data culture knows, training is an absolutely vital success factor for a community of practice. This isn’t limited to developers, administrators, and content creators – you need to empower every user for your data culture to truly thrive.

Although Microsoft has long offered a variety of training resources such as the “in a day” courses[1], until now these offerings were focused on more technical personas and topics, not on the business users who are consuming reports and dashboards.

Guess what this blog post is about.

That’s right! Starting this month any Power BI user can sign up for free online introductory training delivered by Microsoft instructors.

These are synchronous online classes with expert instructors who guide Power BI consumers through some of the most common and most important parts of the Power BI experience. Classes are scheduled for multiple days and multiple times of the day[2], and I expect more to be scheduled on an ongoing basis. Click the link above for the current schedule.

But it gets even better!

If you’re working for an organization with lots of Power BI users[3], Microsoft offers “closed classes” on your schedule, for your users. I don’t have a link to share for this one, but your Microsoft account team wants to help get you started. Talk to your customer success account manager (CSAM) or another member of your account team, and ask about the Microsoft Store Events training for Power BI.

Why wait? Sign up for a course or talk to your account team today!


[1] Take a look here for a list of courses including Dashboard in a Day, Administrator in a Day, Developer in a Day, and other training courses that sadly do not include the phrase “in a day” in their names.

[2] There’s even one course being delivered as I type this post!

[3] We’ve been making this training available for enterprise customers for the past few months, and have already reached tens of thousands of learners while receiving great feedback along the way.

Never underestimate the voice of the customer

Most of what I do in my day job involves listening.

A sculpture of men with ears pressed to a brick wall - Image by Couleur from Pixabay
It’s better to put your ear up to the wall than it is to bang your head against it.

Specifically, I spend a lot of time listening to people who use Power BI[1] talk about why they use it, how they use it, and where they struggle along the way. I ask a few questions to get them started, and then I ask a few follow-up questions to help them along as they tell their story – but mainly I listen.

I listen, and then I use what I hear to help make Power BI a better tool for these people and their organizations’ needs.

I listen for patterns and trends, and for a signal to emerge from the noise of hundreds of conversations. When it does, I make sure that the right decision-makers and leaders hear it and understand it, so they can take it into account as they decide where to invest and how to prioritize those investments.[2]

I listen with an informed ear. I spent almost 15 years building software and data solutions, so I understand a lot of the technical and organizational challenges these people face. I spent the next decade or so building data products and services as a program manager at Microsoft, so I understand a lot of the challenges involved in prioritizing and building product features. And perhaps most importantly, I am genuinely interested in listening to and learning from the people with whom I talk. I truly want to hear their stories.

Of course there’s only one of me, and I can’t spend my whole workday listening to people tell their stories… so I’ve helped build a team to listen[3], and it’s making a difference. The next time you see an announcement about new capabilities in Power BI that will improve the lives of large enterprise customers, odds are this team had a hand in making those capabilities a reality.

Why am I telling you all this? This is my personal blog, and I’m not looking for a job, so who cares, right?

You care.

You care because this type of listening is a thing that you can do to make better software. You can do this whether you’re building commercial software or if you’re part of a BI team that needs to build solutions for larger user populations.[4]

You care because letting the people who use your software – or the people you want to use your software – talk about what what’s important to them… will let you know what’s important to them. And if you’re really listening, you can use what you learn to make better decisions and deliver better products and features.

If you’re genuinely interested in the people who use the software you build, you should consider giving this approach a try.


[1] These people tend to be the business and IT owners of Power BI in large enterprise organizations, but they can be just about anyone involved in implementing, adopting, using, or supporting Power BI.

[2] If you’re interested in learning more about the Power BI product team’s planning process, I cannot recommend highly enough this 2019 presentation from Will Thompson.

[3] As part of building that team we all read and/or re-read Lean Customer Development by Cindy Alvarez. If you have not yet read this awesome book, there’s no time like the present.

[4] I honestly don’t know if this type of rigor is appropriate at a smaller scale, because I haven’t done it and haven’t personally seen it done. I suspect that it would be very valuable, but also suspect that there may be lighter-weight options that would provide a better return on investment.