On joining the product team

Almost two years ago I published a blog post and video on Being a Program Manager at Microsoft, in which I shared some of my personal experience and advice based on my decade plus at Microsoft. I’ve received an incredible amount of feedback[1] on that 2020 post… and now it’s time for a follow-up.

My last few posts have focused on topics related to problem domains and solution domains, including how these concepts relate to Power BI and my work on the Power BI CAT team. What about for people who are currently Power BI or data professionals who might be thinking about changing careers to join the product team? These are pretty abstract concepts – what do they mean to you?

This conversation calls for an analogy, and since I don’t know anything about driving race cars or automotive mechanics, I’m going to use an analogy about race car driving and mechanics[2].

As a Power BI or data professional, you’re like a race car driver.

Your job is to win races. You need to respond to the rapidly-changing demands of the moment to deliver victory for your team[3]. You’re in the spotlight,

As a race car driver, there are two main factors that contribute to your wins and losses:

  1. Your skills as a driver
  2. The performance of the car you’re driving

To win a race, you need to know how to get the most from your car, and make it perform for the conditions on the track, and within the rules of the race.

As a program manager on the product team, you’re like a mechanical engineer responsible for designing, refining and tuning the race car engine.

Your job is to deliver an engine that performs the way your drivers need it to perform, as part of the larger system of the race car as a whole. The race car needs to have the capabilities and characteristics that enable drivers to win races, and the engine is a key part of that… as is the steering, suspension, and whatever other subsystems make up a race car.

Your job is not to drive the race car, and you can excel at your job without ever having won (or driven in) a race.

Re-read that last sentence and pause to let it sink in. If you’re thinking about joining the product team that makes your favorite software tool or platform, you’re also thinking about giving up the day to day job of using that software.

Even more than this, the skills that you’ve built as a race car driver – or as a data professional – are no longer directly relevant. Please don’t get me wrong – if you’ve spent years implementing data and BI solutions this will serve you very well as a program manager on a product team building data and BI tools. But that experience will be valuable in the way that having been a race car driver could be valuable in informing the design and optimization of race car engines.

Having an intimate understanding of – and deep personal experience with – the problem domain can help inform your work in building and improving the solution domain… but your job is no longer what it used to be. You need to learn and master new skills to be successful in the new role, and that means no longer being the expert on the problem domain like you used to be.[4]

This can be a difficult realization and wake-up call for people making a career change, because it means giving up – or at least moving away from – the work they know and love. It can also mean discovering new work and new challenges that they may love even more… but change is hard at the best of times, and it doesn’t always come at the best of times.

I hope this blog post and analogy[5] doesn’t scare anyone off. Working on a product team can be incredibly fun and rewarding – it’s just different from working with the product that the team builds and ships. It’s another situation where the problem domain and solution domain differ in ways that aren’t always obvious until you’ve had a chance to work in both worlds.

Joining the product team after working as a data professional changes your relationship with the product. What used to be your solution domain, where you were the expert responsible for understanding the product and delivering solutions based on its capabilities, now becomes your problem domain. You’re responsible for increasing its capabilities to make it more useful for others to build their solutions, but you’re no longer building those solutions yourself.

 


[1] I continue to believe that this video may have been the best thing I did in 2020. I’ve had dozens of conversations with people around the world who want to be program managers or product manager – at Microsoft or other tech companies, and some of them have already achieved their goal. This feels like I’m actually making a difference in the world when the world is doing its best to make me feel powerless, and that’s like a gift that keeps on giving. I may not be the only one who feels this way. While this post was in draft form waiting to be published, my friend Davide created a website to showcase this video – he even transcribed it! You can check it out here: http://aka.ms/being-a-ms-pm.

[2] Because of course I am. This is an analogy that presented itself to me in one of those conversations mentioned just one footnote earlier, and which works as well as any I’ve found so far.

[3] There are teams in racing, right?

[4] This is where it’s useful to have a person or team responsible for bridging these two worlds.

[5] Another possibly less scary analogy could have involved someone who loves eating becoming a chef in a professional kitchen. Since I know enough about these worlds to be dangerous, I didn’t trust myself to use this analogy without ratholing on lots of details no one but me would care about. Maybe I’ll try that one again when I have more time to think about it…

Simplifying the solution domain

My recent posts have focused on the importance of understanding both the problem domain and solution domain in which you’re working, the value of operating in both the problem domain and the solution domain, and some specific resources and techniques for solution-oriented practitioners to increase their fluency in understanding the problem domain. A lot of the message I’ve been trying to get across in these posts can be summarized as “the problem domain is really important, and we need to get better at appreciating and understanding the problems we work to solve.”

This message is predicated on two basic assumptions:

  1. You, the reader, are a technical professional of some sort[1], and you need to deliver technical solutions to people with problems
  2. The people who have problems can’t solve those problems on their own

Let’s look more closely at that second assumption, because it’s becoming increasingly less valid, and less true. And let’s look at it through the lens of… furniture.

Close your eyes and imagine for a moment that you need a table. Let’s say you need a bedside table. What do you do?

As you read through this list you were probably evaluating each option against your own requirements and preferences. Odds are you’ve chosen one or more options at some point in your life. Odds are the criteria you used to decide which options are viable or preferable have changed as your circumstances, skills, and budget have changed.

I couldn’t find Ikea instructions for a bedside table, but honestly I didn’t try very hard

Of course, the decision doesn’t need to be about a bedside table. It could be about a dining room table, or a bed, or a full set of kitchen cabinets. Each option will have different but similar choices, and you’ll likely use different criteria to evaluate and select the choice that’s right for you.

This is also true if the choice is about business intelligence solutions, which may not technically be considered furniture[2]. Close your eyes once more and imagine for a moment that you need a report, which may or may not include a table.  What do you do?

  • You could use an existing report
  • You could start with an existing report and customize it
  • You could start with an existing dataset use it to build a new report
  • You could start with an existing dataset, customize it using a composite model, and use it to build a new report
  • You could work with a center of excellence or BI team to create a dataset, customize a dataset, create a report, or customize a report to meet your needs
  • You could locate or create data sources and build a custom BI solution end-to-end, maybe getting some help along the way if you don’t already have the tools and expertise required for the project

The parallels are easy to see, and they hold up surprisingly well to closer inspection. Just as ready-to-build options make it possible for someone with no woodworking or cabinetry skills[3] to self-serve their own furniture, DIY BI tools[4] like Power BI make it increasingly easy for someone with no data preparation or data modeling skills to build their own BI solutions.

To step away from the furniture analogy, tools like Power BI simplify the solution domain and make it easy for problem domain experts to solve their own problems without needing to become true experts in the solution domain.

Any finance or supply chain professional can use Power BI Desktop to build reports that do what they need, without needing to ask for help. Their reports may not be the most beautiful or best optimized[5],  but maybe they don’t need to be. Maybe “good enough” is actually good enough. Just as it might make more sense to buy a $40 Ikea bedside table instead of spending hundreds or thousands of dollars on something fancier, it might make sense to stack a few milk cartons on top of each other and call it good, at least until you can save up for something better[6]. There is no one size that fits all.

If you are a Power BI professional, it may be difficult to think of the “boards and cinderblocks bookshelf” approach as acceptable, but in most circumstances it’s not up to you to decide. The tools for DIY BI are widely and freely available, and people will use them if they present the most attractive and accessible option to solve their problems. You can’t stop them from building DIY solutions, but you can help them build better ones.

This is where having an effective and engaged center of excellence comes in. Whether you think in terms of problem and solution domains, or in terms of enabling DIY through readily available tools and guidance[7], you can help make “good enough” better by meeting your problem domain experts where they are. They have the tools they need to get started building their own solutions, but they probably need your expertise and assistance to achieve their full potential.

You should help them.


[1] Probably a Power BI practitioner, but who knows?

[2] I have repeatedly seen Power BI referred to as a “platform” which is basically the same thing as a table, so I’m going to withhold judgment on this one.

[3] Someone like me! I married a carpenter and have never found the motivation to develop these skills myself.

[4] DIY BI has a ring to it that SSBI doesn’t have. Why don’t we start using DIY BI instead. DBIY? D(B)IY? D(BI)Y? Hmmm…

[5] They may indeed be the Power BI versions of my DIY home improvement projects.

[6] Please raise your hand if you too had bookshelves made of random boards and cinderblocks when you were in college, and were also happy with the results.

[7] This is the second place in this post where I’ve shared a link to this Marketplace story. If you run a Power BI COE for your organization, do yourself a favor and listen to the story – it will inspire you to think about your COE in different ways.

Understanding the problem domain

If you’re reading a Power BI-focused blog[1] you’re probably someone who builds solutions. As a BI and data practitioner, you may build reports, datasets, dataflows, databases, or any number of technical artifacts that together represent a solution domain. You have a toolset at your disposal, and you invest your valuable time and energy into refining the skills needed to use those tools.

But what if you’re holding the right tools, while using them to implement the wrong solution? This could happen if you’re holding a hammer and building a chair when what’s really needed is a table, but it could also happen if you’re using Power BI to build a solution that deliver what its users need to make their jobs easier.

Action doesn’t equate with value, and time spent solving a problem you don’t meaningfully understand is often time wasted. So how do you, as a Power BI author, meaningfully understand problems in domains like finance, supply chain, HR, or patient care if you lack personal experience in those domains?

Asking good questions is a great start.

If you want to deliver a fix for a problem someone has reported, asking targeted closed-ended questions to narrow down the scope of the problem can be valuable. What is the error message, when did it start happening, what are the steps to reproduce the problem, does it happen all the time, does it happen on other computers, does it happen for other users… that sort of thing.

This type of question is often great if you need to deliver a fix, but what about when you need to deliver a product? What if you need to deliver a Power BI app that will help a group of people be more productive completing a complex set busines processes?[2]

Asking good questions is still a great start, but you need to ask different questions.

While focused, detailed, targeted questions can be very effective for scoping down a tactical problem to deliver a fix, they tend to not be great for understanding a broader problem domain. Ask closed-ended questions when you want to deliver a fix; ask open-ended questions when you want to understand the problem.

What does your current process look like, where are the places where you struggle, what are the triggers that would cause you to begin the process, why do you do it that way, who works with the output you produce, and what do they do with it… that sort of thing.

If you’re used to asking the first sort of question, the second sort may not be obviously useful. What would you do with these answers? How can you act on them?

The short answer is “it’s complicated” and the longer answer is “read The Customer-Driven Playbook by Travis Lowdermilk and Jessica Rich.”

This amazing book presents a “hypothesis progression framework” that includes tools for learning about customers and their problems before building a solution to those problems. This framework is broken down into four phases:

At the risk of oversimplifying 250 pages of expert guidance, the framework looks something like this:

  • Who is my customer, what does she do, what motivates her, and what problems does she have?
  • What is the scope of my customer’s problem, how does it affect her, and what is its impact?
  • How well the concept you’ve come up with to solve to solve the problem actually fit with your customer’s needs, does its value proposition motivate her to learn more, and how will the limitations of the concept reduce its value?
  • How well does the product or feature you’ve built to implement the validated concept actually solve the customer’s problem it was built to solve?

Each of these phases is presented with best practices for asking the right types of questions of the right people in the right way, with guidance for formulating the right questions, asking them in structured experiments so you can better trust the answers you get, and then making sense of what you’ve heard. That sensemaking process is where your deep expertise in the solution domain will help you the most, as you look closely and mindfully at what you you’ve heard and decide how to act… but most of the process is about learning the details and nuances of the problem, not about solving it.

Albert Einstein once said “If I had an hour to solve a problem I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.” Think about that.

In my experience most technical professionals don’t value and appreciate the importance of the problem domain, and instead focus their energies on solutions to problems they don’t fully understand. We can do better, as individuals and as teams and as an industry, and The Customer-Driven Playbook is an excellent place to start. I hope you’ll read it, and I hope you’ll let me know how it changes the way you approach problems and solutions.


[1] Does BI Polar count as a Power BI-focused blog? These days it feels like I’m posting more about ancillary topics and less about Power BI itself.

[2] For that matter, what if you need to deliver a complex commercial software product like Power BI?

Into the Date Dimension!

Late last year I was spending some vacation days playing with one of my favorite personal data sets[1], and I realized I needed a better date dimension. So I asked for one.

A bunch of folks from the Power BI community shared their favorite Power Query date dimension queries[2] – you can look at the Twitter thread for links if you’re interested. With their inspiration (and code!) I picked what I liked, discarded what I didn’t, and came up with this:

let
// To be turned into parameter
startDate = #date(2017,01,01) as date, 
// To be turned into parameter
endDate = #date(2021,12,31) as date, 
// To be turned into parameter
culture = "en-US" as text,

// Beginning of actual query 👇 
Source = List.Dates(startDate, Duration.Days(Duration.From(endDate - startDate)) + 1, #duration(1, 0, 0, 0)),

TableFromList = Table.FromList(Source, Splitter.SplitByNothing()),
ChangedType = Table.TransformColumnTypes(TableFromList,{{"Column1", type date}}),
RenamedColumns = Table.RenameColumns(ChangedType,{{"Column1", "Date"}}),
InsertYear = Table.AddColumn(RenamedColumns , "Year", each Date.Year([Date]),type text),
InsertQuarterNumber = Table.AddColumn(InsertYear, "Quarter Number", each Date.QuarterOfYear([Date])),
InsertQuarterName = Table.AddColumn(InsertQuarterNumber, "Quarter Name", each "Q" & Number.ToText([Quarter Number])),
InsertMonth = Table.AddColumn(InsertQuarterName, "Month Number", each Date.Month([Date]), type text),
InsertStartOfMonth = Table.AddColumn(InsertMonth, "Start Of Month", each Date.StartOfMonth([Date]), type date),
InsertEndOfMonth = Table.AddColumn(InsertStartOfMonth, "End Of Month", each Date.EndOfMonth([Date]), type date),
InsertDayOfMonth = Table.AddColumn(InsertEndOfMonth, "Day Of Month", each Date.Day([Date])),
InsertDayInt = Table.AddColumn(InsertDayOfMonth, "DateInt", each [Year]*10000 + [Month Number]*100 + [Day Of Month]),
InsertMonthName = Table.AddColumn(InsertDayInt, "Month Name", each Date.ToText([Date], "MMMM", culture), type text),
InsertShortMonthName = Table.AddColumn(InsertMonthName, "Month Name Short", each Date.ToText([Date], "MMM", culture), type text),
InsertShortMonthNameYear = Table.AddColumn(InsertShortMonthName, "Month Short and Year", each [Month Name Short] & " " & Number.ToText([Year]), type text),
InsertCalendarMonth = Table.AddColumn(InsertShortMonthNameYear, "Month and Year", each [Month Name]& " " & Number.ToText([Year]),type text),
InsertCalendarQtr = Table.AddColumn(InsertCalendarMonth, "Quarter and Year", each "Q" & Number.ToText([Quarter Number]) & " " & Number.ToText([Year]), type text),
InsertDayWeek = Table.AddColumn(InsertCalendarQtr, "Day of Week", each Date.DayOfWeek([Date]) + 1),
InsertDayName = Table.AddColumn(InsertDayWeek, "Day Name", each Date.ToText([Date], "dddd", culture), type text),
InsertShortDayName = Table.AddColumn(InsertDayName, "Day Name Short", each Date.ToText([Date], "ddd", culture), type text),
InsertEndOfWeek = Table.AddColumn(InsertShortDayName , "End Of Week", each Date.EndOfWeek([Date]), type date),
InsertedStartOfWeek = Table.AddColumn(InsertEndOfWeek, "Start of Week", each Date.StartOfWeek([Date]), type date),
InsertWeekNumber= Table.AddColumn(InsertedStartOfWeek, "Week of Year Number", each Date.WeekOfYear([Date])),
InsertMonthWeekNumber= Table.AddColumn(InsertWeekNumber, "Week of Month Number", each Date.WeekOfMonth([Date])),
InsertYearMonthNumber = Table.AddColumn(InsertMonthWeekNumber,"Year and Month", each [Year] * 100 + [Month Number]),
InsertYearAndQuarterNumber = Table.AddColumn(InsertYearMonthNumber, "Year and Quarter", each [Year] * 100 + [Quarter Number]),
InsertYearAndWeekNumber = Table.AddColumn(InsertYearAndQuarterNumber, "Year and Week", each [Year] * 100 + [Week of Year Number]),
InsertDecadeName = Table.AddColumn(InsertYearAndWeekNumber, "Decade Name", each Text.Range(Text.From([Year]), 0, 3) & "0s"),
InsertDecadeNumber = Table.AddColumn(InsertDecadeName, "Decade Number", each Text.Range(Text.From([Year]), 0, 3) & "0"),
InsertStartOfQuarter = Table.AddColumn(InsertDecadeNumber, "Start of Quarter", each Date.StartOfQuarter([Date]), type date),
InsertEndOfQuarter = Table.AddColumn(InsertStartOfQuarter, "End of Quarter", each Date.EndOfQuarter([Date]), type date),
InsertDayOfYear = Table.AddColumn(InsertEndOfQuarter, "Day of Year", each Date.DayOfYear([Date]), Int64.Type),
InsertIsWeekday = Table.AddColumn(InsertDayOfYear, "Is Weekday", each if [Day of Week] = 1 or [Day of Week] = 7 then 0 else 1),
InsertIsWeekend = Table.AddColumn(InsertIsWeekday, "Is Weekend", each if [Day of Week] = 1 or [Day of Week] = 7 then 1 else 0),
#"Reordered Columns" = Table.ReorderColumns(InsertIsWeekend,{"DateInt", "Date", "Year", "Year and Quarter", "Year and Month", "Year and Week", "Quarter Name", "Quarter Number", "Quarter and Year", "Start of Quarter", "End of Quarter", "Month Name", "Month Name Short", "Month Number", "Month and Year", "Month Short and Year", "Start Of Month", "End Of Month", "Week of Year Number", "Week of Month Number", "Start of Week", "End Of Week", "Day of Year", "Day Of Month", "Day of Week", "Day Name", "Day Name Short", "Decade Name", "Decade Number", "Is Weekday", "Is Weekend"}),
#"Changed Type" = Table.TransformColumnTypes(#"Reordered Columns",{{"DateInt", Int64.Type}, {"Year", Int64.Type}, {"Year and Quarter", Int64.Type}, {"Year and Month", Int64.Type}, {"Year and Week", Int64.Type}, {"Quarter Number", Int64.Type}, {"Month Number", Int64.Type}, {"Week of Year Number", Int64.Type}, {"Week of Month Number", Int64.Type}, {"Day of Year", Int64.Type}, {"Day Of Month", Int64.Type}, {"Day of Week", Int64.Type}, {"Decade Number", Int64.Type}, {"Is Weekday", Int64.Type}, {"Is Weekend", Int64.Type}, {"Quarter Name", type text}, {"Decade Name", type text}})
in
#"Changed Type"

In case you can’t visually parse that unfortunate wall of text, here’s what it looks like in my data model:

As I keep playing with my pet project I’ll probably keep tweaking the date dimension query to add, remove, or refine it as needed. When I’m done I may turn it into a dataflow, or I may make it into a function that takes my project’s min and max dates as input parameters. Who knows?

What I know for sure is that the next time I’m looking for a date dimension query, I’ll look here.


[1] I made a GDPR request of Spotify, asking for all of the data they had related to my account. It took them about a month to prepare that data and make it available for me to download, but it was worth the wait. You can request your own data too: https://www.spotify.com/us/account/privacy/

[2] And a bunch of other folks shared DAX and SQL solutions.