I write about topics that are of interest to me, mainly because they’re interesting to me but also because there doesn’t seem to be anyone in the Power BI community covering them. There are dozens of blogs and YouTube channels[1] covering topics like data visualization, data modeling, DAX, and Power Query – but I haven’t found any other source that covers the less-tangible side of being a data professional that is so compelling to me.
I write on my own chaotic schedule. Some months I might post every other day, or I might go weeks or months without posting anything[2]. These days I try to post once per week, but that’s definitely a stretch goal rather than something to which I will commit. Sometimes my creativity flows freely, but sometimes writing comes from the same budget of emotion and energy that I need for work and life… and blogging typically ends up near the bottom of my priority list.
And yet, here we are, 40 months and 200 blog posts later. In the past few weeks I’ve seen dozens of people reference Roche’s Maxim of Data Transformation, which started out as a tongue-in-cheek self-deprecating joke and then took on a life of its own. Earlier this week I spent time with another team at Microsoft that has organized been collectively reading and discussing my recent series on problems and solutions, and looking for ways to change how their team works to deliver greater impact. More and more often I talk with customers who mention how they’re using some information or advice from this blog… and it’s still weird every single time.
In these dark days it’s easy to feel isolated and alone. It’s easy to dismiss and downplay online interactions and social media as superficial or unimportant. It’s easy to feel like no one notices, like no one cares, and like nothing I’m doing really makes a difference[3].
So for this 200th post, I wanted to take the time to say thank you to everyone who has taken the time to let me know that I’m not alone, and that someone is listening. It makes a big difference to me, even if I don’t always know how to show it.
[2] Please don’t even get me started on how long it’s been since I posted a new video.
[3] This is a reminder that Talking About Mental Health is Important. I have good days and bad days. Although I make a real effort to downplay the bad and to amplify the good, there’s always a voice inside my head questioning and criticizing everything I do. It’s important to talk about mental health because this is a voice that many people have, and not everyone knows that it’s not just them. Not everyone knows that the voice is lying.
It’s 2021, which means my tech career turns 25[1] this year.
It me
Back in 1995 an awesome book was published: Dynamics of Software Development by Jim McCarthy. Jim was a director on the Visual C++ team at Microsoft back when that was a big deal[2]. The book is one of the first and best books I read as a manager of software development teams, and it has stood the test of time – I still have it on the bookshelf in my office[3].
Of course, I wasn’t managing a software development team in 1995. I discovered Jim’s wisdom in 1996 or 1997 when I was preparing to teach courses on the Microsoft Solutions Framework[4]. When I opened the box that the trainer kit came in, there was a CD-ROM with videos[5] of Jim McCarthy presenting to an internal Microsoft audience on his “rules of thumb” for reliably shipping great software. When I first watched them, it was eye-opening… almost life-changing. Although I did have a mentor at the time, I did not have a mentor like Jim.
The second time I watched the videos, it was with my whole team. Jim’s rules from these videos and his book became part of the team culture[6], and I’ve referred back to them many times over the decades.
Now you can too – the videos are available on YouTube:
Some of the rules may feel a bit dated if you’re currently shipping commercial software or cloud services[7] but even the dated ones are based on fundamental and timeless truths of humans and teams of humans.
I’m going to take some time this week to re-watch these videos, and to start off the new year with this voice from years gone by. If you build software and work with humans, maybe you should too.
[1] Damn, that is weird to type, since I’m sure I can’t even be 25 years old yet.
[2] It may also be a big deal today, but to me at least it doesn’t feel quite as much.
[3] Not that I’ve seen my office or my bookshelf in forever, so this assertion should be taken with a grain of 2020.
[4] Fun fact: if you remember MSF, you’re probably old too.
[5] Because in those days “video” and “internet” weren’t things that went together. This may or may not have been because the bits had to walk uphill both ways in the snow.
[6] To the extent the team could be said to have a shared culture. We were all very young, and very inexperienced, and were figuring things out as we went along.
[7] Back in 1995 CI/CD wasn’t part of the industry vocabulary, and I can count on zero hands the clients I worked with before joining Microsoft who had anything resembling a daily build.
My professional career was enabled by professional certifications. I would not be where I am today – however you want to parse that statement – without my Microsoft certifications[1].
Certification rocks!
Before you read too much into that bold opening statement, please allow me to elaborate by telling my personal certification story[2]. This post was prompted by an online conversation, and responses on Twitter that made me remember that this is a nuanced topic, and that my experience is part of a bigger discussion. Understanding my story will help put my opinions in context.
tl;dr: certifications are a great way to build and validate knowledge and skills, and can help meet career development goals while complementing real world experience.
I started my IT career in 1996 when I got a job as a software applications trainer. I spent around six months teaching end users how to use Word, Excel, PowerPoint, Access, Windows 3.11, and Windows 95. Despite being terrified of public speaking[3], it turned out I was pretty good at it. After those six months I was offered the opportunity to become a Microsoft Certified Trainer and to teach MCSE[4] courses at the training center where I worked. The risks were high[5], but the potential gain was enough to offset the risk. I said yes.
At this point you might be saying something along the lines of “hey wait, you didn’t have any real world experience!” You’re exactly right. And I wasn’t about to get any. I was passing exams so I could teach people with real world experience how to do their jobs that I had never done. I’m as aghast as you are. Please bear with me.
After I passed my first few exams[6] I developed a study routine that looked something like this:
Read the courseware cover to cover
Make a list of topics that I didn’t understand well enough to explain
Investigate these topics in secondary sources like Microsoft TechNet CDs and technical books[7]
Read the courseware again, crossing items off the list as appropriate
Repeat steps 3 and 4 as necessary
Take and pass the exam
Teach the class
Get awesome ratings from my students[8]
Fast forward a few years and I was in a consulting role[9] where training was a much smaller part of my responsibilities. Instead, I was working with clients to design and implement their network infrastructures, and later to design and implement custom software solutions using SQL Server, Visual Basic, and ASP.
This transition had a few bumps, but it was all about the tasks, not the knowledge. The skills and knowledge I had gained by preparing for certification exams and had reinforced through training enabled me to hit the ground running.
I no longer needed to pass exams to teach classes, but I did have another motivation: my company was a Microsoft Partner, and the benefits we received depended in part on the certifications held by our employees. I kept taking exams, but because I had different goals and a different starting point of skills and experience, I took a different approach.
My new exam prep routine looked something like this:
Read the “skills measured” list published by Microsoft on the exam web page
For any skill on the list where I don’t have meaningful knowledge and experience, study and practice the skill
Take and pass the exam
This new routine is shorter and simpler because I now had more experience and a richer personal knowledge base to rely on, and because I wasn’t preparing to teach a class – I just needed to pass the exam and use the product.
I adopted this second exam prep routine over 20 years ago, but I’ve never seen the need to change it, even as my role and motivations have evolved.
Since 2001, I have used certification exams mainly as a skill-building tool. When there is a new tool or technology I want to learn, I will use the exam skill list as my prep guide, and the exam as validation that I have learned it well enough. (More later on what “well enough” means here.) The list of skills measured by an exam is informed by a lot of industry research, and it covers a broader range of topics and features than you’re likely to encounter solely through hands-on project work.
One standout example of this process is BizTalk Server. Using this exam prep approach I was able to quickly build my BizTalk knowledge well enough to use it on a few projects. I wasn’t proficient on day one, but I went into my first BizTalk project with a clear understanding of what features it had, where to use each one, and how they worked together. This meant that as I was figuring out how to use a specific feature to solve a specific project requirement, I knew from the beginning what feature to use, and where to begin. Because I was certified.
But that’s just me. What about the other guy?
In the late 90s and early 00s the phrase “paper MCSE” got kicked around a lot. The idea was that some people would pass the exams and gain the certification without actually earning it, maybe by going to an illegal brain dump site or otherwise cheating. They’d have the letters but not the skills to back them up. What about those guys?
In my experience the paper MCSE was more often heard than encountered, but it was a real problem, as evidenced by articles like this one. I hired a few paper MCSEs, and I fired a few as well.
Based on my personal positive experience with certification, I updated my interviewing and hiring practices to go deeper in other areas – and to pay attention to how the candidate represented the importance of their certifications. If someone acted like their certifications made them God’s Gift To IT, that was a red flag – their certification had become a reason to not hire them. If someone acted like their certifications provided a foundation on which they could build with experience, they were more likely to get hired.
I’m hesitant to compare certifications to college degrees, because I have never completed a degree program – but I’m going to make the comparison anyway. I don’t know of any technical jobs where simply having letters after your name – be it MCP, MCSE, OCP[10], BA, or BS – makes you qualified. If you’ve earned those letters, they contribute to your qualification to one degree[11] or another. The rest of who you are, what you know, and what you do, is what completes the qualification.
I haven’t passed a certification exam since 2012. On my birthday in 2012 – four years into my career at Microsoft, and having just helped ship SQL Server 2012 – I decided I would take the SQL Server 2012 exams on my birthday as a present to myself[12]. All seven of them. And I passed them all.I knew my stuff, and it showed.
But I have failed an exam since then. A year or two later I was at a conference where a testing center was set up. A friend was taking an exam, and the folks at the testing center had an opening (also, as a Microsoft employee, I didn’t need to pay anything) so I said “why not?” I picked an exam on some technology I’d dabbled with, and went for it. I figured my general platform knowledge and experience and my having passed over 75 different Microsoft exams should compensate for my lack of knowledge on the specific topics being tested.
I failed, and not by a small margin. And this was what should have happened. I was not a qualified candidate for the skills being tested on this exam. I should have known better, but tried it anyway. The exam knew better.
Anyway, that’s enough about me. Let’s talk about you.
Rather than simply saying yes, please take a look at the skills measured by this exam and let me ask you if you would like to have those skills validated. Or, ask yourself if these are the skills that hiring managers would want a Power BI data analyst to have.
Then I’ll say yes. You should take the exam.
If it aligns with your goals well enough for you to be asking in the first place, yes you should take and pass the exam. Use the skills measured by the exam to guide your study and preparation, and then use the exam to validate that you know the topics well enough.
If you’re using Power BI every day, odds are you can spend a few weekends studying the things you don’t do very often, and you’ll be ready to pass the exam. If you’ve never used Power BI you’ll probably need to spend a few months studying and practicing before you can pass. But either way, the skills measured by the exam align well with what you need to succeed in the real world – and how could that be a bad thing?
Remember how I said earlier that we’d get back to what “well enough” means in the context of certification validating skills? Think about when you got your drivers license and the test you had to take. Think about the times you failed the test, or the friends who failed it.
Did passing the test and getting your license make you a Formula One driver? No, it validated that you were a minimally qualified driver, and that you were now allowed to legally operate a specific class of motor vehicle because you’d met a predefined bar. This is what certification tests to – they show that you are minimally qualified for the skills or role tested by the exam.
Is anyone interested in a blog post on the value of certification? This came up in a chat earlier today, but I'm trying to see if there's broader interest.
As I’m publishing this post, my question on Twitter has gotten 31 replies, and spawned some interesting discussion. I’m not going to try to reply to all of those topics here, but I’d love to have you join the discussion, and to hear what you think.
Also, when you take and pass the DA-100 Power BI exam, let me know what you think, and if you agree with me.
[1] If you’re interested, you can see my certifications by clicking on this link, and entering the transcript ID 678145 and transcript sharing code MatthewMCPCode1 when prompted.
[2] Part of it, anyway. The whole thing would likely take far too long to tell, and I would be as bored as you.
[3] My previous presentation experience was a college public speaking class where I needed to deliver 5 or 6 short speeches to the rest of the class. I spent 20-30 minutes before each speech being physically ill in the restroom down the hall from the classroom. But the trainer job’s starting pay was double what I had ben making at my retail job, and it was working with computers – combined, that was some serious motivation.
[4] If you think back to 1996, you’ll remember what a big deal the MCSE was. I remember seeing people charging over $1000 per day to deliver the official training courses. If that doesn’t sound like a lot of money to you, please consider that very recently I’d been making under $7.50 per hour. This was the opportunity of a lifetime.
[5] The deal was basically this: My employer would pay for the certification exams for each new course I was asked to teach, and I would pass them. They would also pay for me to attend each course – this was a Microsoft-mandated prerequisite to teaching at that time. For each course day, I would get a paid study/prep day. By the end of the prep days I needed to pass the exam(s) required to teach the course. If I did, I got a raise for each new course I taught. If I didn’t, I would need to pay my employer back my salary for the prep time, and the full retail cost for the course I attended. It’s also worth noting that at this point I had 3 peers who had collectively taken and failed 4 or 5 exams, and I knew zero people who had passed one. So yeah, high stakes, high pressure.
[6] The first exam I took was Implementing and Supporting Microsoft Windows 95. I was so nervous that when I sat down at the testing center computer to take the exam, I looked around trying to figure out where the loud pounding noise was coming from, because I was hoping for a quiet environment for this stressful activity. After a minute I realized that the loud pounding noise was my own racing heart. I am not exaggerating at all – I was terrified, but I prevailed.
[7] This was before “searching online” was a thing. Kids these days and their Googles don’t know how good they have it. <<shakes fist at cloud>>
[8] I don’t mean to brag, but despite my lack of real world experience, I was really good at what I did. Really, really good. I earned “instructor of the month” every month, and because this was disheartening to the applications instructors the training center split this recognition into “applications instructor of the month” and “networking instructor of the month” so someone else would have a chance. Every time I’m in Albuquerque, my ego tells me I should visit to see if the wall of fame is still there. I tell my ego to shut up, because it’s gotten me into enough trouble already.
[9] I need to shout out here to my amazing mentor Jeff Lunsford, without whom I may never have taken this leap. Of all the people in my life to whom I owe an eternal debt of gratitude, you are the one named Jeff. Thank you for believing in me, for opening doors for me, for pushing me when I needed pushing, and for supporting me when I needed support.
[10] Did you know I was Oracle certified too? Around 20 years ago I got my Oracle Certified Professional Developer certification as a way to learn the basics of Oracle database. I fell in love with P/L SQL, but I hated Oracle’s tools so much I never did anything with the certification or the knowledge.
[11] Pun not intended, I swear.
[12] If you look at my transcript, only five of these seven show up on my birthday, but there are other weird date discrepancies throughout the transcript too. I’ve learned to let it go. Also, if you try to schedule 7 exams on the same day please be prepared for the exam people to tell you “no” more than once.
[13] YouTube isn’t letting me see the chat for some reason, so I can’t say who it was.
[14] I’m still laughing at the “certification rocks” caption on that photo. I get all of my random pictures from https://pixabay.com/, just so there’s something to show in the preview when the link is shared. This one works on so many levels… if like me you have the sense of humor of a 12 year old…
It’s easy to look around – especially on social media – and see people making complex and difficult tasks look simple and effortless. It’s also easy to judge your own learning processes, struggles, and failures, against the final edited results of people who have been learning and practicing for years – and to feel disheartened by the comparison.
The final video ended up including around 20 minutes of “talking head” footage, but I had closer to 40 minutes of raw footage before I edited out all most of the mistakes and false starts.
It’s not just me. One of my data community heroes, Power BI MVP Ida Bergum, recently shared a similar example:
If you’re ever feeling like an impostor[2], like there’s no way your efforts could ever measure up to all the “experts” you see online, don’t give up. You cando it.
Every successful example that you see online or at a conference or on a bookstore shelf has gone through the same journey. They may be further down the path – that’s usually the easy thing to see. They also had a different starting point and they probably have a different destination in mind as well, even if it’s close to your goals.
Even if you see someone and think “I wish I…” it’s important to remember that they struggled along the way as well. Odds are they struggled in different ways or in different places, but no one’s journey is free of bumps and detours – even if people usually prefer to share the highlights from the easy parts.
One of my favorite quotes on this subject is from a cartoon alien-dog-thing, and you can find it all over Pinterest:
It never feels great to suck at something you want to not suck at, but everyone sucks at the beginning. Most people just don’t like to admit it – at least not until they’ve moved past the “sorta good” phase.
One overlooked benefit of being earlier on your learning journey is that you can still remember what it was like to suck at something, and what was valuable to you then. If you’ve been doing something every day for 20 years, odds are you may not remember what it was like to just be starting off. But if you are becoming sorta good at something, you have a perspective that you can share with everyone out there who sucks at that thing, and would really like to be sorta good.
Anyway… I didn’t quite figure out how to get the narrative for this post into written form. The story was there in my head, but didn’t want to take shape once I started typing. But I won’t let that stop me, because it doesn’t need to be perfect to be good, and useful.[3]
[1] I’ve included the “if you really want” qualifier deliberately. Motivation and desire are critical for success, because it’s that want that will push you to keep getting back up after you fall down, and will keep you looking for more resources and opportunities to learn and improve.
[2] I still remember the day I first read this blog post by Scott Hanselman. My friend Cheryl had shared it on Facebook, with a comment about how she had thought “it was just me.” I’d known and admired Cheryl for years, and had no idea she felt this way, because I’d only ever seen her kicking butt. This is just another example of why it’s important to talk about mental health.
[3] I swear I didn’t plan this ending, but once it presented itself you’re darned right I was going to run with it.
My awesome friend Christopher also works at Microsoft, but his career and mine have taken different paths since we joined. Here we are together back in 2008, before either one of us worked at Microsoft. He’s the one not wearing a Manowar t-shirt.
No, no, the other one.
Our backgrounds are remarkably similar. We each spent years before joining Microsoft as Microsoft Certified Trainers, with an emphasis on software development rather than systems administration. We each became involved in the MCT community, and in the broader technical community, and to one extent or another this helped us find our first positions at Microsoft.
These days Christopher is working with student developers – I’ve seen him tweeting a lot recently about Django on Twitch, which probably means that he’s still teaching people about software development… or may be watching Quentin Tarantino movies while drinking too much coffee. Honestly, either one is possible.
Anyway… Christopher reached out to me last month and asked if I’d be interested in talking to college students about to graduate about what it’s like being a program manager at Microsoft. I said yes[1], then I said this:
I’ll let the video speak for itself. I tell my story for the first 15 minutes or so, and around the 14:50 mark I talk with Will Thompson and Tessa Hurr from the Power BI team and ask them to share their experiences as well. But there are a few things that I want to add that didn’t really fit well into the video.
First of all, every team has a need for many types of program managers. I’ve blogged about diversity enough that you probably know how I feel about the value of diverse teams, and this applies to PM teams at Microsoft as well. Since this video is targeted at college students and recent college graduates, I’d like to focus briefly on the career diversity dimension.
What does “career diversity dimension” mean? Looking at most PM teams I see program managers falling into three broad groups:
New to career – program managers who are starting their careers after college, or after switching from a non-IT discipline.[2]
Industry hires – program managers who are new to Microsoft, but who have an established career in a related field.[3] This is often someone who has been a consultant, developer or administrator who works hands-on with Microsoft or competitive tools and technologies.
Veterans – program managers who have been at Microsoft long enough to succeed a few times, fail a few times, and understand what PM success can look like on multiple teams.[4]
Reading this list it may be easy to think that there is a progression of value implied, but this is not the case. A successful team will find ways to get from each group the things that only they can contribute. A PM in one group will be able to see things and do things that a PM in other groups will not, and an experienced team leader will be able to direct each PM to the problems that they are best suited to solve, and can add the most value.
The second thing I wanted to add to the video is that each team is different. I said this in the video, but I want to elaborate here. Each team will have its own culture, and some teams will be a better fit than others for a given PM. I’ve worked on multiple teams where I didn’t think I was contributing effectively, or where I felt that my contributions weren’t valued[5]. At one point this culture mismatch almost led me to leave Microsoft, but with the support of my manager at the time I instead found another team in another org where I could thrive.
And this leads me to the the final point I wanted to add: Microsoft is huge. I’ve been a PM at Microsoft since October 2008, but I’ve had 4 or 5 major career changes since then, with very different responsibilities after each change, requiring very different contributions from me.
When I was interviewing for my first position in 2008, the hiring manager asked me why I wanted to work for Microsoft. I already had a successful career as a data warehousing and ETL consultant, and becoming a Microsoft employee would include a reduction in income, at least in the short term. Why give up what I’d built?
I hadn’t expected this question, and my answer was authentic and unscripted, and I’ve thought a lot about it over the past 11+ years. I told him that I wanted to join Microsoft because Microsoft had bigger and more challenging problems to solve than I would ever see as a consultant, and would never run out of new problems for me to help solve. If I joined Microsoft, I would never be bored.
I wanted to join Microsoft because Microsoft had bigger and more challenging problems to solve than I would ever see as a consultant, and would never run out of new problems for me to help solve. If I joined Microsoft, I would never be bored.
And I was right.
If you’re a PM at Microsoft, please share your thoughts and experiences.
If you’re thinking about becoming a PM at Microsoft, please share your questions.
If you’ve joined Microsoft as a PM after watching this video and reading this blog, please send me an email, because I would love to say hi.
[1] It’s a good thing he asked when he did. I haven’t had a haircut since late February, and I don’t know when I’ll let anyone point a camera at my head again…
[2] This was kind of me when I first applied for a position at Microsoft in the 90s, although I already had a few years’ experience. I was not hired.
[3] This was me in 2008 when I was hired.
[4] This is me in 2020. I tend to talk about the successes more, but it’s the failures I think about the most, and where I learned the most along the way. Success is awesome, but it’s a lousy teacher.
[5] Yes, this sucked as much as you could imagine.
At this point almost 12 years later, the problem itself is no longer relevant[1]. While digging around on an unrelated task today I found this chart, which is. You should look at it now.
Before Power BI we had PowerPoint, but data was still Power, even back then
The scope of the problem is measured by the blue series on this chart. You should look at it again. Just look at it!
Both the blue series and the yellow series are net satisfaction (NSAT) scores. There’s a lot of context behind the numbers[2], but for the purposes of this post let’s say that on this scale anything over 150 is “time for a team party and a big round of bonuses” and anything under 100 is “you probably won’t include this job on your resume, and you’re thinking about this a lot because you’ve been sending your resume out a lot this week.”
There are two stories that leap out from this chart.
The first story is pretty obvious: something changed in FY06. That change had a dramatically negative impact on the blue series, and a small (and probably acceptable) negative impact on the yellow series.
The second story may not be as obvious, but it’s vitally important: the yellow series was being used to track the impact of the change. Something changed in FY06, and the people that made the change were measuring its impact.
They were tracking the wrong thing.
Until I joined the team, no one had a chart like this. It wasn’t that the blue series wasn’t being tracked – it was. It just wasn’t recognized as the true success metric until things were well into resume-polishing territory.[3]
The lesson here isn’t that someone made a bad decision and didn’t realize it. The lesson is that sometimes the metric you’re tracking doesn’t mean what you think it means.
As is the case in my personal story, the problem is usually quite obvious in retrospect, but it’s also usually quite opaque in the moment. Although most large companies have a culture of measurement, it’s more rare to see a culture that consistently questions those measurements. Although this approach may not work for everyone, I recommend using this three-year-old approach to defining your most important metrics.
I don’t mean that the approach is three years old. I mean that you should approach the problem like a three-year-old would: by repeatedly asking “why?”
When someone[4] suggests measuring using a given metric, ask why. “Why do you think this is the right way to measure this thing?” When you get an answer, ask why again. “Why do you believe that?” Keep asking why – the more important the metric, the more times you should ask why and expect to get a well-considered answer[5]. And if the answers aren’t forthcoming or aren’t credible… that is an important point to recognize before you’ve invested too much in a project or solution, isn’t it?
[1] Which is why I’m not going to talk about the problem or the solution here, except in the most general, hand-wavey terms.
[3] I should also point out that I wasn’t the person who figured out that we’d been measuring the wrong thing. The person who hired me had figured it out, which was why I was hired. Credit where credit is due.
[4] This someone may or may not be you. But definitely question yourself in the same way, because it’s always hardest to see your own biases.
[5] The person who introduced me to this idea called it “five whys” but I wouldn’t read too much into that specific number. He also never explained what he meant by this, and for months I thought he was referring to some five word phrase where each word started with the letter Y. True story.
I live 2.6 miles (4.2 km) from the epicenter of the coronavirus outbreak in Washington state. You know, the nursing home that’s been in the news, where over 10 people have died, and dozens more are infected.[1]
As you can imagine, this has started me thinking about self-service BI.
Where can I find information I can trust?[2]When the news started to come out covering the US outbreak, there was something I immediately noticed: authoritative information was very difficult to find. Here’s a quote from that last link.
This escalation “raises our level of concern about the immediate threat of COVID-19 for certain communities,” Dr. Nancy Messonnier, director of the CDC’s National Center for Immunization and Respiratory Diseases, said in the briefing. Still, the risk to the general public not in these areas is considered to be low, she said.
That’s great, but what about the general public in these areas?
What about me and my family?
When most of what I saw on Twitter was people making jokes about Jira tickets[3], I was trying to figure out what was going on, and what I needed to do. What actions should I take to stay safe? What actions were unnecessary or unhelpful?
Before I could answer these questions, I needed to find sources of information. This was surprisingly difficult.
Specifically, I needed to find sources of information that I could trust. There was already a surge in misinformation, some of it presumably well-intentioned, and some from deliberately malicious actors. I needed to explore, validate, confirm, cross-check, act, and repeat. And I was doing this while everyone around me seemed to be treating the emerging pandemic as a joke or a curiosity.
I did this work and made my decisions because I was a highly-motivated stakeholder, while others in otherwise similar positions were farther away from the problem, and were naturally less motivated at the time.[4]
And this is what got me thinking about self-service BI.
In many organizations, self-service BI tools like Power BI will spread virally. A highly-motivated business user will find a tool, find some data, explore, iterate, refine, and repeat. They will work with untrusted – and sometimes untrustworthy – data sources to find the information they need to use, and to make the decisions they need to make. And they do it before people in similar positions are motivated enough to act.
But before long, scraping together whatever data is available isn’t enough anymore. As the number of users relying on the insights being produced increases – even if the insights are being produced by a self-service BI solution – the need for trusted data increases as well.
Where an individual might successfully use disparate unmanaged sources successfully, a population needs a trusted source of truth.
At some point a central authority needs to step up, to make available the data that can serve as that single source of truth. This is easier said than done[5], but it must be done. And this isn’t even the hard part.
The hard part is getting everyone to stop using the unofficial and untrusted sources that they’ve been using to make decisions, and to use the trusted source instead. This is difficult because these users are invested in their current sources, and believe that they are good enough. They may not be ideal, but they work, right? They got me this far, so why should I have to stop using them just because someone says so?
This brings me back to those malicious actors mentioned earlier. Why would someone deliberately share false information about public health issues when lies could potentially cost people their lives? They would do it when the lies would help forward an agenda they value more than they value other people’s lives.
In most business situations, lives aren’t at stake, but people still have their own agendas. I’ve often seen situations where the lack of a single source of truth allows stakeholders to present their own numbers, skewed to make their efforts look more successful than they actually are. Some people don’t want to have to rebuild their reports – but some people want to use falsified numbers so they can get a promotion, or a bonus, or a raise.
Regardless of the reason for using untrusted sources, their use is damaging and should be reduced and eliminated. This is true of business data and analytics, and it is true of the current global health crisis. In both arenas, let’s all be part of the solution, not part of the problem.
[1] Before you ask, yes, my family and I are healthy and well. I’ve been working from home for over a week now, which is a nice silver lining; I have a small but comfortable home office, and can avoid the obnoxious Seattle-area commute.
[2] This article is the best single source I know of. It’s not authoritative source for the subject, but it is aggregating and citing authoritative sources and presenting their information in a form closer to the solution domain than to the problem domain.
[3] This is why I’ve been practicing social media distancing.
[4] This is the where the “personal pandemic parable” part of the blog post ends. From here on it’s all about SSBI. If you’re actually curious, I erred on the side of caution and started working from home and avoiding crowds before it was recommended or mandated. I still don’t know if all of the actions I’ve taken were necessary, but I’m glad I took them and I hope you all stay safe as well.
[5] As anyone who has ever implemented a single source of truth for any non-trivial data domain can attest.
[6] You can enjoy the lyrics even if Kreator’s awesome music isn’t to your taste.
You’ve probably seen it more than once. But you’ve only seen it an order of magnitude less than I’ve thought it, because if I posted it multiple times each day I would be part of the problem. Typically when I tweet a variation on this theme, it’s because someone has been lazy, and has stolen my time, and the time of others.
Consider these scenarios.
Have you ever forwarded a lengthy email thread to a group, with “FYI” or “this is interesting” as your only addition, without adding a summary of the thread? If you have, then each person who receives your mail needs to read through the thread to understand what is important for them.
Have you ever sent an email with a meaningless and non-descriptive subject line that’s unrelated to the message content? If you have, then each person who receives your mail needs to read through the message to understand your intent and to prioritize any follow-up actions.
Have you ever sent an email that includes a document or link to a valuable resource, but you don’t include any relevant search terms in the subject or body? If you have, then when your recipients need to find and use that link or document they will not be able to easily search to locate it. You’ve forced each recipient to implement their own discovery process.
Have you ever sent an email that references a shared resource like a web site or an Excel workbook on a SharePoint site, and didn’t include a link to that resource? If you have, then each recipient has needed to manually locate the shared resource – you have wasted the time of every person who received the mail. And to make matters worse, your laziness has introduced ambiguity, and increased the likelihood that people will end up using the wrong resources.
Have you ever sent an email that includes a general description of a specific problem for which you are requesting assistance? If you have, then you are offloading the responsibility for identifying the problem cause to the recipients – and this often means that multiple people are duplicating the effort that you should have put in proactively.
Have you ever sent an email that includes an acronym that you have not explicitly defined? If you have, then you’re again forcing the recipients to do the heavy lifting to figure out what you mean, when you could have saved them this effort by putting in a little effort on your own…
Your periodic reminder that lazy communication is theft.
If you use an acronym without defining it, you are part of the problem, not part of the solution.
Have you ever sent an email related to an event – a technical conference “call for content” announcement, for example – and you haven’t bothered to include the event dates in the mail? If you have, then you have forced every recipient to look up this information before they can act on your mail.
Have you ever asked someone for help solving a technical problem or error, but you haven’t clearly articulated the scope of the problem? Maybe you couldn’t even be bothered to include key details like error messages? If this is the case, you’ve very clearly told the people who could be helping you that you do not value their time, and that you are choosing to make your problem their problem.
Of course, the impact of this laziness isn’t limited to email – email just happens to be where I personally experience it the most. My most recent[2] periodic reminder came when someone on Twitter asked for help, and included an undefined acronym. By the time I noticed the conversation, three or four members of the Power BI team had replied, either asking for clarification or proposing possible answers if the acronym meant what they thought it meant. (I did not join that conversation.)
The common theme of these scenarios – and many more like them – is that a small effort to be mindful in your communication can help reduce the cost on the people with whom you are communicating. If you choose not to put in that effort, your lazy communication is stealing time and productivity from your teammates, peers, and colleagues.
Is that what you want?[3]
Each of these bad habits is easily and simply corrected. In most situations it only takes a moment to clarify the meaning and context of your message, to add a subject, or summary, or link. A moment of your time can save many minutes wasted by every person who receives your communication.
Will you choose to spend that time, and to respect the time of others?
[2] Most recent when I started writing this post, anyway. That was back in early November. It’s taken me so long to finish and publish this post because people keep stealing my time.
Have you ever looked at a problem and immediately known how to solve it?
Have you ever seen that there’s already a group of people who are responsible for solving this problem this problem, and although they’re working on a fix they obviously have not seen the common sense solution that is so apparent to you?
I knew we could park here! Problem solved!
Yeah[1].
In situations like these, there are three[2] potential reasons for what you’re seeing:
The other people are just too stupid to to see the simple and obvious solution that’s been sitting right under their noses all this time.
The other people, due to the time that they have spent closely examining the problem and potential solutions, have realized that the simple and obvious solution doesn’t actually address the problem – even though common sense suggests that it should – or that the solution has unintended side-effects that outweigh its benefits.
The other people are responsible for additional problems and solutions across a broader or more strategic scope, and although the simple solution may make sense in isolation, implementing it has been deferred to enable work on other, higher priority problems.
(At this point I feel compelled to point out that this is something I’ve thought about for decades, and I’ve had a draft blog post on this subject for almost a year. I’m not trying to subtweet[3] anyone. I’ve had several people recently reach out to me to make sure that they’re not the reason I tweet periodic reminders that lazy communication is theft, so a disclaimer here feels appropriate.)
In my experience, many people seem to automatically gravitate to the first choice, and that bias seems exacerbated by differences in perspective. If the observer is in a business group and the solution team is in IT – or vice versa[4] – it’s easy for them to miss the complexity and nuance in something that appears simple and straightforward from the outside.
Part of this challenge is inherent in this type of relationship. It’s natural and appropriate to hide internal complexity from external stakeholders. I’m pretty sure I even learned about this in college… not that I remember college[5].
But part of it is the choice we make when we engage in the cross-team relationship. Every time we see a “simple problem” with an “obvious” or “easy” or “common sense” solution we get to decide how to react. We can assume that we know everything important and that the “other guy” is an idiot… or we can consider the option that there might be something we don’t already know.
The choice is ours.
[1] This is less of an actual footnote than a tool to enforce a dramatic pause as you read. Assuming you read footnotes as you go through the post, rather than reading them all at the end. I may need more telemetry to better understand use behavior. Why aren’t any simple problems actually as simple as they first seem?
[2] If you can think of other reasons that aren’t just variations on these three, I would love to hear about them in the comments. My list used to only include the first two reasons, but over the last few years I’ve added the third.
[3] Sub-blog? is that a thing?
[4] There are scores of other variations on this theme – I picked this one because it’s likely to be familiar to the people I imagine read my blog.
That’s an average of right around two posts per week, although my actual writing output has been much less even and predictable than this number might suggest.
After 100 posts and a little over one year, where should BI Polar go?
What are the topics you would like to see emphasized in the next year and the next hundred posts?
For the past month or so I’ve been sticking to a three-posts-per-week schedule, but I don’t know how sustainable that is. I’m thinking about switching to a Monday-Wednesday schedule, with one post each Monday to accompany the week’s new YouTube video, plus one additional post each week. This feels like a much more reasonable long-term plan.
So… what topics or themes are you interested in? What would you like to see more of, based on what you’ve seen over the last year? I can’t promise I’ll do what you want, but I can promise that I’ll read every comment, and I expect I’ll be inspired by whatever ideas you have…
BI Polar 