BI is dead. Long live BI!

As I was riding the bus home from jury duty the other day[1] I saw this tweet come in from Eric Vogelpohl.

There’s a lot to unpack here. and I don’t expect to do it all justice in this post, but Eric’s thought-provoking tweet made me want to reply, and I knew it wouldn’t fit into 280 characters… but I can tackle some of the more important and interesting elements.

First and foremost, Eric tags me before he tags Marco, Chris, or Curbal. I am officially number one, and I will never let Marco or Chris forget it[2].

With that massive ego boost out of the way, let’s get to the BI, which is definitely dead. And also definitely not dead.

Eric’s post starts off with a bold and simple assertion: If you have the reactive/historical insights you need today, you have enough business intelligence and should focus on other things instead. I’m paraphrasing, but I believe this effectively captures the essence of his claim. Let me pick apart some of the assumptions I believe underlie this assertion.

First, this claim seems to assume that all organizations are “good w/ BI.” Although this may be true of an increasing number of mature companies, in my experience it is definitely not something that can be taken for granted. The alignment of business and technology, and the cultural changes required to initiate and maintain this alignment, are not yet ubiquitous.

Should they be? Should we be able to take for granted that in 2019 companies have all the BI they need? [3]

The second major assumption behind Eric’s first point seems to be that “good w/ BI” today translates to “good w/ BI” tomorrow… as if BI capabilities are a blanket solution rather than something scoped and constrained to a specific set of business and data domains. In reality[4], BI capabilities are developed and deployed incrementally based on priorities and constraints, and are then maintained and extended as the priorities and constraints evolve over time.

My job gives me the opportunity to work with large enterprise companies to help them succeed in their efforts related to data, business intelligence, and analytics. Many of these companies have built successful BI architectures and are reaping the benefits of their work. These companies may well be characterized as being “good w/ BI” but none of them are resting on their laurels – they are instead looking for ways to extend the scope of their BI investments, and to optimize what they have.

I don’t believe BI is going anywhere in the near future. Not only are most companies not “good w/ BI” today, the concept of being “good w/ BI” simply doesn’t make sense in the context in which BI exists. So long as business requirements and environments change over time, and so long as businesses need to understand and react, there will be a continuing need for BI. Being “good w/ BI” isn’t a meaningful concept beyond a specific point in time… and time never slows down.

If your refrigerator is stocked with what your family likes to eat, are you “good w/ food”? This may be the case today, but what about when your children become teenagers and eat more? What about when someone in the family develops food allergies? What about when one of your children goes vegan? What about when the kids go off to college? Although this analogy won’t hold up to close inspection[5] it hopefully shows how difficult it is to be “good” over the long term, even for a well-understood problem domain, when faced with easily foreseeable changes over time.

Does any of this mean that BI represents the full set of capabilities that successful organizations need? Definitely not. More and more, BI is becoming “table stakes” for businesses. Without BI it’s becoming more difficult for companies to simply survive, and BI is no longer a true differentiator that assures a competitive advantage. For that advantage, companies need to look at other ways to get value from their data, including predictive and prescriptive analytics, and the development of a data culture that empowers and encourages more people to do more things with more data in the execution of their duties.

And of course, this may well have been Eric’s point from the beginning…

 


[1] I’ve been serving on the jury for a moderately complex civil trial for most of August, and because the trial is in downtown Seattle during business hours I have been working early mornings and evenings in the office, and taking the bus to the courthouse to avoid the traffic and parking woes that plague Seattle. I am very, very tired.

[2] Please remind me to add “thought leader” to my LinkedIn profile. Also maybe something about blockchain.

[3] I’ll leave this as an exercise for the reader.

[4] At least in my reality. Your mileage may vary.

[5] Did this analogy hold up to even distant observation?

Ignorance as a finite resource (or “Data visualization resources”)

I’m not a data visualization professional, nor do I play one on TV. Even though I’ve been working with data professionally for over 20 years[1], I’ve generally been on the back end of systems, working on data platforms and not data visualization. I’ve been known to say on occasion[2] that I’ll make your data sing, but you probably don’t want me responsible for making it pretty. If I’m your report designer, building them will take five times as long, and the reports will suck ten times as much.

Before I joined Microsoft, I worked for many years as a consultant. When working with potential clients, I would often present my ignorance as an asset. This may sound strange, but think about the truth in this simple pitch: “You and your team know too much. There are some questions you will never think to ask, because they’re just too obvious. One of the strengths I will bring to this engagement is my ignorance – I will ask those questions and together we will find answers and build solutions.”

Sometimes ignorance doesn’t feel like an asset – sometimes it just gets in the way. In the past few months this has been the case as I’ve been trying to get build Power BI reports for some of the data I work with day to day. To be more specific, I’ve been trying to build reports that don’t suck. I already have functional reports, but I believe that there are insights waiting in the data that could be discovered and shared more easily if I had stronger data visualization skills. I read documentation, and followed examples and reached out to multiple friends and colleagues or advice and… and often found that not only did I not have the knowledge to effectively visualize my data, I often struggled to effectively communicate my goals. I was both mega-ignorant and meta-ignorant.

Which brings me to the point of this post: I’m hoping to use this monumental ignorance as an asset.

Earlier this week I reached out on Twitter asking for help. I wanted to find resources that could help me build my data visualization skills, and to build my vocabulary of terms and concepts so I could ask better questions. I wanted to address both the mega-ignorance and the meta-ignorance. So I asked both Alberto Cairo[3] and Alyssa Fowers[4] by name, and asked for help in general, and boy did I get it.

I learned one immediate lesson from the replies to this post: I did not do an effective job in communicating my goals. Previously I had struggled when asking for tactical data visualization help; here I was struggling when asking for strategic help, and most of the responses weren’t even close to what I was asking for[5]. This was frustrating, but it validated my understanding of the problem, and reinforced my belief that I need a deeper understanding of the concepts and nomenclature of data visualization if I’m going to improve.

Of the many resources that were shared with me, this one looks the most promising: Andy Kirk’s “Data Visualization: A Handbook for Data Driven Design.” Based on the thoughtful description[6] and glowing reviews, I think this is where I will begin. I’ve pre-ordered the second edition and when it arrives next month I’ll begin my journey…

book

…which is sort of where I was going with this whole post[7]. Ignorance as a finite resource, remember?

I’ve long observed that when one person is struggling with something, he’s typically not alone. When one person asks a question, a dozen other people breathe a silent sign of relief, because they had the same question but didn’t dare ask it. Working under the assumption that I’m not alone in this starting point, I’m hoping to use my current state of ignorance, and the upcoming process of destroying that ignorance, to start a “data visualization as a second language” series of posts.

If you’re looking for excellent tools that you can use today to build on your existing foundation of data visualization knowledge and skills, click on the Twitter link above because there are over a dozen web sites, books, and other resources – and I’m too lazy to copy them all here. But if you’re interested in joining me on my learning journey, stay tuned until August.

Do you think this type of content has an audience? Are you that audience? I’d love to hear from you


[1] How young is too young to play the “old man” card so consistently? Asking for a friend.

[2] Typically these are occasions when someone is looking at one of my ugly reports, and I’m feeling self-conscious.

[3] You may know Mr. Cairo as a leading expert and author on data visualization, as the Knight Chair at the University of Miami, or as the guy who is too busy to replace the lorem ipsum in his bio page, but I will always think of him as the other person who regularly presents on the intersection of data and heavy metal. \m/

[4] If you don’t follow Ms. Fower’s “Data and Dragons” blog you definitely should, because she already has two parts in her “Sword Graphs” series.

[5] Yes, I know this is how Twitter works. I’m trying to be generous here – work with me.

[6] Click on the “Who is this book for” link and tell me that you don’t wish every technical book had this information so readily available.

[7] Brevity will never be my forte.

[8] I was originally thinking of “Data Viz 101” but that sounded too advanced, and if I went with the second idea of  “Data Viz 001”  I know I would forever be making “James Bond’s analyst colleague” jokes. DVSL wasn’t my first choice, but I think I like it.

Writing effective problem reports

If you build software or data solutions[1] you have probably encountered one or both of these situations:

  1. You’re trying to report a bug, but the developer doesn’t believe that there’s a problem.
  2. Someone is trying to report a bug to you, and you can’t tell what the problem is supposed to be.

The problem report has itself become a problem[2].

Fortunately, there’s a simple approach, and simple template, that can make reporting problems easier. This is the template I typically use when I’m reporting a problem and asking someone else to fix it:

Problem: Concise description of problem behavior

Steps to reproduce problem:

  1. First step
  2. Second step
  3. Third and subsequent steps, as necessary

Desired or expected behavior:

4. What I wanted to happen

Observed behavior:

4. What actually happened, including the full details of any error messages

That’s it – simple and easy.

It can also often be helpful to include screen shots, recordings, or other visual resources to supplement the text descriptions. If you use TechSmith Camtasia (commercial, paid) or ShareX (open source, free) or other screen recording software, it can be trivial to record and attach a video – but remember that the video does not replace the written problem report, it supplements it.

I should mention that if you’re a software developer working on a software development team, you probably have a heavier-weight process already[3]. Follow that process. This approach is intended for more casual problem reporting – the sort of thing that you might send to someone in an email asking for help. The sort of email where if you don’t communicate clearly and effectively, the recipient ends up spending more time asking for information than he spends actually answering the question or solving the problem.

Yes, this is why I wrote this post. I hope I never need to link to it…


[1] Or work in a technical field, or use software…

[2] Because I never metaproblem I didn’t like. Yes, this sounded funnier in my head.

[3] If you search for “how to write good bugs” you’ll find a huge number of excellent resources that go into much more depth than this post.

Managing email and work-life balance

I’ll probably never be the most consistent blogger, but WordPress recently made me aware of something: I only blog regularly when I’m taking time off from work.

Streak

I’m back in the office today after a week of part-time work from home[1] and I realized that without making a conscious to do so I ended up blogging every day that I was away from the office.

This insight got me thinking. Specifically, it got me thinking about how I spend my work days, and about how in recent months[2] I’ve been letting my inbox push me around. Although playing a defensive game can work in some contexts[3], I believe I need to adopt a more aggressive posture in this fight.

Starting today I’m trying this approach to Inbox Zero from MVP Luise Freese. I’m hoping that by managing my email more proactively and strategically I can not only be more productive at work, but also have more mental energy and time remaining for blogging.

My teammate Adam Saxton is doing the same thing; he’s a few days ahead of me and is pleased with his progress so far. I’ll check in with him – and with you – next week to see how things are going. Now back to Outlook; I have more items marked for follow-up today…


[1] Part time work from home and full-time feeling old: My older son graduated from high school last week. Where did the years go?

[2] Years.

[3] Please don’t tell Johannes Liechtenauer I said this.

What is Your Gift?

I have a theory that everyone has a gift. Not a gift that they have been given, but a gift that they can give to others.

blue-box-container-675970.jpg

Your gift is something you’re good at, and which enables you to expend minimal effort to achieve disproportionally large benefit in return.

If you’re an IT professional, you’ve probably encountered situations where you could spend a few minutes configuring a loved one’s hardware or software. For you, this was a trivial task, done in minutes or seconds without any real effort, but for your father or aunt it would have been a stressful ordeal, with no guarantee of success.

Regardless of the context, there’s something that you’re better at. Cooking, baking, drawing, painting, photography, singing, rhyming, writing, carpentry, plumbing… something. As you read this list, hopefully something leaped to mind. For me, this list include a few things where I have serious skills, and a bunch of things where I feel hopelessly challenged[1].

Once you recognize and acknowledge your gift, you just need to keep your eyes open for opportunities to give it. Each day, look for the places where you could invest a few minutes to spare someone a few hours. Look for the chance to invest an hour to save someone days or weeks. And once you identify the opportunities, choose which of them to act upon. It’s not your responsibility to solve every problem you can, but your gift is only valuable when you share it.

What is your gift?

Who will you share it with?


[1] One day I’ll encounter a situation where someone desperately needs a skilled swordsman. But it is not likely to be this day.

Thoughts on Expertise, Ignorance, Chocolate, and Bacon

In late September I had two very different experiences that I’ve found myself thinking about a lot since then. One involved chocolate, and the other involved bacon, and both involved ignorance.

My wife and I were taking a bacon making class one Sunday evening. I’d tried making homemade bacon before, but the results were disappointing, and I wanted some expert instruction. I’d been looking forward to the class for months.

While we were in the area, we decided to stop by Dawn’s Candy and Cake to inquire about tempered chocolate. I’ve been making chocolate treats for decades, and have tempered chocolate many times, but always found it to be a finicky pain in the butt. Dawn’s offers chocolate tempering classes, and I was hoping that there would be some way I could pop in to use chocolate that an employee had tempered to use with truffles I had made myself. This would let me focus on flavors, and let someone else worry about the annoying parts.

Sadly it was not meant to be. Although the young woman who was working that evening was very helpful, all of the tempering classes in the time-frame I needed were completely booked. But all was not lost. This amazing 17-year-old[1] stepped up to save the day. She printed out the tempering instructions they hand out during classes, and walked me through them in detail. They were pretty much what I’d done in the past, but with enough differences that i was happy to receive them. It was obvious she’s done this before; she explained the processes and concepts clearly and concisely…

So I asked her some follow-up questions, abut selecting chocolate with the right viscosity[2] for different purposes. She said she’d never even heard of viscosity, and wasn’t sure what to tell me.

When we got to the bacon class, we learned that the instructions we would be using were the same instructions I had followed in my earlier failed attempt. This was disappointing, but the middle-aged man teaching the class was very forward in letting us know how he’d made bacon scores of times, and how everyone who had tried his bacon was spoiled forever on the store-bought stuff. He let us know very explicitly that he was an expert…

And as the class progressed, he proceeded to read the copied-from-the-internet directions, evade every substantive question, and inform the students that the things they were concerned and asking about weren’t important for unexplained reasons, all while continuing to tout his expertise. The class itself turned into something of an ordeal, as the instructor used every trick in the bulshitting-to-cover-up-your-ignorance-in-front-of-the-class book.[3]

Sigh.

The two experiences came back to back, and they could not have been more different. One instructor was confident enough to admit that she didn’t know, and the other was not. One instructor let his ego get in the way of his students’ learning, and the other didn’t let her ego get involved at all. One was focused on helping, while one was focused on not looking like an idiot. One succeeded, and the other one failed.

I’m not going to dwell on the fact that one of these people was young, and the other was middle-aged, or the fact that one was female and the other was male. While these factors may be significant, I know enough exceptions to the stereotypes[4] that I’m not going to go there. I’d rather focus on the behaviors themselves.

I strongly believe that the willingness to express vulnerability or weakness – including ignorance – only comes with confidence.

When you’re not confident in yourself, if you don’t know your strengths and weaknesses, and if you haven’t made some sort of peace with them, it’s incredibly difficult to say “I don’t know,” especially if you’re in a position of authority.  Some people choose to deny gaps in their knowledge, trying to cover them with bluster and obfuscation. Some may get away with it some of the time, but it’s never a long-term strategy for success. This behavior eventually builds a reputation as an unreliable blowhard.

People who admit their ignorance accomplish two important things. First, they give themselves the opportunity to learn something. Second, they encourage trust and confidence in the people around them[5]. When you show that you’re able and willing to express ignorance, this adds weight and confidence to your words when you’re expressing knowledge and expertise.

Anyway… where was I going with this? What’s the moral of this story?

I think the moral of the story is don’t be that guy.

Following my two culinary learning experiences, I have returned to Dawn’s Candy and Cake on at least three occasions, and have recommended their training events and supplies to multiple people. I’ll never take another class at the other place. And after talking to other participants after the class, I know I’m not alone in that decision.


[1] I know she was 17 because when my wife and I were talking about my upcoming tattoo appointment, she talked excitedly about how she wanted to get a tattoo when she turned 18 in a few months. Yes, I felt as old as I’ve felt in ages.

[2] In case you’re interested, this is a decent overview: https://www.reference.com/food/viscosity-important-candy-making-459c24a3ca10028b

[3] I started my IT career as a trainer, and spent years in the classroom. I could write this book.

[4] And I try to be exceptional, every day.

[5] If the people around you are the type of people who would ridicule you for admitting you don’t know something, you have also benefited from learning that you should be surrounded by different, better people.

Prepare for the BI Power Hour at PASS Summit

If you follow Power BI on Twitter, you may have noticed the #BIPowerHour tag over the last few weeks. For those who have been part of the Microsoft data community for a while, you probably know what this is all about, but I’ve seen quite a few people asking “What is the BI Power Hour?” This post is for them.

The BI Power Hour is a conference session that shows Microsoft data and business intelligence tools, products, and services being used for purposes other than that for which they were built. I’ve heard people describe it as “stupid BI tricks”[1], and as smart as some of the tricks have been, I can’t argue with that description. The session has been part of multiple conferences for over a decade, with some of the best technical presenters around[2] showing off humorous and thought-provoking[3] demos.

Some of my personal favorite BI Power Hour demos over the years include:

  • Building an online dating app using SQL Server Integration Services
  • Using SQL Server Integration Services, Master Data Services, and Data Quality services to respond to all of the social opportunities that came from the previous year’s dating app
  • Data mining the Titanic passenger list using the data mining add-in for Excel
  • Data mining the Oracle price list using the data mining add-in for Excel[4]
  • Building a multiplayer version of Battleship using SQL Server Reporting Services
  • Predicting the end of the world with Power BI[5]
  • Using Power BI and a database of heavy metal bands to demonstrate how Power BI got its name

The list goes on and on – there are too many highlights to call out here. Each year has been different, but each year has also been delightful.

This year we’re back, and we have some “pro tips” for those of you who are attending the PASS Summit next week in Seattle.

  1. Show up early. The last time we had a BI Power Hour at PASS in Seattle, the room was 100% full, with scores of people turned away.
  2. Sit near the front, even if you don’t sit in the front row. We’ll be throwing swag, and much of it is not aerodynamic, so sitting all the way in the back reduces the odds of your getting a t-shirt or other goodies.
  3. Sit next to someone who is a different size than you. This one may sound odd, but if the person next to you catches a t-shirt that doesn’t fit, your odds just increased. Am I right?
  4. Don’t expect to learn anything useful. We’ll mention this one again during the session intro, but it’s important to know in advance that we have explicitly defined real-world skills as Out Of Scope for this session. the BI Power Hour is all about having fun at the end of a long week – if you want to learn something, pick a different session.
  5. Use the #BIPowerHour hashtag everywhere you can!! Because no real-world skills are taught, and no useful topics are covered, some people[6] seem to think that the session isn’t a priority. If you disagree, please share and tweet before, during, and after the session, to let the powers that be understand your opinion.

Do you have any tips you would add to this list?

If you do, please let me know!

I’ll wrap up by casually reminding you that the BI Power Hour put “Power” and “BI” together many years before they ever appeared in an official, marketing-blessed name. I’m not saying that this is the session where the future gets determined, but…

 


[1] For those of you who don’t get the reference, I think this was the inspiration for this label: https://www.youtube.com/watch?v=xu2A8OK4vJg

[2] I have also been involved on occasion

[3] Typically the thoughts it provokes are something along the lines of “who let these people on stage”?

[4] I still can’t believe they did this one

[5] 2012 was such a simpler time, when we joked about the end of the world…

[6] Typically people who are responsible for choosing conference sessions