Is self-service business intelligence a two-edged sword?

I post about Power BI dataflows a lot, but that’s mainly because I love them. My background in data preparation and ETL, combined with dataflows’ general awesomeness makes them a natural fit for my blog. This means that people often think of me as “the dataflows guy” even though dataflows are actually a small part of my role on the Power BI CAT team. Most of what I do at work is help large enterprise customers successfully adopt Power BI, and to help make Power BI a better tool for their scenarios[1].

As part of my ongoing conversations with senior stakeholders from these large global companies, I’ve noticed an interesting trend emerging: customers describing self-service BI as a two-edged sword. This trend is interesting for two main reasons:

  1. It’s a work conversation involving swords
  2. Someone other than me is bringing swords into the work conversation[2]

As someone who has extensive experience with both self-service BI and with two-edged swords, I found myself thinking about these comments more and more – and the more I reflected, the more I believed this simile holds up, but not necessarily in the way you might suspect.

This week in London I delivered a new presentation for the London Power BI User Group – Lessons from the Enterprise: Managed Self-Service BI at Global Scale. In this hour-long presentation I explored the relationship between self-service BI and two-edged swords, and encouraged my audience to consider the following points[4]:

  • The two sharp edges of a sword each serve distinct and complementary purposes.
  • A competent swordsperson knows how and when to use each, and how to use them effectively in combination.
  • Having two sharp edges is only dangerous to the wielder if they are ignorant of their tool.
  • A BI tool like Power BI, which can be used for both “pro” IT-driven BI and self-service business-driven BI has the same characteristics, and to use it successfully at scale an organization needs to understand its capabilities and know how to use both “edges” effectively in combination.

As you can imagine, there’s more to it than this, so you should probably watch the session recording.

ssbi and swords

If you’re interested in the slides, please download them here: London PUG – 2019-06-03 – Lessons from the Enterprise.

If you interested in the videos shown during the presentation, they’re included in the PowerPoint slides, and you can view them on YouTube here:

For those who are coming to the Microsoft Business Applications Summit next week, please consider joining the CAT team’s “Enterprise business intelligence with Power BI” full-day pre-conference session on Sunday. Much of the day will be deep technical content, but we’ll be wrapping up with a revised and refined version of this content, with a focus on building a center of excellence and a culture of data in your organization.

Update 2019-06-10: The slides from the MBAS pre-conference session can be found here: PRE08 – Enterprise business intelligence with Power BI – Building a CoE.

There is also a video of the final demo where Adam Saxton joined me to illustrate how business and IT can work together to effectively respond to unexpected challenges. If you ever wondered what trust looks like in a professional[5] environment, you definitely want to watch this video.

 


[1] This may be even more exciting for me than Power BI dataflows are, but it’s not as obvious how to share this in blog-sized pieces.

[2] Without this second point, it probably wouldn’t be noteworthy. I have a tendency to bring up swords more often in work conversations than you might expect[3].

[3] And if you’ve been paying attention for very long, you’ll probably expect this to come up pretty often.

[4] Pun intended. Obviously.

[5] For a given value of “professional.”

Unlimited dataflow refresh on Power BI Premium

Last month Microsoft announced on the Power BI blog an exciting new capability:

AUTOMATION & LIFE-CYCLE MANAGEMENT

‘Refresh Now’ API provides unlimited data refresh for Power BI Embedded and Power BI Premium

Using the ‘Refresh now’ API, the limitation  on the number of refreshes you can schedule per day is removed and instead  an unlimited number of refreshes can be triggered for each dataset. Combining the refresh now API with incremental refresh, you can build a near real-time dataset that performs small updates of fresh data very often.

Note: The time of existing refresh is not expected to be shorter, so a new refresh of a dataset cannot start before the previous one finishes. Remember that your resource limitations do not change with the introduction of this API, so use these unlimited refreshes with caution and be careful not to overload your resources with unnecessary refreshes.

Although the blog post only explicitly mentions datasets, the same “as many refreshes as you want” capability applies to Power BI dataflows in workspaces assigned to dedicated (Power BI Embedded or Power BI Premium) capacity.

It’s important to note that this is an API-only feature[1]. If you’re setting up a refresh schedule via the UI, you’ll still see the same daily limits, but using the dataflows API you will now be able to have full control over the refresh schedule for your dataflows.


[1] This is by design, and is unlikely to change. A high-frequency refresh schedule can place a significant load on the capacity resources, and is a configuration that should only be made after careful consideration of the implications.

Session resources: Power BI dataflows and Azure Data Lake integration

Last week I delivered two online sessions on the topic of integrating Power BI dataflows with an organizational Azure Data Lake Storage Gen2 storage account. I’ve blogged on this topic before (link | link | link | link) but sometimes a presentation and demo is worth a thousand words.

On April 30th I delivered a “Power BI dataflows 201 – beyond the basics” session for the PASS Business Intelligence Virtual Chapter. The session recording is online here, and you can download the slides here.

On May 4th I delivered a “Integrating Power BI and Azure Data Lake with dataflows and CDM Folders” session for the SQL Saturday community event in Stockholm, Sweden. I was originally planning to deliver the Stockholm session in person, but due to circumstances beyond my control[1] I ended up presenting remotely, which meant that I could more easily record the session. The session recording is online here, and you can download the slides here.

Each of these sessions covers much of the same material. The Stockholm presentation got off to a bit rocky start[2] but it gets smoother after the first few minutes.

Please feel free to use these slides for your own presentations if they’re useful. And please let me know if you have any questions!


[1] I forgot to book flights. Seriously, I thought I had booked flights in February when I committed to speaking, and by the time I realized that I had not booked them, they were way out of my budget. This was not my finest hour.

[2] The presentation was scheduled to start at 6:00 AM, so I got up at 4:00 and came into the office to review and prepare. Instead I spent the 90 minutes before the session start time fighting with PC issues and got everything working less than a minute before 6:00. I can’t remember ever coming in quite this hot…

General availability of Power BI dataflows

Power BI dataflows have been available in public preview since November 2018. For almost five months, customers around the world have been kicking the tires, testing and providing feedback, and building production capabilities using dataflows.

When Microsoft published the latest Business Applications Release Notes, the “new and planned features” list included dataflows general availability with a target date of April 2019, which could typically mean anything before May 1st.

But… April has just arrived, and so has the dataflows GA!

The full details are on the official Power BI blog, so be sure to check it out. Also keep in mind that although dataflows are now generally available, some specific capabilities are still in preview.

 

Free Data Modeling Course

In any industry there is a small set of luminaries – people who everyone knows by name and by reputation. They’ve been around for years and decades. They started off doing amazing work, and they’ve only gotten better over time.

In the world of Microsoft business intelligence, Marco Russo and Alberto Ferrari are two of these luminaries[1]. I’ve been doing this data thing for a while now, and they know more about data and BI than I will ever forget. Or something like that.

In any event, they have just released a free introductory data modeling course, and you should complete it. I know I will.

For context, there are three big reasons why I’m going to complete an online introductory data modeling course even though I’ve been working as a data professional for over 20 years:

  1. Marco and Alberto are experienced trainers and presenters who know how to communicate complex topics in ways that make them easy to understand and apply. Even if I know a given topic already, I will learn details I did not previously know, or will learn new ways to think about the topic.
  2. There’s a lot I need to learn. My background heavily emphasizes ETL and data warehousing, which was a fine place to specialize when I was part of a multi-disciplinary team and someone else was responsible for the analytics model. When working in Power BI as a self-service user, I frequently run into gaps in my knowledge. Although I know where to look to find answers, focusing on proactive learning will make me more efficient every day I work in Power BI.
  3. Data modeling is a topic of vital importance for enterprise Power BI customers. In my day job[2] I see lots of problems that would not have existed if people had started with a well designed data model.
  4. I’m not getting any younger, so if I’m ever going to start a modeling career, it had better be soon.

I hope you’ll take advantage of Marco and Alberto’s generous offer and complete this course. When you do, let me know what you think.


[1] They are not, as the post title and their names might suggest, elite fashion designers.

[2] Although the dataflows-centric subject matter of this blog may lead you to believe otherwise, most of what I do on a daily basis is help large customers succeed with Power BI… I blog about dataflows just because I love them so much.

My slides from SQL Saturday Victoria

Last weekend I had the pleasure of presenting at SQL Saturday in Victoria, BC. I delivered an introductory session on Power BI dataflows, and included an unplanned impromptu musical[1] performance.

This week at the Microsoft MVP Summit in Redmond I have been talking to a lot of MVPs and I realized that I had yet to make any of my presentation resources available. The slide deck below (click on the image or the link below it) is my standard dataflows slide deck, with the SQL Saturday template applied.

Slide deck

SQLSatVictoria – Introduction to Power BI Dataflows

This deck includes more content than is appropriate for a single session, but it has resources that you can use if you want to present on dataflows at a conference or user group meeting. Please feel free to take what’s there and to use the parts that are valuable. Please also feel free to ping me on Twitter if you have any questions or feedback on the content or the content flow.

If you have any critical feedback to share related to my musical performance, please include a link to your Power BI themed sings as reference.


[1] For a given value of “musical”.

Upcoming Dataflows Presentations

I’m not dead!

After having a prolific first few months with this blog, the year-end holidays disrupted my routine, and I’ve been struggling to get everything balanced again. 2019 has been great so far, but its also been stupid crazy busy, and while blogging has been on my backlog, it just hasn’t made the bar for implementation.

Until now, I guess…

Last week I was in Darmstadt, Germany for the SQL Server Konferenz 2019 event, where I presented on Power BI dataflows. My session was well-attended and well-received[1] but I realized that I’d never actually told anyone about it. Now it’s time to correct this oversight for some upcoming events. These are the public events where I’ll be speaking over the next few months:

Event: SQL Saturday #826

Location: Victoria, BC, Canada

Date: March 16, 2019

Session: Introduction to Power BI dataflows

Event: Intelligent Cloud Conference

Location: Copenhagen, Denmark

Date: April 9, 2019

Session: Integrating Power BI and Azure Data Lake with dataflows and CDM Folders

Pro tip: If you’re attending Intelligent Cloud, be sure to attend Wolfgang Strasser‘s “Let your data flow” session earlier in the day. This session will provide a great introduction to Power BI dataflows and will provide the prerequisite knowledge necessary to get the most out of my session.

Event: SQL Saturday #851

Location: Stockholm, Sweden

Date: May 4, 2019

Session: Hopefully two dataflows sessions[2], details still being ironed out.

 


[1] Except for that one guy who rated the session a 2. I’d love to know what I could have done to improve the presentation and demos

[2] Also, swords.