Month: May 2019

Unlimited dataflow refresh on Power BI Premium

On By Matthew RocheIn Dataflows, Power BI2 Comments

Important: This post was written and published in 2019, and the content below may no longer represent the current capabilities of Power BI. Please consider this post to be an historical record and not a technical resource. All content on this site is the personal output of the author and not an official resource from Microsoft.

Last month Microsoft announced on the Power BI blog an exciting new capability:

AUTOMATION & LIFE-CYCLE MANAGEMENT

‘Refresh Now’ API provides unlimited data refresh for Power BI Embedded and Power BI Premium

Using the ‘Refresh now’ API, the limitation  on the number of refreshes you can schedule per day is removed and instead  an unlimited number of refreshes can be triggered for each dataset. Combining the refresh now API with incremental refresh, you can build a near real-time dataset that performs small updates of fresh data very often.

Note: The time of existing refresh is not expected to be shorter, so a new refresh of a dataset cannot start before the previous one finishes. Remember that your resource limitations do not change with the introduction of this API, so use these unlimited refreshes with caution and be careful not to overload your resources with unnecessary refreshes.

Although the blog post only explicitly mentions datasets, the same “as many refreshes as you want” capability applies to Power BI dataflows in workspaces assigned to dedicated (Power BI Embedded or Power BI Premium) capacity.

It’s important to note that this is an API-only feature[1]. If you’re setting up a refresh schedule via the UI, you’ll still see the same daily limits, but using the dataflows API you will now be able to have full control over the refresh schedule for your dataflows.

[1] This is by design, and is unlikely to change. A high-frequency refresh schedule can place a significant load on the capacity resources, and is a configuration that should only be made after careful consideration of the implications.

Session resources: Power BI dataflows and Azure Data Lake integration

On By Matthew RocheIn Azure, Dataflows, Power BI, Presentations, Video13 Comments

Important: This post was written and published in 2019, and the content below may no longer represent the current capabilities of Power BI. Please consider this post to be an historical record and not a technical resource. All content on this site is the personal output of the author and not an official resource from Microsoft.

Last week I delivered two online sessions on the topic of integrating Power BI dataflows with an organizational Azure Data Lake Storage Gen2 storage account. I’ve blogged on this topic before (link | link | link | link) but sometimes a presentation and demo is worth a thousand words.

On April 30th I delivered a “Power BI dataflows 201 – beyond the basics” session for the PASS Business Intelligence Virtual Chapter. The session recording is online here, and you can download the slides here.

On May 4th I delivered a “Integrating Power BI and Azure Data Lake with dataflows and CDM Folders” session for the SQL Saturday community event in Stockholm, Sweden. I was originally planning to deliver the Stockholm session in person, but due to circumstances beyond my control[1] I ended up presenting remotely, which meant that I could more easily record the session. The session recording is online here, and you can download the slides here.

Each of these sessions covers much of the same material. The Stockholm presentation got off to a bit rocky start[2] but it gets smoother after the first few minutes.

Please feel free to use these slides for your own presentations if they’re useful. And please let me know if you have any questions!

Update: I delivered an updated version of this session on September 27, at the Power Platform World Tour in Vancouver, BC. There is no recording of this session, but the updated slides can be downloaded here.

[1] I forgot to book flights. Seriously, I thought I had booked flights in February when I committed to speaking, and by the time I realized that I had not booked them, they were way out of my budget. This was not my finest hour.

[2] The presentation was scheduled to start at 6:00 AM, so I got up at 4:00 and came into the office to review and prepare. Instead I spent the 90 minutes before the session start time fighting with PC issues and got everything working less than a minute before 6:00. I can’t remember ever coming in quite this hot…