These AI generated superhero photos from Pure Storage were a hit. Perhaps I shouldn’t have smiled? On the other hand, even superhero me is ready to serve with a smile!
Biggest Takeaway From Day 2
Today, I got clarity on when to use Dataflows Gen 2 versus Data Factory Pipelines in Microsoft Fabric. Until now, I have held the belief that Power Query, and by extension Dataflows Gen 1 in Power BI and Dataflows Gen 2 in Fabric, was strictly a lightweight ETL tool. To a degree this thinking holds up. Power Query and Dataflows Gen 1 aren’t necessarily meant to scale to an enterprise level. Let me be clear that I LOVE Power Query. But as great as Power Query is, it really has been a citizen developer tool. Miquel Llopis and Abhishek Narain from Microsoft did a fantastic session on Data Factory in Fabric that updated my thinking. Dataflows Gen 2 in Fabric have been given supercharged to run at high scale giving us a Power Query UI experience but with enterprise level performance. In talking with Miguel after the session, he advised that Dataflows can be used to handle data transformations in partnership with Data Factory for orchestration. Having both options in Fabric allows us options to choose the best for our specific scenarios. It’s worth noting that the compute for doing transformations in a Dataflow will cost more than, say, writing the same transformations in a notebook in Data Factory. Miguel equated to it as going to the grocery and buying the ingredients to make dinner yourself verses going to a restaurant to have the same meal prepared by someone else. Sounds about like what we’d expect from Microsoft. I need to dig in more to the details on this, will post what I find.
Technical Talk
I also enjoyed Reid Havens’ session on best practices for shared Power BI datasets. Reid’s talks always have so many nuggets. I learned that it’s possible to display a DAX measure in multiple display folders. To do so, just add semicolon (;) between the folder names in the display folder property. What a useful tip when organizing measures that may fit within more than one place!
Final Thoughts
It’s so hard to believe that the conference ends tomorrow! I’m sure that my brain will be at capacity by the end of the day. I’m also sad though, this has been so much better than I could have imagined! Bring on day 3!
